E-postama gönder SÜRÜ ZEKASI YÖNTEMLERİYLE AŞIRI ÖĞRENME MAKİNESİ’NİN ÖĞRENME PARAMETRELERİ OPTİMİZASYONU
Öz
SÜRÜ ZEKASI YÖNTEMLERİYLE AŞIRI ÖĞRENME MAKİNESİ’NİN ÖĞRENME PARAMETRELERİ OPTİMİZASYONU
Özet
Sinir ağları algoritmalarından olan Aşırı Öğrenme Makinesi (AÖM)’de giriş ağırlığı ve gizli eşik değeri parametrelerinin rastgele seçilmekte ve çıktı katman ağırlıkları analitik olarak hesaplanmaktadır. Bundan dolayı ağın öğrenme işlemi hızlı bir şekilde gerçekleşmektedir. Ayrıca AÖM’nin gradyan temelli algoritmalara göre gizli katmanda ihtiyaç duyduğu nöron sayısı daha fazla olmaktadır. Bu nedenle giriş ağırlıkları ve gizli nöron eşik değerlerinin optimum değerlerinin bulunması AÖM'nin performansına etki etmektedir. Bu çalışmada bu optimum değerlerin belirlenmesinde sürü zekası algoritmalarından Parçacık Sürü Optimizasyonu (PSO) ve Rekabetçi Sürü İyileştirici (RSİ) kullanılmıştır. Optimum giriş ağırlıkları ve gizli eşik değerlerinin belirlenerek çıkış ağırlıkları Moore-Penrose genelleştirilmiş tersiyle analitik olarak hesaplanmıştır. AÖM, RSİ-AÖM ve PSO-AÖM modellerinin çok sınıflı tiroit veri setine uyarlanarak öğrenme parametrelerinin optimizasyonu ile en iyi doğruluk oranları sırasıyla %94.74, %94.86, %95.42 olarak elde edilmiştir. Optimizasyon metotlarının AÖM modellerinin sınıflandırma performansını artırdığı görülmüştür.
Anahtar Kelimeler: Aşırı Öğrenme Makinesi (AÖM), Metasezgisel, Parçacık Sürü Optimizasyonu (PSO), Rekabetçi Sürü İyileştirici (RSİ)
OPTIMIZATION OF LEARNING PARAMETERS OF EXTREME LEARNING MACHINE WITH SWARM INTELLIGENCE METHODS
Abstract
In the Extreme Learning Machine (ELM), which is one of the neural networks algorithms, the input weight and hidden bias value parameters are randomly selected and the output layer weights are calculated analytically. Therefore, the learning process of the network takes place quickly. In addition, the number of neurons needed by the hidden layer is higher than the gradient-based algorithms. Finding optimum values of entry weights and hidden neuron bias values affects the performance of the ELM. In this study, Particle Swarm Optimization (PSO) and Competitive Swarm Optimizer (CSO) were used to determine these optimum values. By determining the optimum input weights and hidden bias values, the output weights were analytically calculated by Moore-Penrose generalized inverse. By adapting the multi-class thyroid data set of ELM, CSO-ELM and PSO-ELM models, the best accuracy rates were obtained as 94.74%, 94.86%, 95.42% respectively. It has been seen that optimization methods increase the classification performance of the ELM models.
Keywords: Extreme Learning Machine (ELM), Metaheuristic, Particle Swarm Optimization (PSO), Competitive Swarm Optimizer (CSO)
Anahtar Kelimeler
Tam Metin:
PDFReferanslar
K. Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural Networks, 1991, doi: 10.1016/0893-6080(91)90009-T.
Y. Ito, “Approximation of continuous functions on Rd by linear combinations of shifted rotations of a sigmoid function with and without scaling,” Neural Networks, 1992, doi: 10.1016/S0893-6080(05)80009-7.
S. Ding, X. Xu, and R. Nie, “Extreme learning machine and its applications,” Neural Computing and Applications, vol. 25, no. 3–4. Springer, pp. 549–556, 12-Dec-2014, doi: 10.1007/s00521-013-1522-8.
W. Sun, C. Wang, and C. Zhang, “Factor analysis and forecasting of CO2 emissions in Hebei, using extreme learning machine based on particle swarm optimization,” J. Clean. Prod., vol. 162, pp. 1095–1101, Sep. 2017, doi: 10.1016/j.jclepro.2017.06.016.
G.-B. Bin Huang et al., “Extreme learning Machine: Theory and Applications,” Neurocomputing, vol. 70, pp. 489–501, 2006, doi: 10.1016/j.neucom.2005.12.126.
C.-U. Yeom and K.-C. Kwak, “Short-Term Electricity-Load Forecasting Using a TSK-Based Extreme Learning Machine with Knowledge Representation,” Energies, vol. 10, no. 10, p. 1613, Oct. 2017, doi: 10.3390/en10101613.
W. Deng, Q. Zheng, and L. Chen, “Real-Time Collaborative Filtering Using Extreme Learning Machine,” in 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, 2009, pp. 466–473, doi: 10.1109/WI-IAT.2009.80.
G. Bin Huang, L. Chen, and C. K. Siew, “Universal approximation using incremental constructive feedforward networks with random hidden nodes,” IEEE Trans. Neural Networks, vol. 17, no. 4, pp. 879–892, Jul. 2006, doi: 10.1109/TNN.2006.875977.
Q. Y. Zhu, A. K. Qin, P. N. Suganthan, and G. Bin Huang, “Evolutionary extreme learning machine,” Pattern Recognit., vol. 38, no. 10, pp. 1759–1763, Oct. 2005, doi: 10.1016/j.patcog.2005.03.028.
Y. Miche, A. Sorjamaa, P. Bas, O. Simula, C. Jutten, and A. Lendasse, “OP-ELM: Optimally pruned extreme learning machine,” IEEE Trans. Neural Networks, vol. 21, no. 1, pp. 158–162, Jan. 2010, doi: 10.1109/TNN.2009.2036259.
B. Subudhi and D. Jena, “Differential evolution and levenberg marquardt trained neural network scheme for nonlinear system identification,” Neural Process. Lett., 2008, doi: 10.1007/s11063-008-9077-x.
J. Cao, Z. Lin, and G. Bin Huang, “Self-adaptive evolutionary extreme learning machine,” Neural Process. Lett., vol. 36, no. 3, pp. 285–305, Dec. 2012, doi: 10.1007/s11063-012-9236-y.
N. Zeng, H. Zhang, W. Liu, J. Liang, and F. E. Alsaadi, “A switching delayed PSO optimized extreme learning machine for short-term load forecasting,” Neurocomputing, 2017, doi: 10.1016/j.neucom.2017.01.090.
J. Kennedy and R. Eberhart, “Particle swarm optimization,” in IEEE International Conference on Neural Networks - Conference Proceedings, 1995, vol. 4, pp. 1942–1948.
R. Hassan, B. Cohanim, O. De Weck, and G. Venter, “A comparison of particle swarm optimization and the genetic algorithm,” in Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, 2005, doi: 10.2514/6.2005-1897.
R. Cheng and Y. Jin, “A competitive swarm optimizer for large scale optimization,” IEEE Trans. Cybern., vol. 45, no. 2, pp. 191–204, Feb. 2015, doi: 10.1109/TCYB.2014.2322602.
S. Gu, R. Cheng, and Y. Jin, “Feature selection for high-dimensional classification using a competitive swarm optimizer,” Soft Comput., vol. 22, no. 3, pp. 811–822, Feb. 2018, doi: 10.1007/s00500-016-2385-6.
Quinlan R, 1987, Thyroid Disease Data Set [online].
https://archive.ics.uci.edu/ml/datasets/Thyroid+Disease [Ziyaret Tarihi: 21 Mayıs 2020].
T. Matias, R. Araújo, C. H. Antunes, and D. Gabriel, “Genetically optimized extreme learning machine,” in IEEE International Conference on Emerging Technologies and Factory Automation, ETFA, 2013, doi: 10.1109/ETFA.2013.6647975.
J. Sánchez-Monedero, C. Hervas-Martinez, P. A. Gutiérrez, M. C. Ruz, M. C. R. Moreno, and M. Cruz-Ramirez, “Evaluating the performance of evolutionary extreme learning machines by a combination of sensitivity and accuracy measures,” Neural Netw. World, vol. 20, no. 7, p. 899, 2010.
M. Eshtay, H. Faris, and N. Obeid, “Improving Extreme Learning Machine by Competitive Swarm Optimization and its application for medical diagnosis problems,” Expert Syst. Appl., vol. 104, pp. 134–152, Aug. 2018, doi: 10.1016/j.eswa.2018.03.024.
G. A. Yüksel Yurtay and N. Z. Bacınoğlu, “Tıbbi Karar Destek Sistemlerinin Yöntemsel Olarak Değerlendirilmesi Üzerine Bir Çalışma,” Int. Symp. Innov. Technol. Eng. Sci.
R. Berglund and S. Belciug, “Improving extreme learning machine performance using ant colony optimization feature selection. Application to automated medical diagnosis,” Ann. Univ. Craiova, Math. Comput. Sci. Ser., 2018.
A. A. Abusnaina, S. Ahmad, R. Jarrar, and M. Mafarja, “Training neural networks using Salp Swarm Algorithm for pattern classification,” in ACM International Conference Proceeding Series, 2018, doi: 10.1145/3231053.3231070.
Madde Ölçümleri
Metrics powered by PLOS ALM
Refback'ler
- Şu halde refbacks yoktur.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Selçuk-Teknik Dergisi ISSN:1302-6178