A Detailed Analysis on Extreme Learning Machine and Novel Approaches Based on ELM
[1]
Ömer Faruk Ertuğrul, Dept. Electrical and Electronic Engineering, Batman University, Batman, Turkey.
[2]
Yılmaz Kaya, Dept. Computer Engineering, Siirt University, Siirt, Turkey.
Extreme learning machine (ELM) is a train method for single hidden layer feed forward neural network. The input weights and biases of ELM are selected randomly and output weights are determined analytically therefore ELM has a fast train stage. Unfortunately randomly selection of input weights and biases causes unstable accurate results. Accuracy of randomly selected input weights and biases (ELM) was compared with new proposed approaches: predefined input weights and biases (ELM-P) and determining input weights and biases by back propagation (ELM-B). Also novel approaches; single layer ELM (sELM), tuning ELM (tELM), ELM based on linear regression (ELMr) were proposed for determining the output weights instead of using Moore–Penrose generalized inverse method. The accuracies of proposed approaches were compared with each other and ELM. The results were showed that the proposed approaches are successful.
Extreme Learning Machine, Backpropagation, Single Layer, Tuning, Linear Regression, Stability, Predefined Weights
[1]
C. Bishop, “Neural Networks for Pattern Recognition”, Oxford: University Press, 1995
[2]
L. Fausett, L, “Fundamentals of Neural Networks”, New York: Prentice Hall, 1994
[3]
G.-B. Huang, Q.-Y. Zhu and C.-K. Siew, “Extreme learning machine: Theory and applications”, Neurocomputing 70 (2006) 489–501
[4]
G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks”, Neural Networks, 2004. 985 - 990 vol.2
[5]
W. Zong, G.-B. Huang, “Face recognition based on extreme learning machine”, Neurocomputing 74 (2011) 2541–2551
[6]
F. Cao, B. Liu and D. S. Park, “Image classification based on effective extreme learning machine”, Neurocomputing 102 (2013) 90–97
[7]
V. Malathi, N.S. Marimuthu and S. Baskar, “Intelligent approaches using support vector machine and extreme learning machine for transmission line protection”, Neurocomputing 73 (2010) 2160–2167
[8]
F. Benoit, M. v. Heeswijk, Y. Miche, M. Verleysen and A. Lendasse, “Feature selection for nonlinear models with extreme learning machines”, Neurocomputing 102 (2013) 111–124
[9]
W. Zong, G.-B. Huang, and Y. Chen, “Weighted extreme learning machine for imbalance learning,” Neurocomputing, vol. 101, pp. 229-242, 2013.
[10]
Y. Yang, Y. Wang, and X. Yuan, "Bidirectional extreme learning machine for regression problem and its learning effectiveness," IEEE Transactions on Neural Networks and Learning Systems, Vol. 23, pp. 1498 - 1505, 2012
[11]
J. Cao, Z. Lin, and G.-B. Huang, “Self-adaptive evolutionary extreme learning machine,” Neural Processing Letters, vol. 36, pp. 285-305, 2012.
[12]
M.-B. Li, G.-B. Huang, P. Saratchandran, and N. Sundararajan, “Fully Complex Extreme Learning Machine,”Neurocomputing, vol. 68, pp. 306-314, 2005.
[13]
N.-Y. Liang, G.-B. Huang, P. Saratchandran, and N. Sundararajan, “A Fast and Accurate On-line Sequential Learning Algorithm for Feedforward Networks," IEEE Transactions on Neural Networks, vol. 17, no. 6, pp. 1411-1423, 2006
[14]
Y. Wang, F. Cao and Yubo Yuan, “A study on effectiveness of extreme learning machine”, Neurocomputing 74 (2011) 2483–2490
[15]
J. M. Martınez-Martınez, P. Escandell-Montero, E. Soria-Olivas, J. D. Martın-Guerrero, R. Magdalena-Benedito and J. Gomez-Sanchis, “Regularized extreme learning machine for regression problems”, Neurocomputing 74 (2011) 3716–3721
[16]
P. Horata , S. Chiewchanwattana, K. Sunat, “Robust extreme learning machine”, Neurocomputing 102 (2013) 31–44
[17]
H.-J. Rong, Y.-S.Ong , A.-H. Tan and Z. Zhu, “A fast pruned-extreme learning machine for classification problem”, Neurocomputing 72 (2008) 359–366
[19]
J.W. Smith, J.E. Everhart, W.C. Dickson, W.C. Knowler, and R.S. Johannes, “Using the ADAP learning algorithm to forecast the onset of diabetes mellitus”, In Proceedings of the Symposium on Computer Applications and Medical Care, IEEE Computer Society Press, pp. 261—265, 1988
[20]
B. V. Ramana, M. S. P. Babu and N. B. Venkateswarlu, “A Critical Comparative Study of Liver Patients from USA and INDIA: An Exploratory Analysis”, International Journal of Computer Science Issues, ISSN: 1694-0784, May 2012.
[21]
B. V. Ramana, M. S. P. Babu and N. B. Venkateswarlu, “A Critical Study of Selected Classification Algorithms for Liver Disease Diagnosis”, International Journal of Database Management Systems (IJDMS), Vol.3, No.2, ISSN: 0975-5705, PP 101-114, May 2011.
[22]
A. Frank and A. Asuncion, UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science
[23]
S. Suresh, S. Saraswathi and N. Sundararajan, “Performance enhancement of extreme learning machine for multi-category sparse data classification problems”, Engineering Applications of Artificial Intelligence, 23 (2010) 1149-1157.
[24]
R. Hai-Jun, O. Yew-Soon, T. Ah-Hwee and Zexuan Zhu, “A fast pruned-extreme learning machine for classification problem”, Neurocomputing, 72 (2008) 359- 366.
[25]
Huang, G.B., X. Ding, and H. Zhou, “Optimization method based extreme learning machine for classification”, Neurocomputing, 74(1-3) (2010) 155-163.
[26]
Guopeng Zhao, Zhiqi Shen, Chunyan Miao and Zhihong Man, “On improving the conditioning of extreme learning machine: A linear case”, 7th International Conference on Information, Communications and Signal Processing, ICICS 2009. , 8-10 Dec. 2009
[27]
Naseem I., Togneri R. and Bennamoun M., “Linear Regression for Face Recognition”, IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol. 32, No. 11, November 2010, pp. 2106-2112
