An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels

被引:1248
作者
Huang, Guang-Bin [1 ]
机构
[1] Nanyang Technol Univ, Sch Elect & Elect Engn, Singapore 639798, Singapore
关键词
Extreme learning machine; Support vector machine; Least square support vector machine; ELM kernel; Random neuron; Random feature; Randomized matrix; MIXED SELECTIVITY; NEURAL-NETWORKS; MODEL; CLASSIFICATION; APPROXIMATION; PERCEPTRON; ELMS; REPRESENTATIONS; INFORMATION; REGRESSION;
D O I
10.1007/s12559-014-9255-2
中图分类号
TP18 [人工智能理论];
学科分类号
140502 [人工智能];
摘要
Extreme learning machines (ELMs) basically give answers to two fundamental learning problems: (1) Can fundamentals of learning (i.e., feature learning, clustering, regression and classification) be made without tuning hidden neurons (including biological neurons) even when the output shapes and function modeling of these neurons are unknown? (2) Does there exist unified framework for feedforward neural networks and feature space methods? ELMs that have built some tangible links between machine learning techniques and biological learning mechanisms have recently attracted increasing attention of researchers in widespread research areas. This paper provides an insight into ELMs in three aspects, viz: random neurons, random features and kernels. This paper also shows that in theory ELMs (with the same kernels) tend to outperform support vector machine and its variants in both regression and classification applications with much easier implementation.
引用
收藏
页码:376 / 390
页数:15
相关论文
共 79 条
[1]
Akusok A, 2013, IEEE INTELL SYST, V28, P41
[2]
[Anonymous], 2010, P 18 EUR S ART NEUR
[3]
[Anonymous], 1961, PRINCIPLES NEURODYNA
[4]
[Anonymous], 1994, The Roots of Backpropagation
[5]
[Anonymous], 2002, Matrices: Theory and Applications
[6]
[Anonymous], 1971, Generalized Inverses of Matrices and its Applications
[7]
[Anonymous], 1956, Autom. Stud., DOI [10.1515/9781400882618-003, DOI 10.1515/9781400882618-003]
[8]
Sparse Extreme Learning Machine for Classification [J].
Bai, Zuo ;
Huang, Guang-Bin ;
Wang, Danwei ;
Wang, Han ;
Westover, M. Brandon .
IEEE TRANSACTIONS ON CYBERNETICS, 2014, 44 (10) :1858-1870
[9]
The Sparseness of Mixed Selectivity Neurons Controls the Generalization-Discrimination Trade-Off [J].
Barak, Omri ;
Rigotti, Mattia ;
Fusi, Stefano .
JOURNAL OF NEUROSCIENCE, 2013, 33 (09) :3844-3856
[10]
The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network [J].
Bartlett, PL .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1998, 44 (02) :525-536