Optimization approximation solution for regression problem based on extreme learning machine

Extreme learning machine (ELM) is one of the most popular and important learning algorithms. It comes from single-hidden-layer feedforward neural networks. It has been proved that ELM can achieve better performance than support vector machine (SVM) in regression and classification. In this paper, mathematically, with regression problem, the step 3 of ELM is studied. First of all, the equation H beta=T are reformulated as an optimal model. With the optimality, the necessary conditions of optimal solution are presented. The equation H beta=T is replaced by (HH)-H-T beta= (HT)-T-T. We can prove that the latter must have one solution at least. Second, optimal approximation solution is discussed in cases of H is column full rank, row full rank, neither column nor row full rank. In the last case, the rank-1 and rank-2 methods are used to get optimal approximation solution. In theory, this paper present a better algorithm for ELM. (C) 2011 Elsevier B.V. All rights reserved.