Weighted least squares training of support vector classifiers leading to compact and adaptive schemes

An iterative block training method for support vector classifiers (SVCs) based on weighted least squares (WLS) optimization is presented. The algorithm, which minimizes structural risk in the primal space, is applicable to both linear and nonlinear machines. In some nonlinear cases (described in detail in the paper), it is necessary to previously find a projection of data onto an intermediate-dimensional space by means of either principal component analysis (PCA) or clustering techniques. The proposed approach yields very compact machines, the complexity reduction with respect to the SVC solution is especially notable in problems with highly overlapped classes. Furthermore, the formulation in terms of WLS minimizations makes the development of adaptive SVCs straightforward, opening up new fields of application for this type of model, mainly on-line processing of large amounts of (static/stationary) data, as well as on-line update in nonstationary scenarios (adaptive solutions). The performance of this new type of algorithm is analyzed by means of several simulations.