OPTIMIZED FEATURE-EXTRACTION AND THE BAYES DECISION IN FEEDFORWARD CLASSIFIER NETWORKS

In this paper we address the problem of multiclass pattern classification using adaptive layered networks. We consider a special class of networks, i.e., feed-forward networks with a linear final layer, that perform generalized linear discriminant analysis. This class is sufficiently generic to encompass the behavior of arbitrary feed-forward nonlinear networks since there is no restriction on the number of nonlinear hidden layers. Additionally, the use of a final layer of linear output nodes allows a formal analysis to be made to predict the optimum network performance. Training the network consists of a least-square approach which combines a generalized inverse computation to solve for the final layer weights, together with a nonlinear optimization scheme to solve for parameters of the nonlinearities. Such an approach performs feature extraction and classification simultaneously, in which the feature extraction is (optimally) matched to the classification scheme. We derive a general analytic form for the feature extraction criterion and interpret it for specific forms of target coding and error weighting. An important aspect of the approach is to exhibit how a priori information regarding nonuniform class membership, uneven distribution between train and test sets and misclassification costs may be exploited in a regularized manner in the training phase of networks.