THE OPTIMIZED INTERNAL REPRESENTATION OF MULTILAYER CLASSIFIER NETWORKS PERFORMS NONLINEAR DISCRIMINANT-ANALYSIS

This paper illustrates why a nonlinear adaptive feed-forward layered network with linear output units can perform well as a pattern classification device. The central result is that minimising the error at the output of the network is equivalent to maximising a particular norm, the network discriminant function, at the output of the hidden units. The first part of the network is explicitly performing a nonlinear transformation of the data into a space in which the classes may be more easily separated. The specific nature of this transformation is constrained to maximise the network discriminant function. If the targets are appropriately chosen, this discriminant function relates the pseudo-inverse of the total covariance matrix and the weighted between-class covariance matrix of the hidden unit patterns. Numerical simulations are presented to illustrate the results. © 1990.