UNIFIED FRAMEWORK FOR MLPS AND RBFNS - INTRODUCING CONIC SECTION FUNCTION NETWORKS

Multilayer perceptrons (MLPs) (Werbos, 1974; Rumelhart et al., 1986) and radial basis function networks (RBI;Ns) (Broomhead and Lowe, 1988; Moody and Darken, 1989) are probably the most widely used neural network models for practical applications. Whereas the former belong to a group of ''classical'' neural networks (whose weighted sums are loosely inspired by biology), the latter have risen only recently from an analogy to regression theory (Broomhead and Lowe, 1988). On first sight, the twomodels-except for being multilayer feedforward networks-do not seem to have much in common. On second thought, however, MLPs and RBFNs share a variety of features, worthy of viewing them in the same context and comparing them to each other with respect to their properties. Consequently, a few attempts on arriving at a unified picture of a class of feedforward networks-with MLPs and RBFNs as members-have been undertaken (Robinson et al., 1988; Maruyama et al., 1992; Dorffner, 1992). Most of these attempts have centered around the observation that the function of a neural network unit can be divided into a propagation rule (net input) and an activation or transfer function. The dot product (weighted sum) and the Euclidean distance are special cases of propagation rules, whereas the sigmoid and Gaussian function are examples of activation functions. This paper introduces a novel neural network model based on a more general conic section function as a propagation rule, containing hyperplane (straight line) and hypersphere (circle) as special cases, thus unifying the net inputs of MLPs and RBFNs with an easy-to-handle continuum in between. A new learning rule-complementing the existing methods of gradient descent in weight space and initialization-is introduced which enables the network to make a continuous decision between bounded and unbounded (infinite half-space) decision regions. The capabilities of conic section function networks (CSFNs) are illustrated with several examples and compared with existing approaches. CSFNs are viewed as a further step toward more efficient and optimal neural network solutions in practical applications.