PLS NEURAL NETWORKS

Performance improvements for feedforward networks (FFNs) are investigated. First, FFNs are shown to be inferior to simpler linear methods for some examples. A topological change to the network, the inclusion of a linear hidden unit, is introduced to address the problem. A proof is developed to demonstrate that this change allows FFNs to recover linear performance while not sacrificing FFNs' abilities to reproduce nonlinear mappings. Another enhancement is developed by incorporating a popular linear technique, partial least squares (PLS). The PLS-based enhancement (PLS/neural) is an improvement over methods that combine principle component analysis (PCA) with FFNs and is particularly useful for noise suppression and sparse data sets. This new method is shown to not have several of limitations of another nonlinear PLS method (Wold et al., 1989). Two examples based on experimental data and a tutorial example are developed to illustrate the methods. This work focuses on network topology and the objective functions used to generate the learning rules. Learning rule enhancements (e.g. using improved optimization methods such as a conjugate gradient) are not investigated but can be easily included to extend further the performance improvements.