Self-organizing algorithms for generalized eigen-decomposition

We discuss a new approach to self-organization that leads to novel adaptive algorithms for generalized eigendecomposition and its variance [such as linear discriminant analysis (LDA)] for a single-layer linear feedforward neural network, First, we derive two novel iterative algorithms for LDA and generalized eigen-decomposition by utilizing a constrained least-mean-squared classification error cost function, and the framework of a two-layer linear heteroassociative network performing a one-of-m classification, By using the concept of deflation, we are able to find sequential versions of these algorithms which extract the LDA components and generalized eigenvectors in a decreasing order of significance. Second, two new adaptive algorithms are described to compute the principal generalized eigenvectors of two matrices (as well as LDA) from two sequences of random matrices, Although iterative algorithms for LDA exist in the literature, we give a rigorous convergence analysis of our adaptive algorithms by using stochastic approximation theory, and prove that our algorithms converge with probability one. As an example, we consider the problem of online interference cancellation in digital mobile communications, Numerical simulations are presented demonstrating the rapid convergence of the adaptive algorithms, and their relative convergence rates.