A signal processing framework based on dynamic neural networks with application to problems in adaptation, filtering, and classification

We present in this paper a coherent neural network-based framework for solving a variety of difficult signal processing problems. The framework relies on the assertion that time-lagged recurrent networks possess the necessary representational capabilities to act as universal approximators of nonlinear dynamical systems. This property applies to modeling problems posed as system identification, time-series prediction, nonlinear filtering adaptive filtering, and temporal pattern classification. We address the development of models of nonlinear dynamical systems, in the form of time-lagged recurrent neural networks, which can be used without further training (i.e., as fixed-weight networks). We employ a weight update procedure based on the extended Kalman filter (EKF); as a solution to the recency effect, which is the tendency for a network to forget earlier learning as it processes new examples, we have developed a technique called multistream training. We demonstrate our training framework by applying it to four problems. First, we show, that a single time-lagged recurrent neural network can be trained not only to produce excellent one-time-step predictions for two different time;series, but also to be robust to severe errors in the provided input sequence. The second problem involves the modeling of a complex system containing significant process noise, which was shown in [1] to lead to unstable trained models. We illustrate how multistream training may be used to enhance the stability of such models. The remaining two problems are drawn from real-world automotive applications. The first of these involves input-output modeling of the dynamic behavior of a catalyst-sensor system which is exposed to art operating engine's exhaust stream. Finally we consider real-time and continuous detection of engine misfire, which is cast as a dynamic pattern classification problem.