UNIVERSAL APPROXIMATION TO NONLINEAR OPERATORS BY NEURAL NETWORKS WITH ARBITRARY ACTIVATION FUNCTIONS AND ITS APPLICATION TO DYNAMICAL-SYSTEMS

The purpose of this paper is to investigate neural network capability systematically. The main results are: 1) every Tauber-Wiener function is qualified as an activation function in the hidden layer of a three-layered neural network, 2) for a continuous function in S' (R(1)) to be a Tauber-Wiener function, the necessary and sufficient conditions that it is not a polynomial, 3) the capability of approximating nonlinear functionals defined on some compact set of a Banach space and nonlinear operators has been shown, which implies that 4) we show the possibility by neural computation to approximate the output as a whole (not at a fixed point) of a dynamical system, thus identifying the system.