REPRESENTATION OF FUNCTIONS BY SUPERPOSITIONS OF A STEP OR SIGMOID FUNCTION AND THEIR APPLICATIONS TO NEURAL NETWORK THEORY

The starting point of this article is the inversion formula of the Radon transform; the article aims to contribute to the theory of three-layered neural networks. Let H be the Heaviside function. Then, for any function f membership sign S(R(n)), there is a function g(f) such that f can be represented on R(n) by an integral integral of H(x . omega - t)g(f)(t, omega) dt d-mu(omega), where mu is the uniform measure on the unit sphere S(n-1), t membership sign R and omega membership sign S(n-1). Furthermore, f can be approximated uniformly arbitrarily well on the whole space R(n) by a finite sum of the form SIGMA-k(a)k(H)(x . omega(k) - t(k)). Let H-sigma be a sigmoid function on R defined by H-sigma(t) = integral of H(t - x . omega) d-sigma(x), where sigma is a spherically symmetric probability measure. Suppose that sigma satisfies a few further conditions. Then, for any f membership sign S(R(n)), there is a function g(f,sigma) such that f can be written integral H-sigma(x . omega - t)g(f,sigma)(t, omega) dt d-mu(omega) with the unscaled sigmoid function H-sigma fixed beforehand. This expression can also be approximated uniformly arbitrarily well on R(n) by a finite sum.