Approximation by ridge functions and neural networks

We investigate the efficiency of approximation by linear combinations of ridge functions in the metric of L-2(B-d) with B-d the unit ball in R-d. If X-n is an n-dimensional linear space of univariate functions in L-2(I), I = [-1; 1], and Omega is a subset of the unit sphere S-d?1 in R-d of cardinality m, then the space Y-n := span {r(x . xi) : r is an element of X-n, omega is an element of Omega} is a linear space of ridge functions of dimension less than or equal to mn. We show that if X-n provides order of approximation O(n(?r)) for univariate functions with r derivatives in L-2(I), and Omega are properly chosen sets of cardinality O(n(d?1)), then Y-n will provide approximation of order O(n(?r?d/2+1/2)) for every function f is an element of L-2(B-d) with smoothness of order r + d/2 ? 1/2 in L-2(B-d). Thus, the theorems we obtain show that this form of ridge approximation has the same efficiency of approximation as other more traditional methods of multivariate approximation such as polynomials, splines, or wavelets. The theorems we obtain can be applied to show that a feed-forward neural network with one hidden layer of computational nodes given by certain sigmoidal function sigma will also have this approximation efficiency. Minimal requirements are made of the sigmoidal functions and in particular our results hold for the unit-impulse function sigma = chi([0, infinity)).