Limitations of the approximation capabilities of neural networks with one hidden layer

Let s greater than or equal to 1 be an integer and W be the class of all functions having integrable partial derivatives on [0, 1](s). We are interested in the minimum number of neurons in a neural network with a Single hidden layer required in order to provide a mean approximation order of a preassigned epsilon > 0 to each function in W. We prove that this number cannot be O(epsilon(-s) log(1/epsilon)) if a spline-like localization is required. This cannot be improved even if one allows different neurons to evaluate different activation functions, even depending upon the target function. Nevertheless, for any delta > 0, a network with O(epsilon(-s-delta)) neurons can be constructed to provide this order of approximation, with localization. Analogous results are also valid for other L(P) norms.