Training multilayer perceptrons via minimization of sum of ridge functions

Motivated by the problem of training multilayer perceptrons in neural networks, we consider the problem of minimizing E(x) = Sigma(i=1)(n) f(i) (xi(i) . x), where xi(i) is an element of R-s, 1 less than or equal to i less than or equal to n, and each f(i) (xi(i).x) is a ridge function. We show that when n is small the problem of minimizing E can be treated as one of minimizing univariate functions, and we use the gradient algorithms for minimizing E when n is moderately large. For large n, we present the online gradient algorithms and especially show the monotonicity and weak convergence of the algorithms.