TRAINING NEURAL NETS THROUGH STOCHASTIC MINIMIZATION

The revival of multilayer neural networks in the mid 1980s originated from the discovery of the back propagation technique as a feasible training procedure. in spite of its shortcomings, it is probably the most widespread technique for training feedforward nets. In recent years, several deterministic methods more efficient than back propagation have been proposed. In this paper a stochastic minimization algorithm, the iterated adaptive memory stochastic search, is described that does not use gradient information and is found to perform better than back propagation on the encoder and parity problems.