LEARNING WITH 1ST, 2ND, AND NO DERIVATIVES - A CASE-STUDY IN HIGH-ENERGY PHYSICS

In this paper different algorithms for training multilayer perceptron architectures are applied to a significant discrimination task in high energy physics. The One-Step Secant technique is compared with on-line backpropagation, the 'Bold Driver' batch version and conjugate gradient methods. In addition, a new algorithm (affine shaker) is proposed that uses stochastic search based on function values and affine transformations of the local search region. Although the affine shaker requires more CPU time to reach the maximum generalization, the technique can be interesting for special-purpose VLSI implementations and for non-differentiable functions.