THE LANGEVIN METHOD IN THE STATISTICAL DYNAMICS OF LEARNING

The statistical dynamics of learning in the presence of noise has been treated recently by Hertz, Krogh and Thorbergson (1989) using a Langevin model. Fluctuations introduced by this model into the space orthogonal to the space of patterns are shown to influence the dynamics of learning in a spurious way. This is avoided in the new Langevin model in which the noise is introduced directly into the cost function by using randomly modulated training patterns. Correspondingly a new kind of response function is defined. Unconstrained learning and learning under constrained thermal fluctuations are discussed as examples. Explicit expressions-e.g. for the learning time-are found to differ substantially from those given by Hertz et al.