A STOCHASTIC-MODEL OF NEURAL NETWORK FOR UNSUPERVISED LEARNING

A two-layered neural network that organizes itself in response to a set of external stimuli is considered. The output layer is a stochastic discrete model of a neural network. The synaptic weights between layers obey a competitive learning law. It is shown that the Long-Term Memory dynamics (weight dynamics) always stabilize by minimizing a Lyapounov function. In the special case where all lateral interactions are uniform, negative and sufficiently strong, this Lyapounov function, within the limit of low noise, is interpreted as the Kullback discrepancy between the probability distribution of the environment and the internal representation of the network.