WILLSHAW MODEL - ASSOCIATIVE MEMORY WITH SPARSE CODING AND LOW FIRING RATES

The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -,>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-f2P) is finite. Memory states are stable (at zero temperature), as long as C>h0==K-1+ and h0>0. When C<h0 or h0<0, P retrieval phases, highly correlated with the memory states, exist. These phases are only partially frozen at low temperature, so that the full memories can be retrieved from them by averaging over the dynamic fluctuations of the neural activity. In particular, when h0<0 the retrieval phases at low temperatures correspond to freezing of most of the population in a quiescent state while the rest are active with a time average that can be significantly smaller than the saturation level. These features resemble, to some extent, the observed patterns of neural activity in the cortex, in experiments of short-term memory tasks. The maximal value of P for which stable retrieval phases exist, scales as f-3/lnf for f1/lnN, and as f-2ln(Nf/lnf) for f1/lnN. Numerical simulations of the model with N=1000 and f=0.04 are presented. We also discuss the possible realization of the model in a biologically plausible architecture, where the inhibition is provided by special inhibitory neurons. © 1990 The American Physical Society.