DERIVATION OF NEURAL NETWORK MODELS AND THEIR COMPUTATIONAL CIRCUITS FOR ASSOCIATIVE MEMORY

Neural network models for associative memory are derived independently on the basis of an optimization principle without resort to any assumptions related to biological principles. All the features of the Hopfield model, such as the updating rule with nonlinear threshold, the outer product algorithm, the symmetric and zero-diagonal interconnection matrix, and asynchronous timing, are automatically derived from a simple optimization principle for bipolar and binary variables. The derivation is extended to generate higher order models that have higher storage capacity and better convergence. The computational circuits to implement the neural network models are also derived naturally from the same principle. Various optical implementations of the computational circuits are also described.