WHAT DETERMINES THE CAPACITY OF AUTOASSOCIATIVE MEMORIES IN THE BRAIN

被引:239
作者
TREVES, A
ROLLS, ET
机构
关键词
D O I
10.1088/0954-898X/2/4/004
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Threshold-linear (graded response) units approximate the real firing behaviour of pyramidal neurons in a simplified form, suited to the analytical study of large autoassociative networks. Here we extend previous results on threshold-linear networks to a much larger class of models, by considering different connectivities (including full feedback, highly diluted and multilayer feedforward architectures), different forms of Hebbian learning rules, and different distributions of firing rates (including realistic, continuous distributions of rates). This allows an evaluation of the main factors which may affect, in real cortical networks, the capacity for storage and retrieval of discrete firing patterns. In each case a single equation is derived, which determines both alpha(c), the maximum number of retrievable patterns per synapse, and I(m), the maximum amount of retrievable information per synapse. It is shown that: 1. Non-specific effects, such as those usually ascribed to inhibition, or to neuromodulatory afferents, alter the overall response, but do not affect the capacity for retrieval. 2. The crucial parameter which affects alpha(c) is a, the sparseness of the neural code. Results previously obtained with binary distributions of rates are shown to hold in general, namely that as a --> 0 (sparse coding) alpha(c) grows proportionally to (aln(l/a))-1, while I(m) depends only weakly on a. 3. When the coding is sparse, alpha(c) and I(m) become independent of the connectivity, and in particular the relative disadvantages of fully connected feedback networks, which are prominent with non-sparse codes, disappear. 4. The precise form of the distribution of rates, and that of the learning rule used, turn out to have rather limited effects on alpha(c) and I(m). It is to be noted, however, that when non-binary patterns are stored using nonlinear learning rules, such as those possibly modelling the action of NMDA receptors, information distortion occurs in retrieval, and results in a marked decrease of I(m). These results may be applied to help understand the organization of a specific network in the hippocampus thought to operate as an autoassociative memory.
引用
收藏
页码:371 / 397
页数:27
相关论文
共 52 条
[1]   Firing patterns of single units in the prefrontal cortex and neural network models [J].
Abeles, M. ;
Vaadia, E. ;
Bergman, H. .
NETWORK-COMPUTATION IN NEURAL SYSTEMS, 1990, 1 (01) :13-25
[2]  
AMARAL DG, 1990, PROG BRAIN RES, V83, P1
[3]   THE COMMISSURAL CONNECTIONS OF THE MONKEY HIPPOCAMPAL-FORMATION [J].
AMARAL, DG ;
INSAUSTI, R ;
COWAN, WM .
JOURNAL OF COMPARATIVE NEUROLOGY, 1984, 224 (03) :307-336
[4]  
Amit D. J., 1989, MODELLING BRAIN FUNC
[5]   Neural potentials as stimuli for attractor neural networks [J].
Amit, Daniel J. ;
Parisi, G. ;
Nicolis, S. .
NETWORK-COMPUTATION IN NEURAL SYSTEMS, 1990, 1 (01) :75-88
[6]   SPIN-GLASS MODELS OF NEURAL NETWORKS [J].
AMIT, DJ ;
GUTFREUND, H .
PHYSICAL REVIEW A, 1985, 32 (02) :1007-1018
[7]   INFORMATION-STORAGE IN NEURAL NETWORKS WITH LOW-LEVELS OF ACTIVITY [J].
AMIT, DJ ;
GUTFREUND, H ;
SOMPOLINSKY, H .
PHYSICAL REVIEW A, 1987, 35 (05) :2293-2303
[8]   STATISTICAL-MECHANICS OF NEURAL NETWORKS NEAR SATURATION [J].
AMIT, DJ ;
GUTFREUND, H ;
SOMPOLINSKY, H .
ANNALS OF PHYSICS, 1987, 173 (01) :30-67
[9]   ELECTROPHYSIOLOGICAL PROPERTIES AND SYNAPTIC RESPONSES IN THE DEEP LAYERS OF THE HUMAN EPILEPTOGENIC NEOCORTEX INVITRO [J].
AVOLI, M ;
OLIVIER, A .
JOURNAL OF NEUROPHYSIOLOGY, 1989, 61 (03) :589-606
[10]   A PHYSIOLOGICAL-BASIS FOR A THEORY OF SYNAPSE MODIFICATION [J].
BEAR, MF ;
COOPER, LN ;
EBNER, FF .
SCIENCE, 1987, 237 (4810) :42-48