Learning by message passing in networks of discrete synapses

被引:111
作者
Braunstein, A [1 ]
Zecchina, R [1 ]
机构
[1] Abdus Salaam Int Ctr Theoret Phys, I-34100 Trieste, Italy
关键词
D O I
10.1103/PhysRevLett.96.030201
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
We show that a message-passing process allows us to store in binary "material" synapses a number of random patterns which almost saturate the information theoretic bounds. We apply the learning algorithm to networks characterized by a wide range of different connection topologies and of size comparable with that of biological systems (e.g., n similar or equal to 10(5)-10(6)). The algorithm can be turned into an online-fault tolerant-learning protocol of potential interest in modeling aspects of synaptic plasticity and in building neuromorphic devices.
引用
收藏
页数:4
相关论文
共 40 条
[31]   Improved message passing for inference in densely connected systems [J].
Neirotti, JP ;
Saad, D .
EUROPHYSICS LETTERS, 2005, 71 (05) :866-872
[32]   Graded bidirectional synaptic plasticity is composed of switch-like unitary events [J].
O'Connor, DH ;
Wittenberg, GM ;
Wang, SSH .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2005, 102 (27) :9679-9684
[33]   THE WEIGHT-SPACE OF THE BINARY PERCEPTRON [J].
PENNEY, RW ;
SHERRINGTON, D .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1993, 26 (22) :6173-6185
[34]   All-or-none potentiation at CA3-CA1 synapses [J].
Petersen, CCH ;
Malenka, RC ;
Nicoll, RA ;
Hopfield, JJ .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 1998, 95 (08) :4732-4737
[35]   STATISTICAL-MECHANICS OF LEARNING FROM EXAMPLES [J].
SEUNG, HS ;
SOMPOLINSKY, H ;
TISHBY, N .
PHYSICAL REVIEW A, 1992, 45 (08) :6056-6091
[36]   SOLUTION OF SOLVABLE MODEL OF A SPIN GLASS [J].
THOULESS, DJ ;
ANDERSON, PW ;
PALMER, RG .
PHILOSOPHICAL MAGAZINE, 1977, 35 (03) :593-601
[37]  
Tsodyks M. V., 1990, Modern Physics Letters B, V4, P713, DOI 10.1142/S0217984990000891
[38]   Neuromorphic architectures for nanoetectronic circuits [J].
Türel, Ö ;
Lee, JH ;
Ma, XL ;
Likharev, KK .
INTERNATIONAL JOURNAL OF CIRCUIT THEORY AND APPLICATIONS, 2004, 32 (05) :277-302
[39]   Constructing free-energy approximations and generalized belief propagation algorithms [J].
Yedidia, JS ;
Freeman, WT ;
Weiss, Y .
IEEE TRANSACTIONS ON INFORMATION THEORY, 2005, 51 (07) :2282-2312
[40]   Dynamic synapses in the cortex [J].
Zador, AM ;
Dobrunz, LE .
NEURON, 1997, 19 (01) :1-4