Convexity, internal representations and the statistical mechanics of neural networks

被引:3
作者
Opper, M
Kuhlmann, P
Mietzner, A
机构
[1] Inst. für Theor. Physik III, Universität Würzburg, D-97074 Würzburg, Am Hubland
来源
EUROPHYSICS LETTERS | 1997年 / 37卷 / 01期
关键词
D O I
10.1209/epl/i1997-00113-x
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
We present an approach to the statistical mechanics of feedforward neural networks which is based on counting realizable internal representations by utilizing convexity properties of the weight space. For a toy model, our method yields storage capacities based on an annealed approximation, which are in close agreement with one-step replica symmetry-breaking results obtained from a standard approach. For a single-layer perceptron, a combinatorial result for the number of realizable output combinations is recovered and generalized to fixed stabilities.
引用
收藏
页码:31 / 36
页数:6
相关论文
共 20 条
[1]  
[Anonymous], OPTIMIZATION PROBLEM
[2]  
[Anonymous], 1987, WORLD SCI LECT NOTES
[3]   STATISTICAL-MECHANICS OF A MULTILAYERED NEURAL NETWORK [J].
BARKAI, E ;
HANSEL, D ;
KANTER, I .
PHYSICAL REVIEW LETTERS, 1990, 65 (18) :2312-2315
[4]   BROKEN SYMMETRIES IN MULTILAYERED PERCEPTRONS [J].
BARKAI, E ;
HANSEL, D ;
SOMPOLINSKY, H .
PHYSICAL REVIEW A, 1992, 45 (06) :4146-4161
[5]   GEOMETRICAL AND STATISTICAL PROPERTIES OF SYSTEMS OF LINEAR INEQUALITIES WITH APPLICATIONS IN PATTERN RECOGNITION [J].
COVER, TM .
IEEE TRANSACTIONS ON ELECTRONIC COMPUTERS, 1965, EC14 (03) :326-&
[6]   Multifractal analysis of the coupling space of feedforward neural networks [J].
Engel, A ;
Weigt, M .
PHYSICAL REVIEW E, 1996, 53 (03) :R2064-R2067
[7]   STORAGE CAPACITY AND LEARNING ALGORITHMS FOR 2-LAYER NEURAL NETWORKS [J].
ENGEL, A ;
KOHLER, HM ;
TSCHEPKE, F ;
VOLLMAYR, H ;
ZIPPELIUS, A .
PHYSICAL REVIEW A, 1992, 45 (10) :7590-7609
[8]   THE SPACE OF INTERACTIONS IN NEURAL NETWORK MODELS [J].
GARDNER, E .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1988, 21 (01) :257-270
[9]   STORAGE CAPACITY AND OPTIMAL LEARNING OF POTTS-MODEL PERCEPTRONS BY A CAVITY METHOD [J].
GERL, F ;
KREY, U .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1994, 27 (22) :7353-7372
[10]  
Kinzel W., 1991, MODELS NEURAL NETWOR, P149