ONLINE LEARNING IN THE COMMITTEE MACHINE

被引:35
作者
COPELLI, M
CATICHA, N
机构
[1] Inst. de Fisica, Sao Paulo Univ.
来源
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL | 1995年 / 28卷 / 06期
关键词
D O I
10.1088/0305-4470/28/6/016
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
The dynamics of learning from examples in the K = 3 non-overlapping committee machine with single presentation of examples is studied. The optimal algorithm, in the sense of mean generalization, is obtained from a variational analysis of the differential equations which describe the dynamics. The agreement of the theoretical predictions and the results of numerical simulations is excellent. The optimized dynamics has the extra advantage with respect to the non-optimized cases in that it uncouples the differential equations which describe the evolution of the relevant parameters, i.e, the student-teacher overlap and the norm of the student synaptic vector. This, in turn, translates into the possibility of constructing useful practical optimized on-line algorithms. For the optimal algorithm the generalization error decays as similar to 0.88 alpha(-1), the same nominal error as for the simple perceptron with optimized dynamics.
引用
收藏
页码:1615 / 1625
页数:11
相关论文
共 12 条
  • [1] BICHI M, 1994, LEARNING ONLINE GRAD
  • [2] COPELLI M, UNPUB
  • [3] 3 UNFINISHED WORKS ON THE OPTIMAL STORAGE CAPACITY OF NETWORKS
    GARDNER, E
    DERRIDA, B
    [J]. JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1989, 22 (12): : 1983 - 1994
  • [4] GYORGYI G, 1989, STATISTICAL THEORY L
  • [5] PERFECT LOSS OF GENERALIZATION DUE TO NOISE IN K=2 PARITY MACHINES
    KABASHIMA, Y
    [J]. JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1994, 27 (06): : 1917 - 1927
  • [6] OPTIMAL GENERALIZATION IN PERCEPTRONS
    KINOUCHI, O
    CATICHA, N
    [J]. JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1992, 25 (23): : 6243 - 6250
  • [7] LOWER BOUNDS ON GENERALIZATION ERRORS FOR DRIFTING RULES
    KINOUCHI, O
    CATICHA, N
    [J]. JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1993, 26 (22): : 6161 - 6171
  • [8] KINOUCHI O, 1992, THESIS U SAO PAULO S
  • [9] KINOUCHI O, 1991, PHYSICA A, V185, P411
  • [10] IMPROVING A NETWORK GENERALIZATION ABILITY BY SELECTING EXAMPLES
    KINZEL, W
    RUJAN, P
    [J]. EUROPHYSICS LETTERS, 1990, 13 (05): : 473 - 477