ONLINE LEARNING WITH A PERCEPTRON

被引:45
作者
BIEHL, M [1 ]
RIEGLER, P [1 ]
机构
[1] UNIV WURZBURG,INST THEORET PHYS,D-97074 WURZBURG,GERMANY
来源
EUROPHYSICS LETTERS | 1994年 / 28卷 / 07期
关键词
D O I
10.1209/0295-5075/28/7/012
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
We study on-line learning of a linearly separable rule with a simple perceptron. Training utilizes a sequence of uncorrelated, randomly drawn N-dimensional input examples. In the thermodynamic limit the generalization error after training such examples with P can be calculated exactly. For the standard perceptron algorithm it decreaes like (NIP)(1/3) for large P/N, in contrast to the faster (NIP)(1/2)-behaviour of the so-called Hebbian learning. Furthermore, we show that a specific parameter-free on-line scheme, the AdaTron algorithm, gives an asymptotic (N/P)-decay of the generalization error. This coincides (up to a constant factor) with the bound for any training process based on random examples, including off-line learning. Simulations confirm our results.
引用
收藏
页码:525 / 530
页数:6
相关论文
共 21 条
[1]   THE ADATRON - AN ADAPTIVE PERCEPTRON ALGORITHM [J].
ANLAUF, JK ;
BIEHL, M .
EUROPHYSICS LETTERS, 1989, 10 (07) :687-692
[2]  
BARKAI N, 1949, ONLINE LEARNING DICH
[3]   The Perceptron Algorithm is Fast for Nonmalicious Distributions [J].
Baum, Eric B. .
NEURAL COMPUTATION, 1990, 2 (02) :248-260
[4]   LEARNING DRIFTING CONCEPTS WITH NEURAL NETWORKS [J].
BIEHL, M ;
SCHWARZE, H .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1993, 26 (11) :2651-2665
[5]  
BIEHL M, 1994, LUTP9410 LUND U PREP
[6]   SYSTEMS THAT CAN LEARN FROM EXAMPLES - REPLICA CALCULATION OF UNIFORM-CONVERGENCE BOUNDS FOR PERCEPTRONS [J].
ENGEL, A ;
VANDENBROECK, C .
PHYSICAL REVIEW LETTERS, 1993, 71 (11) :1772-1775
[7]   STATISTICAL-MECHANICS CALCULATION OF VAPNIK-CHERVONENKIS BOUNDS FOR PERCEPTRONS [J].
ENGEL, A ;
FINK, W .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1993, 26 (23) :6893-6914
[8]   LEARNING AND RETRIEVAL IN ATTRACTOR NEURAL NETWORKS ABOVE SATURATION [J].
GRINIASTY, M ;
GUTFREUND, H .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1991, 24 (03) :715-734
[9]  
HEBB DO, 1949, ORG BEHAVIOR
[10]  
Hertz J., 1991, INTRO THEORY NEURAL