Metalearning and neuromodulation

被引:425
作者
Doya, K [1 ]
机构
[1] Japan Sci & Technol Corp, CREST, ATR, Human Informat Sci Labs, Kyoto 6190288, Japan
关键词
metalearning; neuromodulator; dopamine; serotonin; noradrenaline; acetylcholine; reinforcement learning; discount factor;
D O I
10.1016/S0893-6080(02)00044-8
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper presents a computational theory on the roles of the ascending neuromodulatory systems from the viewpoint that they mediate the global signals that regulate the distributed learning mechanisms in the brain. Based on the review of experimental data and theoretical models, it is proposed that dopamine signals the error in reward prediction, serotonin controls the time scale of reward prediction, noradrenaline controls the randomness in action selection, and acetylcholine controls the speed of memory update. The possible interactions between those neuromodulators and the environment are predicted on the basis of computational theory of metalearning. (C) 2002 Elsevier Science Ltd. All rights reserved.
引用
收藏
页码:495 / 506
页数:12
相关论文
共 83 条
[21]   Simplified dynamics in a model of noradrenergic modulation of cognitive performance [J].
Gilzenrat, MS ;
Holmes, BD ;
Rajkowski, J ;
Aston-Jones, G ;
Cohen, JD .
NEURAL NETWORKS, 2002, 15 (4-6) :647-663
[22]   A STOCHASTIC REINFORCEMENT LEARNING ALGORITHM FOR LEARNING REAL-VALUED FUNCTIONS [J].
GULLAPALLI, V .
NEURAL NETWORKS, 1990, 3 (06) :671-692
[23]  
HASSELMO ME, 1994, J NEUROSCI, V14, P3898
[24]   ACETYLCHOLINE AND MEMORY [J].
HASSELMO, ME ;
BOWER, JM .
TRENDS IN NEUROSCIENCES, 1993, 16 (06) :218-222
[25]   Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events [J].
Horvitz, JC .
NEUROSCIENCE, 2000, 96 (04) :651-656
[26]  
Houk J., 1995, Models ofInformation Processing in the Basal Ganglia, P249
[27]   Control of exploitation-exploration meta-parameter in reinforcement learning [J].
Ishii, S ;
Yoshida, W ;
Yoshimoto, J .
NEURAL NETWORKS, 2002, 15 (4-6) :665-687
[28]   Actor-critic models of the basal ganglia: new anatomical and computational perspectives [J].
Joel, D ;
Niv, Y ;
Ruppin, E .
NEURAL NETWORKS, 2002, 15 (4-6) :535-547
[29]   Dopamine: generalization and bonuses [J].
Kakade, S ;
Dayan, P .
NEURAL NETWORKS, 2002, 15 (4-6) :549-559
[30]  
KAKADE S, 2001, COMPUTATIONAL LEARNI