Metalearning and neuromodulation

被引:425
作者
Doya, K [1 ]
机构
[1] Japan Sci & Technol Corp, CREST, ATR, Human Informat Sci Labs, Kyoto 6190288, Japan
关键词
metalearning; neuromodulator; dopamine; serotonin; noradrenaline; acetylcholine; reinforcement learning; discount factor;
D O I
10.1016/S0893-6080(02)00044-8
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper presents a computational theory on the roles of the ascending neuromodulatory systems from the viewpoint that they mediate the global signals that regulate the distributed learning mechanisms in the brain. Based on the review of experimental data and theoretical models, it is proposed that dopamine signals the error in reward prediction, serotonin controls the time scale of reward prediction, noradrenaline controls the randomness in action selection, and acetylcholine controls the speed of memory update. The possible interactions between those neuromodulators and the environment are predicted on the basis of computational theory of metalearning. (C) 2002 Elsevier Science Ltd. All rights reserved.
引用
收藏
页码:495 / 506
页数:12
相关论文
共 83 条
[1]  
AOSAKI T, 1994, J NEUROSCI, V14, P3969
[2]  
ASTONJONES G, 1994, J NEUROSCI, V14, P4467
[3]  
Barto A. G., 1995, HDB BRAIN THEORY NEU, P804
[4]   NEURONLIKE ADAPTIVE ELEMENTS THAT CAN SOLVE DIFFICULT LEARNING CONTROL-PROBLEMS [J].
BARTO, AG ;
SUTTON, RS ;
ANDERSON, CW .
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS, 1983, 13 (05) :834-846
[5]  
Barto AG., 1995, Models of information processing in the basal ganglia, P215
[6]  
Baxter J., 2000, INT C MACH LEARN
[7]  
Brown J, 1999, J NEUROSCI, V19, P10502
[8]   Serotonin receptors in cognitive behaviors [J].
Buhot, MC .
CURRENT OPINION IN NEUROBIOLOGY, 1997, 7 (02) :243-254
[9]   Impulsive choice induced in rats by lesions of the nucleus accumbens core [J].
Cardinal, RN ;
Pennicott, DR ;
Sugathapala, CL ;
Robbins, TW ;
Everitt, BJ .
SCIENCE, 2001, 292 (5526) :2499-2501
[10]   Opponent interactions between serotonin and dopamine [J].
Daw, ND ;
Kakade, S ;
Dayan, P .
NEURAL NETWORKS, 2002, 15 (4-6) :603-616