Neural Basis of Reinforcement Learning and Decision Making

被引:313
作者
Lee, Daeyeol [1 ,2 ]
Seo, Hyojung [1 ]
Jung, Min Whan [3 ]
机构
[1] Yale Univ, Sch Med, Dept Neurobiol, Kavli Inst Neurosci, New Haven, CT 06510 USA
[2] Yale Univ, Dept Psychol, New Haven, CT 06520 USA
[3] Ajou Univ, Neurosci Lab, Inst Med Sci, Sch Med, Suwon 443721, South Korea
来源
ANNUAL REVIEW OF NEUROSCIENCE, VOL 35 | 2012年 / 35卷
关键词
prefrontal cortex; neuroeconomics; reward; striatum; uncertainty; LATERAL INTRAPARIETAL CORTEX; TEMPORALLY DISCOUNTED VALUES; POSTERIOR PARIETAL CORTEX; SACCADIC EYE-MOVEMENTS; ORBITOFRONTAL CORTEX; PREFRONTAL CORTEX; BASAL GANGLIA; REWARD SIGNALS; PREDICTION ERRORS; NEURONAL-ACTIVITY;
D O I
10.1146/annurev-neuro-062111-150512
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Reinforcement learning is an adaptive process in which an animal utilizes its previous experience to improve the outcomes of future choices. Computational theories of reinforcement learning play a central role in the newly emerging areas of neuroeconomics and decision neuroscience. In this framework, actions are chosen according to their value functions, which describe how much future reward is expected from each action. Value functions can be adjusted not only through reward and penalty, but also by the animal's knowledge of its current environment. Studies have revealed that a large proportion of the brain is involved in representing and updating value functions and using them to choose an action. However, how the nature of a behavioral task affects the neural mechanisms of reinforcement learning remains incompletely understood. Future studies should uncover the principles by which different computational elements of reinforcement learning are dynamically coordinated across the entire brain.
引用
收藏
页码:287 / 308
页数:22
相关论文
共 149 条
[61]   Neural correlates, computation and behavioural impact of decision confidence [J].
Kepecs, Adam ;
Uchida, Naoshige ;
Zariwala, Hatim A. ;
Mainen, Zachary F. .
NATURE, 2008, 455 (7210) :227-U55
[62]   Role of Striatum in Updating Values of Chosen Actions [J].
Kim, Hoseok ;
Sul, Jung Hoon ;
Huh, Namjung ;
Lee, Daeyeol ;
Jung, Min Whan .
JOURNAL OF NEUROSCIENCE, 2009, 29 (47) :14701-14712
[63]   Prefrontal coding of temporally discounted values during intertemporal choice [J].
Kim, Soyoun ;
Hwang, Jaewon ;
Lee, Daeyeol .
NEURON, 2008, 59 (01) :161-172
[64]   Prefrontal Cortex and Impulsive Decision Making [J].
Kim, Soyoun ;
Lee, Daeyeol .
BIOLOGICAL PSYCHIATRY, 2011, 69 (12) :1140-1146
[65]   Encoding of action history in the rat ventral striatum [J].
Kim, Yun Bok ;
Huh, Namjung ;
Lee, Hyunjung ;
Baeg, Eun Ha ;
Lee, Daeyeol ;
Jung, Min Whan .
JOURNAL OF NEUROPHYSIOLOGY, 2007, 98 (06) :3548-3556
[66]   A neostriatal habit learning system in humans [J].
Knowlton, BJ ;
Mangels, JA ;
Squire, LR .
SCIENCE, 1996, 273 (5280) :1399-1402
[67]   Value representations in the primate striatum during matching behavior [J].
Lau, Brian ;
Glimcher, Paul W. .
NEURON, 2008, 58 (03) :451-463
[68]   Learning and decision making in monkeys during a rock-paper-scissors game [J].
Lee, D ;
McGreevy, BP ;
Barraclough, DJ .
COGNITIVE BRAIN RESEARCH, 2005, 25 (02) :416-430
[69]   Reinforcement learning and decision making in monkeys during a competitive game [J].
Lee, D ;
Conroy, ML ;
McGreevy, BP ;
Barraclough, DJ .
COGNITIVE BRAIN RESEARCH, 2004, 22 (01) :45-58
[70]   Game theory and neural basis of social decision making [J].
Lee, Daeyeol .
NATURE NEUROSCIENCE, 2008, 11 (04) :404-409