Reinforcement learning signals predict future decisions

被引:257
作者
Cohen, Michael X.
Ranganath, Charan
机构
[1] Univ Calif Davis, Ctr Neurosci, Davis, CA 95616 USA
[2] Univ Bonn, Dept Epileptol, D-53105 Bonn, Germany
[3] Univ Bonn, Ctr Mind & Brain, D-53105 Bonn, Germany
关键词
reward prediction error; ERN; decision-making; reinforcement learning; dopamine; event-related potential;
D O I
10.1523/JNEUROSCI.4421-06.2007
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Optimal behavior in a competitive world requires the flexibility to adapt decision strategies based on recent outcomes. In the present study, we tested the hypothesis that this flexibility emerges through a reinforcement learning process, in which reward prediction errors are used dynamically to adjust representations of decision options. We recorded event-related brain potentials (ERPs) while subjects played a strategic economic game against a computer opponent to evaluate how neural responses to outcomes related to subsequent decision-making. Analyses of ERP data focused on the feedback-related negativity (FRN), an outcome-locked potential thought to reflect a neural prediction error signal. Consistent with predictions of a computational reinforcement learning model, we found that the magnitude of ERPs after losing to the computer opponent predicted whether subjects would change decision behavior on the subsequent trial. Furthermore, FRNs to decision outcomes were disproportionately larger over the motor cortex contralateral to the response hand that was used to make the decision. These findings provide novel evidence that humans engage a reinforcement learning process to adjust representations of competing decision options.
引用
收藏
页码:371 / 378
页数:8
相关论文
共 68 条
[21]   Dissociable executive functions in the dynamic control of behavior: Inhibition, error detection, and correction [J].
Garavan, H ;
Ross, TJ ;
Murphy, K ;
Roche, RAP ;
Stein, EA .
NEUROIMAGE, 2002, 17 (04) :1820-1829
[22]   BURST FIRING INDUCED IN MIDBRAIN DOPAMINE NEURONS BY STIMULATION OF THE MEDIAL PREFRONTAL AND ANTERIOR CINGULATE CORTICES [J].
GARIANO, RF ;
GROVES, PM .
BRAIN RESEARCH, 1988, 462 (01) :194-198
[23]   Functions of the medial frontal cortex in the processing of conflict and errors [J].
Gehring, WJ ;
Fencsik, DE .
JOURNAL OF NEUROSCIENCE, 2001, 21 (23) :9430-9437
[24]   A NEURAL SYSTEM FOR ERROR-DETECTION AND COMPENSATION [J].
GEHRING, WJ ;
GOSS, B ;
COLES, MGH ;
MEYER, DE ;
DONCHIN, E .
PSYCHOLOGICAL SCIENCE, 1993, 4 (06) :385-390
[25]   The medial frontal cortex and the rapid processing of monetary gains and losses [J].
Gehring, WJ ;
Willoughby, AR .
SCIENCE, 2002, 295 (5563) :2279-2282
[26]   Representation of a perceptual decision in developing oculomotor commands [J].
Gold, JI ;
Shadlen, MN .
NATURE, 2000, 404 (6776) :390-394
[27]   A computational model of action selection in the basal ganglia. I. A new functional anatomy [J].
Gurney, K ;
Prescott, TJ ;
Redgrave, P .
BIOLOGICAL CYBERNETICS, 2001, 84 (06) :401-410
[28]   Brain potentials associated with expected and unexpected good and bad outcomes [J].
Hajcak, G ;
Holroyd, CB ;
Moser, JS ;
Simons, RF .
PSYCHOPHYSIOLOGY, 2005, 42 (02) :161-170
[29]   Source localization (LORETA) of the error-related-negativity (ERN/Ne) and positivity (Pe) [J].
Herrmann, MJ ;
Römmler, J ;
Ehlis, AC ;
Heidrich, A ;
Fallgatter, AJ .
COGNITIVE BRAIN RESEARCH, 2004, 20 (02) :294-299
[30]  
HEWIG J, 2006, IN PRESS CEREB CORTE