FUNCTIONAL PROPERTIES OF THE BASAL GANGLIA'S RE-ENTRANT LOOP ARCHITECTURE: SELECTION AND REINFORCEMENT

被引:119
作者
Redgrave, P. [1 ]
Vautrelle, N. [1 ]
Reynolds, J. N. J. [2 ]
机构
[1] Univ Sheffield, Dept Psychol, Neurosci Res Unit, Sheffield S10 2TP, S Yorkshire, England
[2] Univ Otago, Dept Anat & Struct Biol, Brain Hlth Res Ctr, Dunedin, New Zealand
基金
英国惠康基金;
关键词
basal ganglia; selection; reinforcement learning; superior colliculus; dopamine; agency; MIDBRAIN DOPAMINERGIC-NEURONS; NIGRA PARS COMPACTA; EARLY VISUAL-CORTEX; SUPERIOR COLLICULUS; SUBSTANTIA-NIGRA; COMPUTATIONAL MODEL; DIRECT PROJECTION; LATERAL HABENULA; REWARD; RESPONSES;
D O I
10.1016/j.neuroscience.2011.07.060
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Multifunctional agents with limited motor resources must decide what actions will best ensure their survival. Moreover, given that in an unpredictable world things don't always work out, considerable advantage is to be gained by learning from experience - instrumental behaviour that maximises reward and minimises punishment. In this review we will argue that the re-entrant looped architecture of the basal ganglia represents biological solutions to these fundamental behavioural problems of selection and reinforcement. A potential solution to the selection problem is provided for by selective disinhibition within the parallel loop architecture that connects the basal ganglia with external neural structures. The relay points within these loops permit the signals of a particular channel to be modified by external influences. In part, these influences have the capacity to modify overall selections so that the probability of re-selecting reinforced behaviours in the future is altered. This is the basic process of instrumental learning, which we suggest decomposes into two sub-problems for the agent: (i) learning which external events it causes to happen and learning precisely what it is doing that is causal; and (ii) having determined agency and discovered novel action-outcome routines, how best to exploit this knowledge to maximise future reward acquisitions. Considerations of connectional architecture and signal timing suggest that the short-latency, sensory-evoked dopamine response, which can modulate the re-entrant loop structure within the basal ganglia, is ideally suited to reinforce the determination of agency and the discovery of novel actions. Alternatively, recent studies showing that presence or absence of reward can selectively modulate the magnitude of signals in structures providing input signals to the basal ganglia, offer an alternative mechanism for biasing selection within the re-entrant loop architecture. We suggest that this mechanism may be better suited to ensure the prioritisation of inputs associated with reward. This article is part of a Special Issue entitled: Function and Dysfunction of the Basal Ganglia. (C) 2011 Published by Elsevier Ltd on behalf of IBRO.
引用
收藏
页码:138 / 151
页数:14
相关论文
共 132 条
[1]   PARALLEL ORGANIZATION OF FUNCTIONALLY SEGREGATED CIRCUITS LINKING BASAL GANGLIA AND CORTEX [J].
ALEXANDER, GE ;
DELONG, MR ;
STRICK, PL .
ANNUAL REVIEW OF NEUROSCIENCE, 1986, 9 :357-381
[2]   Responses of tonically discharging neurons in the monkey striatum to primary rewards delivered during different behavioral states [J].
Apicella, P ;
Legallet, E ;
Trouche, E .
EXPERIMENTAL BRAIN RESEARCH, 1997, 116 (03) :456-466
[3]   Space, time and dopamine [J].
Arbuthnott, Gordon W. ;
Wickens, Jeff .
TRENDS IN NEUROSCIENCES, 2007, 30 (02) :62-69
[4]   Sequential super-stereotypy of an instinctive fixed action pattern in hyper-dopaminergic mutant mice: a model of obsessive compulsive disorder and Tourette's [J].
Berridge, KC ;
Aldridge, JW ;
Houchard, KR ;
Zhuang, XX .
BMC BIOLOGY, 2005, 3 (1)
[5]   What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? [J].
Berridge, KC ;
Robinson, TE .
BRAIN RESEARCH REVIEWS, 1998, 28 (03) :309-369
[6]   The debate over dopamine's role in reward: the case for incentive salience [J].
Berridge, Kent C. .
PSYCHOPHARMACOLOGY, 2007, 191 (03) :391-431
[7]   Cortical metabotropic glutamate receptors contribute to habituation of a simple odor-evoked behavior [J].
Best, AR ;
Thompson, JV ;
Fletcher, ML ;
Wilson, DA .
JOURNAL OF NEUROSCIENCE, 2005, 25 (10) :2513-2517
[8]   COLLATERAL PROJECTIONS OF PREDORSAL BUNDLE CELLS OF THE SUPERIOR COLLICULUS IN THE RAT [J].
BICKFORD, ME ;
HALL, WC .
JOURNAL OF COMPARATIVE NEUROLOGY, 1989, 283 (01) :86-106
[9]   Habituation to painful stimulation involves the antinociceptive system [J].
Bingel, U. ;
Schoell, E. ;
Herken, W. ;
Buechel, C. ;
May, A. .
PAIN, 2007, 131 (1-2) :21-30
[10]   EXCITATORY AMINO-ACID RECEPTORS MODULATE HABITUATION OF THE RESPONSE TO VISUAL-STIMULATION IN THE CAT SUPERIOR COLLICULUS [J].
BINNS, KE ;
SALT, TE .
VISUAL NEUROSCIENCE, 1995, 12 (03) :563-571