Learning and cognitive flexibility: frontostriatal function and monoaminergic modulation
Introduction
The ability to learn, or acquire associations between stimuli, actions and environmental outcomes, and flexibly adapt ongoing behaviour according to salient changes in the environment or our current intentions, carries survival value and is arguably fundamental to what cognitive agents perceive as voluntary action. Research on neural mechanisms subserving learning and flexibility over the last decade has focused on the prefrontal cortex (PFC) and basal ganglia, particularly the striatum, and their interactions through their multiple serial and parallel loops [1, 2], subject to neuromodulatory inputs by the monoamines (dopamine (DA), serotonin (5-hydroxytryptamine, 5-HT) and noradrenaline (NA)).
We review developments in recent years that have contributed to the state of the science, progressing through a conceptual gradient of cognitive processes and paradigms. First, we address basic reinforcement learning, where behaviour is critically dependent on environmental signals or (reward) feedback. Subsequently, we focus on reversal learning, reflecting the ability to switch responding to a previously non-reinforced exemplar, which relies on feedback but critically incorporates flexibility. We proceed to attentional set-shifting, which also indexes the ability to adapt behaviour flexibly following feedback, but pertains to broader stimulus dimensions rather than a specific exemplar. Finally, we address task switching, a relatively purer form of cognitive flexibility uncontaminated by learning and feedback processing.
The literature cited represents a corresponding gradient in spatiotemporal scale as a function of the complexity of cognitive paradigms, and the feasibility of interspecies translation. Basic neuronal and molecular mechanisms of reinforcement learning and low level flexibility emerge predominantly from animal studies employing invasive techniques such as single unit recordings, microdialysis and selective gene knock-out. More cognitively complex forms of flexibility are tapped mostly through systemic pharmacology (across species), lesion studies in animals and functional neuroimaging in humans.
Section snippets
Cortico-striatal anatomy
Before addressing the cognitive neuroscience of cortico-striatal interactions, we note developments in the field of anatomy. The landmark proposal of anatomically and functionally segregated cortico-striatal loops by Alexander and colleagues is gradually becoming superseded by evidence for more complex cross-talk and convergence enabling integration of processing across these circuits [3, 4]. Thus, functional connectivity network effects that may underpin learning and flexible cognition are
Reinforcement learning
For some years it has been clear that both the striatum and the frontal cortex play key roles in learning, being active during the expectation of reward and following its delivery, as well as coding prediction error during learning [8, 9]. Beyond the postulated role of synaptic plasticity [10], monkey cell recording suggests that changes in dynamic network states of caudate and lateral PFC neurons may also be critical for learning [11]. These regions exhibit sustained activity during simple
Reversal learning
Deficits in reversal learning, the ability to switch responding to a previously non-reinforced stimulus, are present not just in chronic illness but also in the early stages of disorders such as Parkinson's disease (PD) [20], schizophrenia and bipolar disorder [21•, 22•]. Catecholamines, specifically cortico-striatal DA, and indoleamines, in particular orbitofrontal (OFC) 5-HT, are considered critical for this type of flexibility. Reversal learning is impaired not only following infusions of a
Attentional set-shifting
In contrast to reversal learning which pertains to a specific exemplar, attentional set-shifting (extra-dimensional shifting (EDS)) refers to switching between higher order modalities (e.g. from lines to shapes, from texture to odour) on the basis of feedback. The neural correlates of attentional set-shifting in humans were elucidated using fMRI [36••], associating OFC with reversal learning and ventrolateral PFC (vlPFC) with EDS. In the rat, a well-controlled lesion study of strategy shifting
Task switching
Task switching addresses cognitive flexibility divorced from the ability to adapt ongoing behaviour to feedback, hence unconfounding it from the learning mechanisms that underlie attentional set-shifting. It involves switching between rules that determine well-learned stimulus and response (S–R) mappings, on the basis of cues. Task switching deficits are seen in neurodegenerative disease such as PD [45•] and psychiatric illness, for example, OCD [46], since frontoparietal areas and the basal
Conclusions
Recent advances in understanding the contributions of neurons in the striatum, orbital and prefrontal regions of the frontal cortex to learning and flexibility reflect developments in methods and techniques in animal and human experimentation. The roles of cortico-striatal DA and orbitofrontal 5-HT in basic forms of learning and flexibility have received growing support, and their contribution to different cognitive subprocesses has been relatively well demarcated. Conversely, the NA-ergic
References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as:
• of special interest
•• of outstanding interest
Acknowledgements
This work was supported by a Wellcome Trust Programme Grant (076274/Z/04/Z) to TWR, BJ Everitt, AC Roberts and BJ Sahakian and was completed at the University of Cambridge Behavioural and Clinical Neuroscience Institute supported by a joint award from the Medical Research Council and the Wellcome Trust (G00001354). AAK holds an Isaac Newton fellowship and is additionally supported by the Parkinson's Disease Society. GKM is supported by the Medical Research Council and NARSAD (National Alliance
References (56)
- et al.
A neural substrate of prediction and reward
Science
(1997) - et al.
Effect of lesion of cortical dopamine terminals on subcortical dopamine receptors in rats
Nature
(1980) - et al.
Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior
Proc Natl Acad Sci U S A
(2009) - et al.
Reinforcement and reversal learning in first-episode psychosis
Schizophr Bull
(2008) - et al.
Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior
Nat Neurosci
(2005) - et al.
Decoding sequential stages of task preparation in the human brain
Neuroimage
(2009) - et al.
Effects of the NMDA antagonist ketamine on task-switching performance: evidence for specific impairments of executive control
Neuropsychopharmacology
(2006) - et al.
Mechanisms of cognitive set flexibility in Parkinson's disease
Brain
(2001) - et al.
Parallel organization of functionally segregated circuits linking basal ganglia and cortex
Annu Rev Neurosci
(1986) - et al.
Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum
J Neurosci
(2000)
Evidence for segregated and integrative connectivity patterns in the human basal ganglia
J Neurosci
Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning
J Neurosci
Functional integration across a gradient of corticostriatal channels controls UP state transitions in the dorsal striatum
Proc Natl Acad Sci U S A
Functional connectivity of human striatum: a resting state FMRI study
Cereb Cortex
Basal ganglia functional connectivity based on a meta-analysis of 126 positron emission tomography and functional magnetic resonance imaging publications
Cereb Cortex
Temporal difference models and reward-related learning in the human brain
Neuron
Triplets of spikes in a model of spike timing-dependent plasticity
J Neurosci
Learning substrates in the primate prefrontal cortex and striatum: sustained activity related to successful actions
Neuron
Transient and selective overexpression of D2 receptors in the striatum causes persistent deficits in conditional associative learning
Proc Natl Acad Sci U S A
Transient and selective overexpression of dopamine D2 receptors in the striatum causes persistent abnormalities in prefrontal cortex functioning
Neuron
Reversal of antipsychotic-induced working memory deficits by short-term dopamine D1 receptor stimulation
Science
Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning
Science
Reward-predictive cues enhance excitatory synaptic strength onto midbrain dopamine neurons
Science
Dichotomous dopaminergic control of striatal synaptic plasticity
Science
Probabilistic reversal learning is impaired in Parkinson's disease
Neuroscience
Discrimination learning, reversal, and set-shifting in first-episode schizophrenia: stability over six years and specific associations with medication type and disorganization syndrome
Biol Psychiatry
Dissociation in prefrontal cortex of affective and attentional shifts
Nature
Cognitive inflexibility after prefrontal serotonin depletion is behaviorally and neurochemically specific
Cereb Cortex
Cited by (319)
Neonatal hypoxia impairs serotonin release and cognitive functions in adult mice
2024, Neurobiology of DiseaseHuman creativity: Functions, mechanisms, and social conditioning
2024, Advances in Experimental Social PsychologyActivity in the Dorsomedial Striatum Underlies Serial Reversal Learning Performance Under Probabilistic Uncertainty
2023, Biological Psychiatry Global Open ScienceRole of adenosine A2A receptors in hot and cold cognition: Effects of single-dose istradefylline in healthy volunteers
2023, European Neuropsychopharmacology