The interaction between reinforcement and inhibitory control in ADHD: A review and research guidelines
Introduction
Attention-deficit/hyperactivity disorder (ADHD) is a common child and adolescent developmental disorder with prevalence rates of 5–10% (Scahill & Schwab-Stone, 2000). ADHD is mainly characterized by age-inappropriately high levels of inattention and/or impulsivity and hyperactivity. Three symptomatic subtypes are currently distinguished: ADHD-inattentive subtype, ADHD-hyperactive/impulsive subtype, and ADHD-combined subtype (American Psychiatric Association, 2013).
Theoretical causal models of ADHD have suggested that deficits in inhibitory control lead to secondary impairments in other cognitive-control functions, resulting in inattention, hyperactivity, and impulsivity (Barkley, 1997). This theory has triggered a burgeoning literature of behavioral and functional magnetic resonance imaging (fMRI) studies. Findings indicate that ADHD is indeed associated with poor cognitive control, particularly inhibitory control as measured with the stop task (Chamberlain et al., 2011, Lijffijt et al., 2005, Lipszyc and Schachar, 2010, Logan et al., 1997, Logan and Sergeant, 1998, Willcutt et al., 2005). Generally, individuals with ADHD have slower stop signal reaction times (SSRT), suggesting that it takes them longer to inhibit prepotent responses. The inhibitory deficit in ADHD is associated with both structural and functional abnormalities in fronto-striatal and fronto-parietal neural circuitries often revealing hypoactivation in prefrontal areas during stop or go/no-go tasks as compared to typical populations (Castellanos and Proal, 2012, Cortese et al., 2012, De La Fuente et al., 2013, Hart et al., 2013, Paloyelis et al., 2007, Valera et al., 2007, Valera and Seidman, 2005). Despite the established relation between ADHD and cognitive control deficits, meta-analyses have shown that effect sizes are small to moderate: poor inhibitory control is typical of only 50% of all patients with ADHD (Logan and Sergeant, 1998, van Mourik et al., 2005, Nigg et al., 2005, Willcutt et al., 2005).
In addition to impairments in cognitive control, altered reinforcement sensitivity is considered one of the important deficits in ADHD (Castellanos et al., 2006, Castellanos and Tannock, 2002, Douglas and Parry, 1994, Sonuga-Barke, 2002, Sonuga-Barke, 2003, Sonuga-Barke, 2005, Sonuga-Barke and Fairchild, 2012, Tripp and Wickens, 2012). Reinforcement sensitivity may be defined as a tendency that varies across individuals and is often measured with questionnaires. As such, it is assumed that higher levels of reward sensitivity will be correlated with larger benefits in task performance when this is reinforced with rewards (Luman, van Meel, Oosterlaan & Geurts, 2012). Importantly, Fosco, Hawk, Rosch, and Bubnik (2015) tested this assumption and demonstrated that higher questionnaire-based reward sensitivity is indeed associated with larger increases in task performance when it is rewarded compared to when it is not rewarded. Research using questionnaire-based measures of reinforcement sensitivity has demonstrated that indeed, children and adolescents with ADHD are more sensitive to rewards than controls (Fosco et al., 2015, Luman et al., 2012). However, in experimental research, findings are less conclusive: while there is some evidence that the positive effects of reinforcers on task performance are stronger in those with ADHD than controls, psychophysiological research suggests decreased reinforcement sensitivity at that level of analysis in individuals with ADHD (see Luman et al. (2005) for a review).
A more consistent finding of altered reinforcement sensitivity in ADHD is a relatively strong preference for small immediate rewards compared to larger delayed rewards (Antrop et al., 2006, Barkley et al., 2001, Bitsakou et al., 2009, Demurie et al., 2012, Luman et al., 2005, Marco et al., 2009, Scheres et al., 2010, Sergeant, 2000, Shiels et al., 2009, Solanto et al., 2001, Sonuga-Barke et al., 1992, Tripp and Alsop, 2001, Wilson et al., 2011 but also see Plichta et al., 2009, Scheres et al., 2006, Solanto et al., 2007, Wilbertz et al., 2013). In terms of brain activation during choices between small immediate and larger delayed rewards, only two studies have been published so far (Plichta et al., 2009, Rubia et al., 2009a). Plichta et al. (2009) showed ventral striatal hypoactivation in adults with ADHD while deciding between smaller sooner and larger later reward options. Additionally, they reported hyperactivation of the dorsal caudate nucleus and amygdala in those with ADHD when the soon option was not immediate. These findings suggest reduced neural reward processing in individuals with ADHD, and are consistent with the delay aversion theory (Sonuga-Barke et al., 1992). The second study (Fabiano et al., 2009) found that preferences for small immediate rewards were correlated with hyperactivity symptoms. When contrasting delayed vs. immediate choices, adults with ADHD showed hypoactivation in orbital and inferior prefrontal cortices, putamen, thalamus, inferior parietal lobe, posterior cingulate/precuneus, and cerebellum. Given the involvement of these regions in other processes including temporal processes (e.g. temporal discounting), inhibition, and attention, Rubia, Halari, Christakou et al. (2009) suggested that a combination of such skills and abilities is needed to wait for large delayed rewards, and that these may be compromised in ADHD.
Secondly, a recent meta-analysis (Plichta & Scheres, 2014; but see von Rhein et al., 2015, Plichta and Scheres, 2014) of functional MRI studies has revealed medium-sized hypoactivation of the ventral striatum during anticipation of potential monetary reward in adolescents and adults with ADHD as measured with the monetary incentive delay (MID) task (Knutson, Westdorp, Kaiser, & Hommer, 2000). In contrast, there are very few fMRI studies on reward outcome in relation to ADHD (Paloyelis et al., 2012, Strohle et al., 2008, Wilbertz et al., 2012). These findings are interesting though mixed regarding the role of ventral striatum and orbitofrontal cortex during reward outcome in individuals (adolescents and adults) with ADHD. More research is needed to determine the relation between ADHD and reward outcome processing at different developmental stages, and studies examining both anticipation and outcome of rewards within the same paradigm and sample are of particular interest (see von Rhein et al., 2015).
Taken together, although ADHD is marked by heterogeneity and multiple mechanisms are involved, there is accumulating evidence supporting the notion of altered reinforcement effects in ADHD. Additionally, a link between ADHD and inhibitory control deficits had already been established. The importance of both processes in ADHD is reflected in the roles both play in behavioral interventions: desired behaviors (including inhibitory control) are often trained by the use of reinforcers (Antshel et al., 2011, Daley et al., 2014, DuPaul and Stoner, 2014, Fabiano et al., 2009a). Together, altered reinforcement effects and inhibitory control deficits may form one of the fundamental mechanisms for the diversity of ADHD symptoms.
Inhibitory control and reinforcement effects can be studied in isolation as has been done in the majority of the studies described above. However, goal directed behavior often involves aiming to achieve a positive outcome (reward) or avoiding a negative outcome (e.g. punishment) in daily life situations. Therefore, reinforcement may be expected to play a vital role in inhibitory control, requiring adequate integration of these two functions in order to serve appropriate goal directed behavior. For example, a child may be required to pay attention during a school lecture and inhibit the temptation to talk to friends during to get better grades. In this example, reward (getting good grades) increases the likelihood of someone to demonstrate cognitive control (in this case, inhibition of interacting with friends). Thus, one way in which reinforcement can interact with cognitive control is that incentives lead to behavioral improvements (ameliorating effects). Another, much less frequently studied manner in which reinforcement and inhibitory control interact is in the opposite way, i.e., reinforcement may impair inhibitory control. Specifically, stimuli or responses to stimuli that have resulted in a reward or the avoidance of punishment will be harder to avoid or inhibit than (responses to) stimuli that have not been associated with reinforcement. For example, a child may find it more difficult to stop being the class clown if his/her behavior resulted in the approval of peers than if it did not. The impairing effects of reinforcement or motivational significance on cognitive control and its neural correlates have been classified as “hot” forms of cognitive control by Metcalfe and Mischel (1999) and later fine-tuned by Zelazo and Müller (2002). This is in contrast to a “cool” form of cognitive control which is purely abstract, such as measured by the stop task (Logan et al., 1997).
Given that reinforcement plays an important role in inhibitory control in various ways, it is encouraging that the combination of inhibitory control and reinforcement has been increasingly incorporated in theoretical models of multiple causal mechanisms of ADHD (Castellanos and Tannock, 2002, Nigg, 2003, Nigg and Casey, 2005, Nigg et al., 2005, Sagvolden et al., 2005b, Sonuga-Barke et al., 2010, Sonuga-Barke, 2002, Sonuga-Barke, 2003, Sonuga-Barke, 2005). Specifically, the dual-pathway model suggests an impaired dorsolateral cortical-striatal brain circuitry to be associated with (cool) cognitive control impairments, whereas dysfunctions in the medial and orbital prefrontal-ventral striatal circuits are linked to altered reinforcement sensitivity. Although earlier versions of this model primarily viewed these pathways as independent, more recently, Sonuga-Barke, Sergeant, Nigg, and Willcutt (2008) proposed that cognitive control and reinforcement interact, despite the fact that strong preferences for immediate rewards and poor cognitive control are distinguishable to some extent in their relation to ADHD (Solanto et al., 2001). Thus, ADHD has progressively come to be viewed as a multi-systemic disorder with diverse neuropsychological profiles (i.e. altered reinforcement effects, and/or inhibitory/cognitive control deficits) (Nigg and Casey, 2005, Sonuga-Barke et al., 2010, de Zeeuw et al., 2012).
Nonetheless, empirical studies examining the conjunction of reinforcement and inhibitory control pathways in ADHD are still sparse. Although a number of behavioral studies have examined the ameliorating effects of reinforcement on inhibition, impairing effects of reinforcement on inhibitory control have only been investigated in ADHD in one study (Wodka et al., 2007). Additionally, functional neuroimaging studies on the integration of reinforcement and cognitive control have only just started and are limited to the ameliorating effects of reinforcement on cognitive control. We argue that more research is needed in which the integration between these two processes is examined in individuals with ADHD, because more variance in the symptoms of ADHD may be explained when measuring the conjunction of these two important functions than when measuring each in isolation.
Section snippets
Approach
While other executive functions are impaired in ADHD as well, inhibitory control deficits have been shown to most robustly differentiate individuals with ADHD from controls (e.g., Willcutt et al., 2005). For this reason, as well as for the sake of brevity, the focus of this review will be on the integration of inhibitory control and altered reinforcement effects. We will (a) provide a comprehensive qualitative review of behavioral studies in which the interaction between these was measured. We
A comparison of methodological details
All studies in this review included non-medicated, clinically diagnosed children and adolescents with ADHD and healthy control groups in the age range of 6–18 years. To our knowledge, there were no studies with adult ADHD groups. The main dependent variables for response inhibition are SSRT for the stop task, and false alarm rate (proportion of no-go trials to which a participant responded) for the go/no-go task. As for response execution, mean reaction time (RT) and RT variability are the main
Functional neuroimaging research
Neuroimaging techniques can be applied to provide insight into the neural correlates associated with reinforcement and inhibition interactions. However, to our knowledge, there are no functional neuroimaging studies on how reward affects inhibition directly in ADHD. Nonetheless, there are two studies in which such effects were addressed indirectly, the findings of which will be discussed here. Secondly, in order to set up guidelines for future functional neuroimaging research in individuals
Theoretical models: implications for studying the integration of reinforcement and cognitive control in ADHD
In this section we will briefly describe 3 theoretical models of ADHD and some of their implications for future behavioral work on inhibition–reinforcement interactions in ADHD. Next, we will briefly describe 3 neurobiological models on the integration of reinforcement and cognitive control, and how these may aid in the formulation of neural hypotheses for future functional neuroimaging ADHD studies.
Disorder specificity
It should be noted that hot cognitive control abnormalities in ADHD may be associated with comorbid behavioral disorders such as oppositional defiant disorder and conduct disorder. This notion arises from studies that have employed a rewarded continuous performance task (a rewarded sustained attention task) combined with fMRI (Rubia, Smith et al., 2009). Here, the rewarded target trials were contrasted against the neutral target trials and abnormal activation of orbital frontal areas was only
Conclusion
Reinforcement and cognitive control interactions reflect daily life conditions and this interplay is increasingly studied in typically developing populations as well as in psychiatric disorders. The current review and meta-analyses demonstrated that youth with ADHD benefited more from reinforcement contingencies than healthy controls on inhibitory control tasks. Meta-analyses further demonstrated that youth with ADHD may normalize inhibitory control during reinforcement to the baseline
Author disclosure
This work was supported by a VIDI grant, project number 016.105.363, of The Netherlands Organization for Scientific Research (NWO) to AS. The authors have no conflict of interest.
Acknowledgements
This work was supported by a VIDI grant, project number 016.105.363, of The Netherlands Organization for Scientific Research (NWO) to AS.
References (142)
- et al.
Inhibition and impulsivity: behavioral and neural basis of response control
Progress in Neurobiology
(2013) - et al.
Delay aversion in attention deficit/hyperactivity disorder: an empirical investigation of the broader phenotype
Neuropsychologia
(2009) - et al.
Functional neuroimaging of attention-deficit/hyperactivity disorder: a review and suggested future directions
Biological Psychiatry
(2005) - et al.
Anterior cingulate cortex dysfunction in attention-deficit/hyperactivity disorder revealed by fMRI and the counting Stroop
Biological Psychiatry
(1999) - et al.
Large-scale brain systems in ADHD: beyond the prefrontal-striatal model
Trends in Cognitive Sciences
(2012) - et al.
Characterizing cognition in ADHD: beyond executive dysfunction
Trends in Cognitive Sciences
(2006) - et al.
Varieties of attention-deficit/hyperactivity disorder-related intra-individual variability
Biological Psychiatry
(2005) - et al.
Translational approaches to frontostriatal dysfunction in attention-deficit/hyperactivity disorder using a computerized neuropsychological battery
Biological Psychiatry
(2011) - et al.
A review of fronto-striatal and fronto-cortical brain abnormalities in children and adults with attention deficit hyperactivity disorder (ADHD) and new evidence for dysfunction in adults with ADHD during motivation and attention
Cortex
(2012) - et al.
A meta-analysis of behavioral treatments for attention-deficit/hyperactivity disorder
Clinical Psychology Review
(2009)
Effects of motivation and medication on electrophysiological markers of response inhibition in children with attention-deficit/hyperactivity disorder
Biological Psychiatry
The primate basal ganglia: parallel and integrative networks
Journal of Chemical Neuroanatomy
FMRI visualization of brain activity during a monetary incentive delay task
NeuroImage
Impact of emotional salience on episodic memory in attention-deficit/hyperactivity disorder: a functional magnetic resonance imaging study
Biological Psychiatry
Atypical prefrontal connectivity in attention-deficit/hyperactivity disorder: pathway to disease or pathological end point?
Biological Psychiatry
The impact of reinforcement contingencies on AD/HD: a review and theoretical appraisal
Clinical Psychology Review
Causal heterogeneity in attention-deficit/hyperactivity disorder: do we need neuropsychologically impaired subtypes?
Biological Psychiatry
Interactions between cognition and motivation during response inhibition
Neuropsychologia
Developmental changes in brain function underlying the influence of reward processing on inhibitory control
Developmental Cognitive Neuroscience
How do emotion and motivation direct executive control?
Trends in Cognitive Sciences
Arrested development? Reconsidering dual-systems models of brain function in adolescence and disorders
Trends in Cognitive Sciences
Ventral–striatal responsiveness during reward anticipation in ADHD and its relation to trait impulsivity in the healthy population: a meta-analytic review of the fMRI literature
Neuroscience and Biobehavioral Reviews
Neural hyporesponsiveness and hyperresponsiveness during immediate and delayed reward processing in adult attention-deficit/hyperactivity disorder
Biological Psychiatry
Neurocognitive endophenotypes of impulsivity and compulsivity: towards dimensional psychiatry
Trends in Cognitive Sciences
Methylphenidate normalises activation and functional connectivity deficits in attention and motivation networks in medication-naive children with ADHD during a rewarded continuous performance task
Neuropharmacology
Epidemiology of ADHD in school-age children
Child and Adolescent Psychiatric Clinics of North America
Temporal and probabilistic discounting of rewards in children and adolescents: effects of age and ADHD symptoms
Neuropsychologia
Striatal dopamine and the interface between motivation and cognition
Frontiers in Psychology
ADHD and delay aversion: the influence of non-temporal stimulation on choice for delayed rewards
Journal of Child Psychology and Psychiatry
Advances in understanding and treating ADHD
BMC Medicine
Diagnostic and statistical manual of mental disorders
Stop-signal inhibition disrupted by damage to right inferior frontal gyrus in humans
Nature Neuroscience
Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD
Psychological Bulletin
Executive functioning, temporal discounting, and sense of time in adolescents with attention deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD)
Journal of Abnormal Child Psychology
Anterior cingulate cortex and symptom severity in attention-deficit/hyperactivity disorder
Journal of Abnormal Psychology
Impulsive action but not impulsive choice determines problem gambling severity
PloS One
Reinforcement enhances vigilance among children with ADHD: comparisons to typically developing children and to the effects of methylphenidate
Journal of Abnormal Child Psychology
The adolescent brain
Annals of the New York Academy of Sciences
Quantitative morphology of the caudate nucleus in attention deficit hyperactivity disorder
American Journal of Psychiatry
Neuroscience of attentiondeficit/hyperactivity disorder: the search for endophenotypes
Nature Reviews. Neuroscience
Positive affect versus reward: emotional and motivational influences on cognitive control
Frontiers in Psychology
Statistical power analysis for the behavioral sciences
Toward systems neuroscience of ADHD: a meta-analysis of 55 fMRI studies
American Journal of Psychiatry
Understanding adolescence as a period of social–affective engagement and goal flexibility
Nature Reviews Neuroscience
Behavioral interventions in attention-deficit/hyperactivity disorder: a meta-analysis of randomized controlled trials across multiple outcome domains
Journal of the American Academy of Child and Adolescent Psychiatry
A review of attention-deficit/hyperactivity disorder from the perspective of brain networks
Frontiers in Human Neuroscience
Distinct age-related differences in temporal Discounting and Risk Taking in Adolescents and Young Adults
Child Development
Deficits in cognitive control, timing and reward sensitivity appear to be dissociable in ADHD
PloS One
Common alterations in sensitivity to type but not amount of reward in ADHD and autism spectrum disorders
Journal of Child Psychology and Psychiatry
Temporal discounting of monetary rewards in children and adolescents with ADHD and autism spectrum disorders
Developmental Science
Cited by (39)
Brain tissue iron neurophysiology and its relationship with the cognitive effects of dopaminergic modulation in children with and without ADHD
2023, Developmental Cognitive NeuroscienceEvent rate effects on children with attention-deficit/hyperactive disorder: Test predictions from the moderate brain arousal model and the neuro-energetics theory using the diffusion decision model
2022, Research in Developmental DisabilitiesCitation Excerpt :Attention-deficit/hyperactive disorder (ADHD) is a neuropsychiatric disorder characterised by impulsive, hyperactive, and inattentive symptoms that lead to social and academic impairments (Thomas et al., 2015). Extensive research has been conducted on the disinhibition reaction of children with ADHD, and the mean reaction time (MRT) and error rate have been widely used as indicators of high-order inhibition control (Barkley, 1997; Chmielewski et al., 2019; Ma et al., 2016). Compared with typical developmental children (TDC), children with ADHD showed more variability, performed worse, had longer MRT, and made more mistakes (Metin & Krebs, 2015a; Metin et al., 2012; Pievsky & McGrath, 2018).
Exposure to ambient air pollution and executive function among Chinese primary schoolchildren
2020, International Journal of Hygiene and Environmental HealthCitation Excerpt :Inhibitory control, another key component of executive function, regulates the self-control of resisting temptations and acting impulsively, and selective attention (Diamond and Ling, 2016). Impaired inhibitory control has been related to several mental health problems such as addictive behaviors (Jentsch and Pennington, 2014) or attention-deficit/hyperactivity disorder (Ma et al., 2016). In this study, PM2.5 exposure was associated with higher time interference score in the Stroop test.