ReviewUsing task performance to inform treatment planning for youth with ADHD: A systematic review
Introduction
Attention Deficit Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder characterized by clinically significant symptoms of inattention and hyperactivity/impulsivity (American Psychiatric Association, 2013). It is one of the most common mental disorders diagnosed in children: current evidence indicates that approximately 7% of school-age youth meet criteria for the disorder (Visser et al., 2013). Children and adolescents with ADHD often experience impairment in several life domains (for reviews, see Hoza, 2007, Loe and Feldman, 2007). In academics, youth with ADHD tend to receive lower grades and perform more poorly on standardized tests than their peers (Massetti et al., 2008, Molina et al., 2009). They are also at increased risk for grade retention and dropping out before earning their diploma (Barbaresi et al., 2007, Kent et al., 2011). Socially, youth with ADHD are at an increased risk of peer rejection and have difficulty using social skills consistently (Bagwell et al., 2001, Landau and Moore, 1991). Additionally, youth with ADHD are at increased risk for poor health outcomes, such as substance abuse and obesity (Cortese et al., 2008, Disney et al., 1999, Molina et al., 2013). Given the array of impairments that are commonly associated with ADHD, accurate and informative assessment is paramount to choosing appropriate treatment targets.
Numerous recommendations have been published regarding evidence-based strategies for assessing ADHD in youth (American Academy of Pediatrics, 2011, Pelham et al., 2005, Pliszka and AACAP Work Group on Quality Issues, 2007). Several unifying themes emerge among these recommendations, including the importance of gathering the perspectives of multiple observers (i.e., parents and teachers), evaluating for common comorbid conditions and alternative diagnoses, and evaluating the impairment caused by the individual's symptoms. However, the recommendations are less explicit regarding the specific methodologies of assessment. Some psychometrically validated tools, including interview protocols and behavior rating scales have a wealth of evidence supporting their utility and are considered to be vital parts of an ADHD assessment (Pelham et al., 2005). However, the evidence is much more mixed regarding the utility of other common assessment instruments. For example, published recommendations do not make strong statements about the use of neuropsychological tasks in evidence-based assessment, instead considering them as optional additions to assessment protocols (Pliszka and AACAP Work Group on Quality Issues, 2007) or refraining from commenting on their use (American Academy of Pediatrics, 2011, Pelham et al., 2005). However, these tasks are frequently used in psychological assessments, including those where ADHD is a possible diagnosis (Rabin, Barr, & Burton, 2005).
One reason these tasks remain a component of many ADHD assessment batteries is the fact that they purportedly measure the cognitive deficits thought to underlie the disorder. Major theories of ADHD have emphasized core deficits in the cognitive processes associated with attention and executive function (EF), which includes processes such as behavioral inhibition, working memory, and organization and planning skills (Anderson, 2002, Pennington and Ozonoff, 1996). Barkley's (1997) theory of ADHD posited that symptoms of ADHD are largely due to deficits in EF, and therefore a diagnosis of ADHD should be consistently associated with deficits in these abilities. Other theories of ADHD have also argued for a prominent role of these deficits (e.g., Rapport et al., 2001, Sonuga-Barke, 2005). Further, a wealth of empirical evidence demonstrates significant group differences between samples of youth with and without ADHD on neuropsychological tasks that tap attention and EF abilities (for a review, see Seidman, 2006). Therefore, these tasks may provide useful clinical information about the underlying deficits associated with an individual's ADHD symptoms.
The neuropsychological tasks most frequently used in ADHD assessment batteries are those thought to measure sustained attention abilities. The Conners' Continuous Performance Test (CPT; Conners, 2000) and the Test of Variables of Attention (TOVA; Greenberg, Kindschi, Dupuy, & Hughes, 1994) are two of the more prominent tests in use today. These tests are designed to evaluate sustained attention through the completion of a prolonged task. Other attention tests, such as the Tests of Everyday Attention for Children (TEA-Ch; Manly et al., 2001) attempt to capture attentional abilities in the context of real-world tasks. With respect to EF, numerous neuropsychological tasks are available depending on the specific cognitive process a clinician or researcher wishes to examine. Common EF tasks used in ADHD assessments include the Wisconsin Card Sort Task (WCST; Heaton, 1981) for measuring task shifting, the Tower of London (Shallice, 1982) and Rey-Osterrieth Complex Figure (ROCF; Osterrieth, 1944) for measuring planning abilities, the Stroop task (Stroop, 1935) for measuring cognitive flexibility, and the Stop-signal task (Logan, 1994) for measuring behavioral inhibition. Additionally, subtests of other cognitive batteries are also used to evaluate EF abilities. For example, the Digit Span tasks of the Wechsler Intelligence Scales for Children (Wechsler, 2003)are often used as assessments of working memory, and research has indicated that performance on these tests is often poorer than expected in youth with ADHD (Mayes & Calhoun, 2006). As tasks have been developed and refined, comprehensive batteries, such as the Delis-Kaplan Executive Function System (D-KEFS; Delis, Kaplan, & Kramer, 2001) and the Cambridge Neuropsychological Testing Automated Battery (CANTAB; Fray, Robbins, & Sahakian, 1996), have emerged that include a range of tasks and have normative data available for children and adolescents.
Neuropsychological tasks often generate multiple scores and may be interpreted in different ways depending upon theories regarding the construct of interest. This is especially true for EF, for which multiple theoretical models are available in the literature that vary in their organization of the specific abilities within the broader EF construct (Jurado & Rosselli, 2007). For example, some theoretical models of EF divide abilities into two domains: “hot” and “cool” EF processes (Zelazo & Carlson, 2012). “Hot” EF processes are those that manage in-the- moment behaviors and typically occur in the context of emotionally valent situations, such as inhibition and emotion regulation. In contrast, “cool” EF processes are those oriented more towards future behaviors, such as planning, organization and self-monitoring of behavior. However, other models use more narrowly-defined categories to organize the abilities encompassed by EF. For example, Anderson's (2002) model of the development of EF contains four distinct subdomains: cognitive flexibility (e.g., shifting attention, working memory), behavioral regulation (e.g., inhibition, self-monitoring), information processing (e.g., fluency, processing speed), and goal setting (e.g., planning, organization). This variance in the theorized relationships between cognitive processes has led to differing interpretations of performance.
Some studies of youth with ADHD have focused on performance for each individual task (Miller and Hinshaw, 2010, Tseng and Gau, 2013), which allows more direct links to be made between a specific task and other variables of interest. However, relying on a single task carries several significant risks. First, there is often disagreement about the specific cognitive ability that a task measures. For instance, the Digit Span task is often conceptualized as a working memory task, but a growing body of research suggests that it may tap other processes, such as sustained attention, depending on which version of the task (e.g., forward vs. backward) is used (Zokaei, Burnett Heyes, Gorgoraptis, Budhdeo, & Husain, 2015). A related issue is the recognition that many neuropsychological tasks require the use of multiple cognitive functions; this is sometimes referred to as “task impurity” (Miyake et al., 2000). As a result, research has trended toward a more holistic interpretation of task performance. Some studies have used poor performance on multiple individual tasks as evidence of a general deficit in EF (Biederman et al., 2004, Lambek et al., 2010). Other studies have used latent factor analysis to identify common constructs measured by similar tasks within a full battery (Miyake et al., 2000). However, these studies have varied on the number and type of factors identified, which is largely dependent on the specific tasks used in each study. In samples of youth with ADHD, latent factor analyses have ranged from identifying a single factor (Huang-Pollock, Mikami, Pfiffner, & McBurnett, 2009) to as many as five factors (Willcutt, Pennington, Olson, Chhabildas, & Hulslander, 2005).
In line with the major theories regarding ADHD and based upon findings showing ADHD and comparison group differences, researchers began to investigate the utility of neuropsychological task performance to diagnose ADHD. Some studies have found that tasks may have diagnostic utility for children with ADHD (e.g., Berlin et al., 2004, Holmes et al., 2010). However, overall the evidence has been much more mixed than was initially expected. Common critiques include difficulty distinguishing between youth with ADHD and youth with other psychological disorders (e.g., McGee et al., 2000, Savitz and Jansen, 2003) and high rates of false negative results (e.g., Doyle, Biederman, Seidman, Weber, & Faraone, 2000). Articles synthesizing these studies have suggested the current state of evidence is not sufficient to recommend the use of neuropsychological tasks for diagnosing ADHD in children and adolescents. For example, a review by Nichols and Waschbusch (2004) found that several tasks, including continuous performance tasks and the Stop-signal task, have not demonstrated adequate diagnostic validity for use in a clinical manner. Further, a review by Sergeant, Geurts, and Oosterlaan (2002) and a Willcutt, Doyle, Nigg, Faraone, and Pennington (2005) meta-analysis found that deficits in EF abilities – as indicated by task performance – are not universal in individuals with ADHD, nor is there a single pattern of EF abilities that consistently identifies those with the disorder. Together, these findings suggest significant limits on the effectiveness of neuropsychological tasks as diagnostic tools.
Although neuropsychological task performance may not yet be reliably useful in the diagnosis of ADHD, it may still be a useful part of a comprehensive ADHD assessment. Specifically, task performance may provide valuable information regarding the steps that follow diagnosis: recommendations for treatment and potential targets of intervention. As mentioned previously, children and adolescents with ADHD are at an increased risk for a variety of poor outcomes. However, these impairments are not universal, guaranteed outcomes. On the contrary, some youth with ADHD exhibit impairment in one aspect of functioning but not others, while other youth exhibit impairment across multiple behaviors and domains. Further, trajectories of impairment vary significantly across time for youth with ADHD. Given this heterogeneity, it would be clinically useful to be able to provide caregivers with information about which domains of impairment are most likely to be problematic long-term. If neuropsychological tasks have utility to identify current or future areas of impairment, they could be included in ADHD assessments to provide clinicians with guidance in choosing treatment targets. Empirical evidence from studies examining associations between task performance and attention problems broadly provide preliminary support for this possibility. For example, Thorell (2007) found that task performance mediated the relationship between ADHD symptom severity and academic functioning in a general sample of Kindergarten students. Additionally, evidence from combined samples of youth with and without ADHD have found that performance on tasks measuring working memory, planning, and inhibition are associated with social functioning (Bunford et al., 2015, Hilton et al., 2016, Kofler et al., 2011).
Besides the possibility of identifying treatment targets, neuropsychological tasks could also provide insights into potential treatment modalities. An array of pharmacologic and behavioral therapies are available as efficacious methods of treatment for ADHD (for reviews, see Sonuga-Barke et al., 2013; Wigal et al., 1999). However, both therapy domains carry limitations. Pharmacologic interventions may be accompanied by adverse side effects and non-response rates as high as one in three youth have been reported, and behavior therapy requires a significant time and resource commitment for families (Bussing et al., 2012, Hodgkins et al., 2012). Given the limitations associated with each treatment modality, ideally assessment feedback sessions would include meaningful recommendations about the likely efficacy of treatment options. Some brain imaging research has shown early promise for helping identify children and adults who may respond to stimulant medication (Arns et al., 2008, Schweitzer et al., 2003), but these techniques have significant feasibility issues in daily practice. Neuropsychological evaluations, in contrast, are already prevalent in many clinical settings.
Importantly, prior research in typically developing populations has found that performance on neuropsychological tasks is associated with many outcomes that are of particular concern for youth with ADHD. For example, multiple studies have linked neuropsychological task performance in general samples of children and adolescents to performance on standardized testing across academic subjects (Latzman et al., 2010, Waber et al., 2006) and to competence in specific academic skills (Bull and Scerif, 2001, Hooper et al., 2002). Similarly, research focused on social functioning in general populations has found that neuropsychological task performance in childhood – especially on tasks measuring executive functioning – is predictive of social competence later in life (Riggs, Jahromi, Razza, Dillworth-Bart, & Mueller, 2006). Finally, research examining health outcomes has found that neuropsychological task performance is predictive of substance use (Nigg et al., 2006, Tarter et al., 2003) and obesity (Liang et al., 2014, Maayan et al., 2011) in community-based samples of children and adolescents.
Evidence is also available from non-ADHD samples to support the assertion that task performance may be useful for making recommendations about treatment modality. Indeed, there is a substantial body of research focused on predicting response to pharmacologic treatment for depressive disorders, which have also been associated with deficits in EF (Snyder, 2013). Specifically, numerous studies with adult samples have examined predictors of treatment response to anti-depressant medication, including neuropsychological task performance (for a meta-analysis, see McLennan & Mathias, 2010). For example, Dunkin et al. (2000) found that non-responders to selective serotonin reuptake inhibitors (SSRIs) had poorer pre-treatment performance on the WCST and the Stroop task than responders. Similarly, Gorlyn et al. (2008) found that poor performance on the n-back task, A-not-B reasoning task, and the Trailmaking task was predictive of non-response to SSRI treatment. Other evidence indicates that task performance may also be predictive of non-pharmacologic treatment. For example, recent studies have linked pre-treatment neuropsychological task performance to psychotherapy response for individuals with Obsessive-Compulsive Disorder (OCD). D'alcante et al. (2012) reported that performance on the California Verbal Learning Test (CVLT) – which taps working memory and cognitive flexibility – was associated with better response to Cognitive-Behavior Therapy (CBT) in an adult sample. Flessner et al. (2010) examined pre-treatment task performance in an adolescent population, finding that ROCF performance was predictive of response to treatment of OCD with CBT. Overall, using task performance to predict treatment response appears to hold promise and is worth exploring in other clinical populations. However, research examining the use of neuropsychological tasks to inform treatment recommendations for youth with ADHD. Researchers have recognized this potential and have called for a more in-depth examination (Nichols and Waschbusch, 2004, Seidman, 2006). For these key reasons, the current review was undertaken.
The primary goal of this review is to provide insight into the validity of using neuropsychological task performance to inform treatment planning for children and adolescents with ADHD. Performance on tasks that measure attention or EF abilities are of particular interest in this review, given the strong theoretical ties between these cognitive processes and the course of ADHD (Barkley, 1997, Rapport et al., 2001) and the heterogeneity in task performance across individuals with ADHD (Willcutt et al., 2005). Two major aims guided the review. The first aim was to evaluate evidence for associations between neuropsychological task performance and academic functioning, social functioning, and increased risk of poor health outcomes. The second aim was to evaluate existing evidence for using task performance to predict response to evidence-supported treatments. For both aims, the available evidence was evaluated in the context of the study's methodological strengths and weaknesses (e.g., sample size and cross-sectional v. longitudinal). Further, the method used to interpret task performance (e.g., single tasks, factor scores, failure of a particular number of tasks) and the inclusion/exclusion of relevant covariates, such as general intelligence (i.e., IQ), were evaluated for each study.
Section snippets
Study search procedure
Relevant studies were identified through searches in two major scientific publication databases: PsycINFO and PubMed. The base for each search included terms to identify studies using samples of individuals with ADHD (e.g., “Attention-Deficit Hyperactivity Disorder”, “Attention-Deficit Disorder”, “ADHD”, “ADD”) and terms to identify the use of neuropsychological tasks (e.g., “Neuropsychological Tasks”, “Neuropsychological Performance”, “Executive Function Tests”). Added to these base terms were
Review of available studies
Twenty-five studies met all of the specified inclusion/exclusion criteria and were examined in detail. These studies are summarized in Table 1, Table 2, which are divided by outcome category (i.e., predicting a functional domain or predicting treatment response). Five studies evaluated outcomes in two domains; although the findings for both domains are included in the same table entry, their findings were evaluated in each relevant category.
As expected, an array of tasks was used across
Discussion
The goal of this review was to evaluate whether neuropsychological task performance assessing aspects of attention and EF could be used with children and adolescents with ADHD to inform targets of treatment and response to specific treatment modalities. A total of 25 studies were included in the review, with 17 studies examining potential links with specific treatment domains (e.g., academic functioning, social functioning, and health outcomes) and six examining response to methylphenidate
Conclusions
The current review evaluated the available evidence regarding the ability of neuropsychological task performance to inform treatment targets or treatment response for youth with ADHD. Findings from 25 studies were included that evaluated performance on tasks designed to measure attention and EF abilities. The available evidence suggests that task performance is useful for identifying children and adolescents at risk for impaired academic functioning and for identifying youth who are likely to
Role of funding sources
No financial support was provided or used in the development or preparation of this manuscript.
Contributors
Stephen Molitor conducted the systematic review and identified the final studies included in the review. Both Stephen Molitor and Joshua Langberg contributed to the preparation of the full manuscript, and both authors approved the final manuscript.
Conflict of interest
Both authors declare that they have no conflicts of interest to disclose.
References (108)
- et al.
Investigating the predictive roles of working memory and IQ in academic attainment
Journal of Experimental Child Psychology
(2010) - et al.
Investigating the link between cognitive skills and learning in non-comorbid samples of ADHD and SLI
International Journal of Educational Research
(2014) - et al.
Attention-deficit hyperactivity disorder and problems in peer relations: Predictions from childhood to adolescence
Journal of the American Academy of Child & Adolescent Psychiatry
(2001) - et al.
Willingness to use ADHD treatments: a mixed methods study of perceptions by adolescents, parents, health professionals and teachers
Social Science & Medicine
(2012) - et al.
Impact of executive functions on school and peer functions in youths with ADHD
Research in Developmental Disabilities
(2014) - et al.
The neuropsychological effects of chronic methylphenidate on drug-I boys with attention-deficit/hyperactivity disorder
Biological Psychiatry
(2007) - et al.
Neuropsychological predictors of response to randomized treatment in obsessive–compulsive disorder
Progress in Neuro-Psychopharmacology and Biological Psychiatry
(2012) - et al.
The role of impulsivity in the development of substance use and eating disorders
Neuroscience & Biobehavioral Reviews
(2004) - et al.
Executive dysfunction predicts nonresponse to fluoxetine in major depression
Journal of Affective Disorders
(2000) - et al.
A meta-analysis of working memory impairments in children with attention-deficit/hyperactivity disorder
Journal of the American Academy of Child & Adolescent Psychiatry
(2005)
The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis
Cognitive Psychology
Adolescent substance use in the multimodal treatment study of attention-deficit/hyperactivity disorder (ADHD)(MTA) as a function of childhood ADHD, random assignment to childhood treatments, and subsequent medication
Journal of the American Academy of Child & Adolescent Psychiatry
The MTA at 8 years: prospective follow-up of children treated for combined-type ADHD in a multisite study
Journal of the American Academy of Child & Adolescent Psychiatry
Poor response inhibition as a predictor of problem drinking and illicit drug use in adolescents at risk for alcoholism and other substance use disorders
Journal of the American Academy of Child & Adolescent Psychiatry
Practice parameter for the assessment and treatment of children and adolescents with attention-deficit/hyperactivity disorder
Journal of the American Academy of Child & Adolescent Psychiatry
Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA Division 40 members
Archives of Clinical Neuropsychology
Executive function and the promotion of social–emotional competence
Journal of Applied Developmental Psychology
Neuropsychological functioning in people with ADHD across the lifespan
Clinical Psychology Review
How specific is a deficit of executive functioning for attention-deficit/hyperactivity disorder?
Behavioural Brain Research
Causal models of attention-deficit/hyperactivity disorder: From common simple deficits to multiple developmental pathways
Biological Psychiatry
ADHD: Clinical practice guideline for the diagnosis, evaluation, and treatment of attention-deficit/hyperactivity disorder in children and adolescents
Pediatrics
Diagnostic and Statistical Manual of Mental Disorders (DSM-5)
Assessment and development of executive function (EF) during childhood
Child Neuropsychol.
Neurocognitive and behavioral predictors of math performance in children with and without ADHD
Journal of Attention Disorders
EEG phenotypes predict treatment outcome to stimulants in children with ADHD
Journal of Integrative Neuroscience
Long-term school outcomes for children with attention-deficit/hyperactivity disorder: a population-based perspective
Journal of Developmental & Behavioral Pediatrics
Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD
Psychological bulletin
How well do measures of inhibition and other executive functions discriminate between children with ADHD and controls?
Child Neuropsychology
Impact of executive function deficits and attention-deficit/hyperactivity disorder (ADHD) on academic outcomes in children
Journal of Consulting and Clinical Psychology
Executive functioning as a predictor of children's mathematics ability: Inhibition, switching, and working memory
Developmental Neuropsychology
Attention-deficit/hyperactivity disorder symptoms mediate the association between deficits in executive functioning and social impairment in children
Journal of Abnormal Child Psychology
Body weight and ADHD: examining the role of self-regulation
PloS One
Conner's Continuous Performance Test II. Technical Guide and Software Manual
Attention-deficit/hyperactivity disorder (ADHD) and obesity: a systematic review of the literature
Critical Reviews in Food Science and Nutrition
Delis-Kaplan executive function system (D-KEFS)
Effects of ADHD, conduct disorder, and gender on substance use and abuse in adolescence
Am. J. Psychiatry
Diagnostic efficiency of neuropsychological test scores for discriminating boys with and without attention deficit–hyperactivity disorder
Journal of Consulting and Clinical Psychology
The impact of neuropsychological functioning on treatment outcome in pediatric obsessive–compulsive disorder
Depression and Anxiety
Neuorpsychiatyric applications of CANTAB
International Journal of Geriatric Psychiatry.
Neuropsychological characteristics as predictors of SSRI treatment response in depressed subjects
Journal of Neural Transmission
Co-occurring weight problems among children with attention deficit/hyperactivity disorder: the role of executive functioning
International Journal of Obesity
Test of Variables of Attention continuous performance test
Neurocognitive predictors of substance use disorders and nicotine dependence in ADHD probands, their unaffected siblings, and controls: a 4-year prospective follow-up
Journal of Child Psychology and Psychiatry
Determining medication treatment effects using teacher ratings and classroom observations of children with ADHD: Does neuropsychological impairment matter
Educational and Child Psychology
Executive impairment determines ADHD medication response: implications for academic achievement
Journal of Learning Disabilities
A manual for the Wisconsin card sorting test
Attention problems as a mediator of the relation between executive function and social problems in a child and adolescent outpatient sample
Journal of Abnormal Child Psychology
Amfetamine and methylphenidate medications for attention-deficit/hyperactivity disorder: complementary treatment options
European Child & Adolescent Psychiatry
The diagnostic utility of executive function assessments in the identification of ADHD in children
Child and Adolescent Mental Health
Executive functions in elementary school children with and without problems in written expression
Journal of Learning Disabilities
Cited by (9)
Fifty years of executive control research in attention-deficit/hyperactivity disorder:What we have learned and still need to know
2023, Neuroscience and Biobehavioral ReviewsAttention deficit hyperactivity disorder
2020, Handbook of Clinical NeurologyCitation Excerpt :No neurologic, genetic, neuropsychologic, or behavioral tests have sufficient positive and negative predictive power to accurately classify ADHD with sufficient success to recommend them for clinical diagnosis (Pliszka, 2007). However, cognitive/neuropsychologic assessment can be helpful in specifying the phenotype, deciphering some differential diagnoses, guiding families, and providing valuable information for interventions (Molitor and Langberg, 2017). For most children and adolescents with ADHD, these attention and inhibition difficulties are manifested primarily in settings requiring self-restraint, persistence, and a high level of concentration for relatively uninteresting activities (Subcommittee on Attention-Deficit/Hyperactivity Disorder et al., 2011).
Predicting the Response of Children and Adolescents With ADHD to Methylphenidate: A Systematic Review
2023, Journal of Attention DisordersCognitive and Executive Contributions to Trail-Making Task Performance on Adolescents With and Without Attention Deficit Hyperactivity Disorder
2022, Journal of Attention Disorders“Being ADHD”: a Qualitative Study
2022, Advances in Neurodevelopmental Disorders