Inconsistencies between Subjective Reports of Cognitive Difficulties and Performance on Cognitive Tests are Associated with Elevated Internalising and Externalising Symptoms in Children with Learning-related Problems

Up to 30% of school children receive additional support for learning-related difficulties in the United Kingdom and United States (Department for Education, 2018; National Center for Education Statistics, 2019). Referrals for support typically begin with parent or teacher reports of slow rates of progress in learning, and/or behavioural difficulties such as problems paying attention. In some cases, referrals result in a diagnosis of one or more neurodevelopmental disorders such as attention deficit/hyperactivity disorder (ADHD) and autism following a psychological or psychiatric assessment that draws heavily on subjective reports and observations of a child’s behaviour. Learning difficulties are often linked to deficits in core cognitive domains such as attention and working memory (WM; Follmer, 2018; Holmes et al., 2020; Landerl & Kolle, 2009; Peng & Fuchs, 2016; Peng et al., 2018; Yeniad et al., 2013). However, a common observation in mental health settings is that not every child referred for psycho-educational assessment (based on subjective teacher or parent reports of learning difficulties) performs poorly on performance-based assessments of cognitive ability (e.g. Astle et al., 2019). This means that subjective reports of difficulties can be inconsistent with performance on task-based measures of cognition. In this study, we refer to this as an inconsistent type of cognitive profile (ICP).

The prevalence and aetiology of an ICP has received little attention in child research, despite the potential implications for increased risk of misdiagnoses and inappropriate recommendations for support and intervention. This study sought to estimate the prevalence and symptom profiles of ICPs in a transdiagnostic sample of children and adolescents. Children were first referred to the study by health and education practitioners as experiencing learning-related problems. Their parents then rated their learning, cognitive and mental health difficulties on behaviour rating scales used widely in clinical and educational settings. The children also completed a set of performance-based cognitive tasks. Thus, the recruitment strategy closely resembled the typical routes young people follow for access to educational support and mental health services in the UK.

There is limited information on the prevalence of ICPs among children with learning-related difficulties, but research into the cognitive profiles of individuals with ADHD symptoms provides some insight. ADHD is characterised by elevated levels of inattention and problems inhibiting impulsive and hyperactive behaviours (American Psychiatric Association, 2013). These symptoms are associated with deficits in higher-level cognitive functions such as WM (Castellanos et al., 2006; Lui & Tannock, 2007; Rogers et al., 2011) and behavioural inhibition (Barkley, 1997; Castellanos et al., 2006). Despite this association, studies show that a substantial proportion of children diagnosed with ADHD perform within the typical range on performance-based tests of cognition (Nigg et al., 2005; Willcutt et al., 2005). For example, Solanto et al. (2001) reported that performance-based measures of behavioural impulsivity correctly discriminated 61% of children with ADHD, and performance-based measures of reward sensitivity correctly discriminated 72% of cases. A combination of both captured 88% of cases. Similarly, Nigg et al. (2005) found that 35–50% of children with an ADHD diagnosis had deficits based on performance-based measures of inhibitory cognitive control. The remaining 50–65% were purported to have some alternative aetiology. This is not to say that individuals with ADHD do not have executive function or other cognitive problems. Rather, these findings suggest that many cases of ADHD are not characterised by impaired performance-based assessments of cognition, despite verbal reports of behavioural symptoms that are consistent with poor cognitive control.

A disconnect between verbal symptom reports and biobehavioural markers of cognitive performance is widely documented in later life. A recent systematic review found that approximately 24% of older adults presenting to memory clinics with self-reported memory complaints have age-typical cognitive performance on neuropsychological assessments (McWhirter et al., 2019). Such individuals are described as having functional cognitive difficulties; that is, cognitive difficulties unrelated to brain disease that are secondary consequences of dysregulated attention, meta-cognitive errors and heightened psychological and emotional distress (Hill et al., 2016; Buckley et al., 2013; Farina et al., 2020; McWhirter et al., 2019). In fact, subjective reports of cognitive difficulties are often associated with increased anxiety (Jenkins et al., 2019), depression (Fischer et al., 2008; Schweizer et al., 2018), low self-esteem and somatic complaints (Collins & Abeles, 1996; Hänninen et al., 1994). Performance-based cognitive tasks capture the processing efficiency of cognitive abilities in tightly controlled experimental and structured conditions, but they have been criticised for lacking ecological validity in relation to the day-to-day adaptive use of cognitive skills (Castellanos et al., 2006; Isquith et al., 2013). In contrast, subjective reports or ratings of cognitive problems are open to rater bias (e.g., parental bias; Reid et al., 1987; Stone et al., 2013), but provide a useful measure of functional impairments in cognition (Isquith et al., 2013). Thus, in the case of older adults, a range of emotional difficulties might impact on self-reported functional memory complaints and contribute to their ICP.

We propose the same might be true for some children with learning-related problems. For example, recent studies point to the existence of a discrete subgroup of children presenting with neurodevelopmental difficulties that are derived from emotional rather than cognitive mechanisms (e.g., Karalunas et al., 2019; Nigg et al., 2020a, b, c). For example, Vaidya et al. (2020) identified a subgroup of children with lower emotional regulation and flexibility, in addition to subgroups with lower inhibitory control and other cognitive impairments, in a sample comprised of autistic children, children with ADHD, and those with no diagnosis. Of those with an ADHD diagnosis, 19% had primary problems with emotional regulation and flexibility. Consistent with this, including emotional regulation measures alongside cognitive tasks increases the likelihood of predicting whether a child has neurodevelopmental difficulties: in the case of ADHD, as many as 90% of ADHD cases can be predicted (Sjöwall et al., 2013). Together these studies suggest that emotional and behavioural regulation difficulties likely contribute to the expression of what may appear to be cognitive difficulties in children with neurodevelopmental disorders. However, it has not yet been established whether emotional and behavioural dysregulation are associated with discrepancies between subjective reports and task-based measures of cognition in children referred for learning-related problems. Understanding whether their cognitive abilities reflect mental health difficulties has important implications for clinical services: for some children presenting with cognitive difficulties, therapeutic approaches targeting mental health might be more suitable than educational or cognitive interventions.

The first aim of the current study was to estimate the percentage of children with an ICP among a large transdiagnostic sample identified as experiencing cognitive problems by education and health professionals. It included children with relatively mild problems judged to be compromising their academic progress, who would likely not meet diagnostic thresholds, in addition to many children whose more marked problems would. Some children had a single diagnosis, others had multiple diagnoses, but the majority were undiagnosed despite coming to the attention of a professional for experiencing cognitive difficulties that were affecting their school progress. By adopting a transdiagnostic perspective we were therefore able to include children who were viewed to be experiencing cognitive difficulties by a practitioner, and test whether their cognitive difficulties manifested in both parent’s subjective ratings and on performance-based tasks. To pre-empt the findings, there were some children who did not have difficulties on either the performance-based cognitive tasks or the parent ratings. While they were still viewed as struggling learners by practitioners, we used this group as a comparison sample for those who showed cognitive impairments on our measures.

The cognitive domains we assessed included working memory and attention. These were chosen because they are both implicated in neurodevelopmental disorders and associated with children’s learning outcomes (e.g. Holmes et al., 2014, 2020; Lui & Tannock, 2007; Rubia, 2018). For example, children with working memory problems typically perform relatively poorly on school-based evaluations of learning and standardised measures of reading and maths (e.g., Alloway & Alloway, 2010; Swanson & Sachse-Lee, 2001). Working memory problems are also common in children with a wide range of neurodevelopmental disorders including ADHD (Holmes et al., 2014; Martinussen et al., 2005) and dyslexia (Holmes, 2012; Jeffries & Everatt, 2004). Similarly, children with elevated levels of inattention have impaired reading and maths abilities (e.g., Loe & Feldman, 2007). Attentional difficulties are common among children with neurodevelopmental disorders, most notably those with ADHD (e.g., American Psychiatric Association, 2013).

The two constructs of working memory and attention were treated separately in the analyses. Although they are highly related (e.g., Oberauer, 2019), we treat them as separate constructs, consistent with an extensive literature suggesting that functioning in the two domains make independent contributions to clinical and academic problems (Brocki et al., 2008; Harmer et al., 2002; Holmes et al., 2020; Slattery et al., 2022). To identify children with consistent and inconsistent cognitive profiles we first identified children with subjective cognitive difficulties as rated by their parents. We then split this group into those who had corresponding difficulties on the performance-based tasks (Consistent Cognitive Profile; CCP) and those who did not have impairments on the performance-based tasks (Inconsistent Cognitive Profile; ICP). The cut-offs used to identify whether a child had difficulties was guided by previously established thresholds for high/low cognitive performance (Bondi et al., 2014; Jessen et al., 2020). The definition of an ICP included those with subjective parent ratings of cognitive problems in the absence of difficulties on performance-based tasks, and not the reverse: we did not include those with task-based deficits in the absence of subjectively rated cognitive problems. This due to our interest in identifying whether functional cognitive difficulties – those that are observed in everyday situations and not in tightly controlled experimental settings– are related to mental health problems. This mirrors the work conducted with older adults that has shown people presenting with complaints of everyday memory problems do not always perform poorly on cognitive tasks, but often experience psychological distress (e.g., McWhirter et al., 2019).

A second aim was to investigate whether symptoms of poor mental health were associated with a discrepancy between subjective parental reports of cognitive difficulties and task-based performance. Based on findings from older adults and children with ADHD, we hypothesised that elevated symptoms of mental health difficulties would predict functional cognitive difficulties as captured by parent ratings, even in the absence of impaired cognitive task-performance. The Strengths and Difficulties Questionnaire (Goodman, 1997), a scale used to capture externalising and internalising problems, was used as a measure of mental health. We adopted two statistical approaches to explore whether discrepancies between subjective and performance-based cognitive difficulties were associated with increased mental health problems. First, the effect of cognitive profile (consistent vs inconsistent) on internalising and externalising sub-scales from the Strengths and Difficulties Questionnaire (SDQ; Goodman et al., 2010) was tested. Second, a continuous analysis was conducted to overcome the limitations of using cut-offs to arbitrarily define whether a child had difficulties. The discrepancy between parent ratings and performance-based measures of cognition was quantified using a regression-based approach: residuals were derived that captured the variance in subjective reports of cognitive difficulties not accounted for by performance-based tasks. These values were correlated with the internalising and externalising measures of the SDQ. We expected that both analyses would reveal an association between internalising and externalising difficulties and an ICP.

Many children with learning-related problems have associated cognitive difficulties. However, not every child referred for psycho-educational/clinical assessment based on a practitioner’s observation of learning problems performs poorly on performance-based measures of cognitive function. The current study estimated the prevalence of this inconsistent cognitive profile (ICP) in a large sample of children referred for problems in attention, learning and / or memory by health and education practitioners, and explored whether inconsistencies between subjective ratings and performance-based tests of cognitive problems were associated with elevated internalising and externalising symptoms. ICPs were highly prevalent in the sample. Children with ICPs and those with consistent cognitive profiles (CCPs; both parent-rated cognitive difficulties and impaired cognitive task-performance) had elevated levels of internalising and externalising problems relative to children with age-typical cognition. Children classified as having CCPs for attention had greater externalising problems than those with ICPs, but there were no other differences between these two groups. Discrepancies between the ratings of working memory difficulties provided by parents and those predicted by performance on tasks of working memory were associated with increased symptoms of internalising and externalising difficulties. For measures of attention, these discrepancies were only associated with externalising difficulties. These findings are discussed in turn below.

Our unique transdiagnostic sample of over 700 children was comprised entirely of individuals who were identified by health or educational providers as experiencing cognitive or learning problems that were affecting their school progress. Among these, 47% had an ICP for attention and 54% for WM: in both cases, almost half of the sample had an inconsistency between subjective parent-ratings of cognitive difficulties and their performance on task-based measures of cognition. More children had a consistent pattern of difficulties across subjective parent ratings and performance-based measures for attention than WM; 43% compared to 30%. These differences might reflect the ease with which attentional lapses and difficulties sustaining attention can be observed, and the relative familiarity parents will have with attention-based problems. It is comparatively easy to observe when a child loses focus or becomes distracted. In contrast, identifying working memory failures is less commonplace, and the concept of working memory is less well integrated into everyday language. Differences in parents understanding of attentional and working memory failures might therefore underlie the differences in consistencies between their ratings and task-based performance across the two cognitive domains.

Our findings suggest that almost half (43–54%) of children with practitioner-observed cognitive and learning-difficulties do not exhibit any deficit on performance-based measures of WM and sustained attention, which contrasts the reported ubiquity of these cognitive difficulties in struggling learners (Follmer, 2018; Holmes et al., 2020; Landerl & Kolle, 2009; Peng & Fuchs, 2016; Peng et al., 2018; Yeniad et al., 2013). This figure is consistent with the percentage of unclassified children from ADHD discriminatory studies (Nigg et al., 2005; Solanto et al., 2001), and studies exploring functional cognitive difficulties in older adults (McWhirter et al., 2019; Minett et al., 2008; Hill et al., 2016). These findings highlight the importance of assessing children’s performance on cognitive tasks in addition to observing their behaviours to fully understand where their difficulties lie. This will help determine appropriate approaches to intervention.

Children with parent-reports of cognitive difficulties, either as part of an ICP or CCP, were rated as experiencing greater internalising and externalising difficulties than children who did not meet the threshold for attention and WM difficulties. This is consistent with our hypothesis that children with higher subjective parent ratings of difficulties in attention and WM, even in the absence of performance-based deficits in these areas, would experience more symptoms of internalising and externalising problems. It also aligns with literature indicating that children with neurodevelopmental difficulties are at increased risk of mental health problems (e.g. Bryant et al., 2020; Francis et al., 2019; Holmes et al., 2021), and supports growing links between emotional processes and learning-related difficulties (Nigg et al., 2020a, c; Vaillancourt et al., 2017; Yoshimasu et al., 2012).

While elevated mental health problems were not specific to children with an ICP, we did find an association between an ICP and elevated internalising and externalising symptoms consistent with our hypothesis. Although this study is cross-sectional and will require further research to establish causal pathways, we tentatively propose that everyday cognitive difficulties in these children may arise, in part, through mental health problems as they do for older adults (e.g. McWhirter et al., 2019): negative mood states might impair cognitive functioning. Indeed, the cognitive load associated with down regulation of negative emotional states is greater in younger children than in adults and can lead to an increase in everyday cognitive failures irrespective of baseline ability (Scheibe & Blanchard-Fields, 2009). These cognitive failures might occasion subjective reports of cognitive difficulties from parents and/or educators. Evidence suggests that cognitive failures may also trigger environmental consequences that further impede performance and psycho-social functioning (Sonuga-Barke, 2005; Nigg et al., 2005). First, a sense of failure can negatively impact mood and sense of self, which exacerbates the original negative affective states (Farina et al., 2020). Second, the anticipation of cognitive difficulties is associated with increased cognitive fatigue and performance deficits (Lenaert et al., 2021). Third, contexts associated with cognitive failures may become aversive and prompt withdrawal, thus limiting opportunities for cognitive growth and academic and social development (Sonuga-Barke, 2005; Loe & Feldman, 2007). Such consequences can also increase opportunities for subjective reports of cognitive difficulties from parents and/or educators. In this way, we tentatively propose that subjective reports of cognitive difficulties that occur without any performance-based deficits may be a functional consequence of mental health difficulties. This requires empirical testing via future experimental and longitudinal studies.

Externalising problems were more common among children with CCPs in attention than in children with an ICP in attention. This suggests that children with attentional difficulties measured by both parent ratings and task performance are more hyperactive and experience more conduct problems than children with subjective ratings of attentional problems without problems on a sustained attention task. This was an unexpected finding, which might be explained by theories ascribing a core role to cognitive control in behavioural regulation (e.g. Barkley, 1999; Brocki & Bohlin, 2006; Casey et al., 2001; Scheres et al., 2004). Sustained attention tasks require both focus and the ability to inhibit distractions. Those with a CCP, who performed poorly on the task, may therefore have poorer cognitive control or inhibitory skills than those with an ICP.

The pattern of associations found between discrepancies in subjectively rated cognitive problems and task-based measures of working memory and attention using a categorical grouping approach was largely replicated when a continuous approach to the analysis was adopted. In these analyses, parent rated difficulties were poorly predicted by task performance in children with heightened levels of externalising problems for both working memory and attention. This aligns with the findings from the categorical analysis showing that children with subjective ratings of cognitive difficulties in the absence of task-based deficits had elevated externalising problems relative to the comparison group. Similarly, a larger inconsistency between the parent reports of working memory problems, and those predicted by performance on working memory tasks, was associated with elevated internalising symptoms. This was consistent with the outcomes of the group-based approach showing that children with an ICP for working memory had elevated internalising symptoms relative to the comparison group.

The only inconsistency across the methods was that an ICP for attention was associated with increased internalising symptoms relative to the comparison group in the categorical analysis, yet internalising difficulties were not associated with the variance remaining in parent reported attentional problems after the variance predicted by task-performance had been removed. The continuous analysis is likely more sensitive than the categorical, suggesting subjective everyday problems with attention might not be related to functional problems arising from internalising symptoms. Our data therefore suggest internalising symptoms explain more of the discrepancies between everyday difficulties and task performance for working memory than for attention, and that externalising symptoms may explain some of the discrepancies between everyday difficulties and task performance for both attention and working memory. This pattern is consistent with earlier work reporting associations between working memory and depression, but not between attention and depression (Matthews et al., 2008), and links between working memory and internalising and externalising symptoms (Donati et al., 2021).

A final noteworthy finding is that parent ratings of difficulties in attention and working memory were more poorly predicted by task performance for girls than boys. This suggests parents may have been less able to detect or observe cognitive difficulties in girls. This might reflect implicit gender biases and stereotyping (e.g., discussed in Anderson, 1997; Sciutto et al., 2004) that lead parents to rate boys as experiencing more difficulties. Alternatively, it might be driven by different expressions of difficulties in boys and girls, and in particular by the tendency for girls to mask their problems in everyday situations (e.g., Dhuey & Lipscomb, 2010; Hiller et al., 2014; Hull et al., 2020).

Drawing together the findings from both analytic approaches, cognitive difficulties, whether part of a CCP and an ICP, are associated with increased externalising and internalising problems. Further, when cognitive task performance is a poor predictor of subjective everyday cognitive difficulties, externalising symptoms predict functional impairments in working memory and attention, and internalising symptoms predict functional impairments in working memory. Cognition and mental health interact across development. The cognitive reserve hypothesis suggests poor cognitive function impairs the downregulation of negative emotional responses, such as worry or sadness, leading to poor mental health (LeMoult & Gotlib, 2019). Conversely, the interference hypothesis suggests psychological distress disrupts cognitive processing by shifting cognitive resources away from task-relevant information and onto negative thoughts (Llewellyn et al., 2008; Stawski et al., 2006), resulting in both short- and long-term cognitive difficulties (Dolcos et al., 2020). The dynamic mutualism hypothesis integrates these two opposing theories, arguing that mental health and cognitive function reciprocally interact over time, leading to a dynamic cycle of exacerbation across the lifespan (Fuhrmann et al., 2020). It is not possible to tease apart these hypotheses based on the current data, but our findings do add to a growing body of work demonstrating associations between cognitive function and mental health in childhood.