Parent, Teacher and Observational Reports of Emotional and Behavioral Problems in Young Autistic Children

To conduct a thorough assessment of EBPs in childhood, guidelines suggest assessment of impairment across settings (National Institute for Health & Care Excellence, 2013). The use of multiple informants (e.g. parent and teacher) and naturalistic observation ensures a comprehensive understanding of the individual across contexts (Mash & Hunsley, 2005). Informant discrepancies, where informants provide different reports of similar constructs, are expected and commonplace during assessments of EBPs in autism (Stratis & Lecavalier, 2015). This can lead to uncertainty in clinical decision making and treatment planning, but may give information on factors driving or maintaining EBPs.

There are a range of factors, not related to measurement error, that may influence reporting of child EBPs and lead to informant discrepancies. A key factor is situational specificity (De Los Reyes et al., 2013); the nature of the settings in which different individuals interact with the child could impact how informants understand and report EBPs. For example, the structured nature of the school environment may be associated with different displays of EBPs at school. Discrepancies across reports may therefore yield important information on how the child behaves in different contexts and can be used to inform environmental modification and intervention for EBPs.

Levels of EBPs may vary across settings and literature comparing reports from parents and teachers of EBPs among autistic children are mixed. Some studies report higher levels of teacher-reported oppositionality and anxiety (Reed & Osborne, 2013), whereas others report higher levels of parent-reported EBPs compared to teachers (Jepsen et al., 2012; Llanes et al., 2020). Although comparable to agreement in the general population (Achenbach et al., 1987), levels of agreement between parents and teachers on reports of EBPs for autistic children are often moderate at best (Stratis & Lecavalier, 2015). Whilst it is likely that situational specificity is playing a role, parent-teacher agreement in autism remains low when teachers and parents are observing and reporting on child behavior in the home (Reed & Osborne, 2013). This indicates that factors other than situational specificity are likely to influence reports of EBPs and better understanding of these factors is warranted. Furthermore, few studies have explored how observational measures of child behavior, often considered the ‘gold standard’, are associated with other informant reports of EBPs among autistic children.

Given the varied findings in the literature of factors associated with reports of EBPs in autistic children, there were three key objectives of this study. First, we compared levels of EBPs in autistic children reported by parents and teachers. We hypothesized that parents would report higher levels of EBPs than teachers. Second, we explored whether child and informant characteristics were associated with levels of parent-, teacher- and researcher-rated observed EBPs. We hypothesized that less verbal language, greater autism severity and higher rates of researcher-rated observed behaviors that challenge (BTC; e.g. non-compliance, hyperactivity) would be associated with more parent- and teacher-reported EBPs, and that autism severity and less verbal language would be associated with more researcher-rated observed BTC. We expected that higher self-reported parenting stress and lower parental wellbeing would correlate with greater parent-reported EBPs, and that special school placement would be associated with lower teacher-reported EBPs as teachers in these settings are likely to have differing thresholds and have more support for child EBPs. We did not have a specific hypothesis for whether parent characteristics would be associated with teacher-reported and researcher-rated observed child EBPs, but instead explored the associations between parental reports of parenting stress and wellbeing and these measures. Finally, we explored agreement between parent and teacher reports of EBPs and whether informant agreement between teachers and parents varied according to score severity. We hypothesized that parent-teacher agreement would be modest, but that agreement would increase as the severity of parent and teacher rated EBPs increased.

Participants were 83 young autistic children and their parents (see Table 1 for further details) participating in the Autism Spectrum Treatment and Resilience (ASTAR) study (Charman et al., 2021; Palmer et al., 2019), as part of the Improving Autism Mental Health (IAMHealth) research program (https://​iamhealthkcl.​net). Most children were male (n = 71, 85.5%) and the mean age was 6.70 years (SD = 1.21 years). Verbal language levels varied across the sample; 39 (47.0%) were minimally verbal and 44 (53.0%) were verbal (see description of verbal ability grouping in measures section below). In addition, 76 education professionals (mainly Class Teachers, but occasionally a school Special Educational Needs Coordinator, Learning Support Assistant, or Head Teacher; referred to hereafter as ‘teachers’ for simplicity) completed questionnaires about the children.

Participants were enrolled in the ASTAR study, a non-randomized feasibility study followed by a pilot randomized controlled trial (RCT) of two novel parent-mediated group interventions. Ethical approval for the study was granted from NHS Camden and Kings Cross Research Ethics Committee (ref: 16/LO/1769). Written informed consent was obtained from all participating parents and teachers and child assent was obtained wherever appropriate.

Twenty-one parent–child dyads participated in the feasibility study and 62 in the pilot RCT, recruited from local autism diagnostic teams, education professionals, support groups, consented databases, and self-referral routes in four boroughs of South London. To be eligible to take part, the child had to have a clinical diagnosis of an Autism Spectrum Disorder and be between 4 years 0 months and 8 years 11 months at randomization. The study was designed to be as inclusive as possible, so no exclusion based on levels of child EBPs was used; nor for verbal language level or IQ. Additional inclusion/exclusion criteria can be found in Palmer et al. (2019).

Data collected during the baseline assessments were used in the current study. Parents completed questionnaires on their child’s EBPs and their own parenting stress and wellbeing. With parental consent, the child’s teacher completed questionnaires about the child’s EBPs at school. The baseline assessment also involved a research visit where observations of parent–child/researcher-child interaction and autism severity were obtained.

Data analysis was conducted in Stata 15 (StataCorp, 2015). All variables were assessed for normality. Teacher ABC child irritability scores violated the assumptions of normality and where possible, the appropriate non-parametric tests were used. Where there was no non-parametric alternative, we conducted sensitivity analysis using the log transformation of the variable. The interpretation of the results was unchanged when using the log-transformed variable, so the untransformed variable was retained and used for all parametric analyses. First, we examined differences in mean parent- and teacher-reported scores on the ABC and ACB subscales using paired sample t-tests and Cohen’s d was used to calculate the size of the effects between the groups.

Next, we used a hypothesis-driven approach to test whether child and informant characteristics were associated with levels of parent-, teacher- and researcher-rated observed EBPs. First, we explored the correlations between all potential factors (i.e. child and informant characteristics) and measures of EBPs. To dichotomize school placement, specialist units in a mainstream school were included with mainstream schools, and specialist only schools were separate. This is consistent with previous research that has combined special units in mainstream schools in this way (e.g. Simonoff et al., 2019).

As observed child BTC were rated on a different scale to the measures completed by parents and teachers, associations with observed BTC were tested using a univariate multiple linear regression model. Independent variables included child verbal language, parenting stress and parental wellbeing. School placement was not included in this regression given the overlap between verbal language grouping and school placement and we did not have a clear hypothesis about how school placement would influence researcher-rated observed BTC above and beyond verbal language.

Inter-rater agreement between parents and teachers was explored in two ways. First, we obtained the intraclass correlation coefficients (ICCs) and 95% confidence intervals (CIs) to assess agreement across parent and teacher scores on the ABC and the ACB. F tests were used to test whether the ICC were significantly different from zero. Bland–Altman plots (BA, Bland & Altman, 1995) were also produced to visually depict systematic trends in informant agreement across the range of scores on the ABC and ACB subscales. We tested whether parent and teacher agreement was associated with score severity on each subscale using Spearman’s rho (as in Bland & Altman, 1995).

Table 2 displays the descriptive statistics and the results of the t-tests comparing parent and teacher scores on the ABC and ACB subscales and associated effect sizes. As hypothesized, parents reported significantly more irritability and hyperactivity on the ABC and more emotional and behavioral problems on the ACB compared to teachers; all effect sizes were moderate.

Supplementary Table S1 displays the bivariate correlations between the child and informant characteristics and parent-, teacher- and researcher-reports of child EBPs (see Table 1 for descriptives of the clinical measures). Parenting stress and wellbeing were significantly positively correlated with the majority of parent-reported EBPs (r’s ranged from 0.37–0.61, p’s all < 0.001 and r’s were − 0.20– − 0.51, p’s from 0.071 to < 0.001 for parenting stress and wellbeing, respectively). Observed BTC was positively correlated with teacher-reported behavioral problems on the ACB (r = 0.23, p = 0.048).

Table 3 displays the standardized coefficients and their associated p values for the multivariate linear multiple regression models examining parent- and teacher-reported EBPs. Figure 1 is a forest plot showing the standardized coefficients for these models.

Higher parental ABC irritability scores were uniquely associated with the child being verbal and greater parenting stress. Post-estimation tests showed that children whose parents reported greater parenting stress had significantly higher parent-, but not teacher-reported, child irritability (p < 0.001). The post-estimation test for verbal language grouping was not significant (p = 0.093).

For ACB behavioral problems, children who were verbal and had parents reporting greater parenting stress had significantly higher parent-reported behavioral problems; post-estimation tests showed these variables were significant correlates of parent-reported behavioral problems when compared to teacher reports (p = 0.004 and p = 0.004).

For the ACB emotional problems scores, higher parenting stress was associated with more child emotional problems, with post-estimation tests showing that higher levels of parenting stress were associated with higher levels of parent-reported, but not teacher-reported, child emotional problems (p = 0.013). For teachers, more observed BTC was significantly associated with higher teacher reports of child emotional problems, however, post-estimation tests showed that when compared to parent-report, observed child BTC was not significantly associated with teacher-reported emotional problems (p = 0.703).

Supplementary Table S1 displays the correlations between the child and informant characteristics and researcher-rated observed child BTC rates. Being minimally verbal was associated with more observable child BTC. The standardized coefficients and associated p values for the univariate multiple linear regression model examining researcher-rated observed child BTC rates are also displayed in Table 3. When controlling for other variables, being minimally verbal was the only factor associated with more observable child BTC.

Table 2 presents the ICCs between parent and teacher scores and associated p values on the ABC and ACB measures. The ICCs examining agreement between parents and teachers ranged from 0.04 (child emotional problems) to 0.30 (child hyperactivity). ICCs were statistically significant for all variables with the exception of ACB child emotional problems.

Informant agreement and variation in agreement by score severity was further examined using Bland–Altman plots and the accompanying regression analysis (see Supplementary Fig. S1). Supplementary Table S2 presents the mean differences in parent and teacher scores and combined parent-teacher scores. No significant relationships between the difference scores and the average combined scores were found for the ABC irritability, ABC hyperactivity and ACB behavioral problems (see Supplementary Fig. S1 for ρ and p values). However, for the ACB emotional problems, there was a significant relationship indicating that agreement varied based on severity of emotional problems (ρ = 0.29, p = 0.011), with a general pattern of larger discrepancy between informants as emotional problem scores increase.

As hypothesized, in the current sample of 4–8-year-old autistic children, parents reported more EBPs than teachers and this pattern held across both emotional and behavioral problems. This aligns with previous literature in non-autistic (Rescorla et al., 2014) and autistic populations (Chandler et al., 2016; Jepsen et al., 2012). The differences in levels of EBPs reported in home and educational settings lends support for the notion of situational specificity, although because the reports are provided by different individuals, other informant characteristics cannot be ruled out. The nature of these different environments is likely to influence which behaviors are displayed. One possibility is that the structured, routine-based school environment reduces the likelihood of a child encountering unexpected events or stimuli, a key trigger for EBPs in autism (Bearss et al., 2015). A further explanation, often described anecdotally by parents and clinicians, is a delayed display of the pressures faced by autistic children during a typical school day. It may be that autistic children are ‘holding in’ manifestations of EBPs (Bearss et al., 2015) in school environments resulting in reports of less EBPs by teachers and greater manifestations of EBPs when with parents. This may only be relevant for a sub-group of autistic individuals who have the cognitive ability to reflect and greater self-control to modify their behavior. Indeed, in this study, the child being verbal (suggesting higher IQ and more cognitive ability) was generally associated with higher parent-, but not teacher-, reported EBPs.

We also explored child and informant characteristics associated with parent-, teacher- and researcher-rated EBPs. For emotional problems, higher levels of parenting stress were associated with more parent-reported child emotional problems. For behavioral problems, higher levels of parenting stress and the child being verbal were associated with increased parent-reported behavioral problems. This finding is at odds with previous research that has reported more irritability and hyperactivity among minimally verbal children, albeit in a sample of older autistic children and young people aged 6–18 years (Fok & Bal, 2019). This inconsistent finding could also be due to difficulties in measuring EBPs in autistic children, particularly among those who are minimally verbal or have an intellectual disability (Flynn et al., 2017). However, the relationship between the child being verbal and elevated parent-reported behavioral problems was consistent across the measures used in this study, including the ACB, a measure of EBPs developed specifically for autistic populations and shown to be reliable and valid across the spectrum of developmental ability in autism (Tarver et al., 2021b). Interestingly, parental wellbeing was not a correlate of EBPs. It may be that stress related to the parenting role has a stronger relationship with child EBPs than more general parent wellbeing (e.g. feeling relaxed and confident, can deal with day-to-day problems). This could be especially true given that the parenting stress measure used in this study was specific to stress related to autistic traits and co-occurring EBPs.

That parenting stress is associated with parent-reported EBPs but not teacher nor observed child difficulties in our study is important. This adds to previous research with both non-autistic and autistic children which found parenting stress is associated with greater discrepancies in reports of EBPs between parents and teachers (Langberg et al., 2010; van der Oord et al., 2006). The findings from this study could suggest that stress is influencing parent reports of EBPs in this sample (Najman et al., 2000); that the association between elevated parenting stress and parent-reported EBPs was consistent across multiple parent-reported measures acts as further support for this. However, it is also possible that parenting stress is elevated because EBPs are predominantly challenging in the home context, or in situations when children are more likely to be with their parents (e.g. in the supermarket, when on holiday). If so, the findings of this study may be suggestive of situational specificity, and that young autistic children are more likely to manifest EBPs with their parents. This is not to say that autistic children do not display EBPs at school, but instead, they become more visible when children get home (Bearss et al., 2015). When we tested whether the same characteristics were also associated with teacher reports of EBPs, a different pattern of results was found. None of the child or informant characteristics included in the models were associated with teacher reports of child irritability, hyperactivity, behavioral or emotional problems. Contrary to our hypothesis, verbal ability was not associated with teacher-reported EBPs. On the other hand, being minimally verbal was associated with more researcher-rated observed BTC, which is in line with other literature (e.g. Einfeld et al., 2011). No parental characteristics were associated with the rate of child BTC seen during this observation.

In general, agreement between parent and teacher reports of EBPs was low. In line with non-autistic populations (e.g. Rescorla et al., 2014), agreement between parents and teachers appeared to be stronger for more observable behaviors (hyperactivity; behavioral problems), with lower agreement found for measures of irritability and emotional problems (domains which include questions interrogating internal states and mood). This pattern of findings is also similar to the levels of agreement reported between parents and teachers of young autistic individuals (Llanes et al., 2020; Stratis & Lecavalier, 2015), although agreement between parents and teachers in the current sample is lower for emotional problems in particular. This may be in part due to the small sample size and lack of precision as confidence intervals were wide.

Agreement was also explored using Bland–Altman plots to visually depict trends in informant agreement. For the measures of child irritability, hyperactivity and behavior problems, there was no relationship between parent and teacher agreement and the severity of EBPs. However, for emotional problems, agreement varied based on how severe the scores were with a general pattern of larger differences between informants as emotional problems increased. This suggests that when emotional problems are reported as problematic in one context, they may not necessarily be problematic in another context. For example, anxiety could be more ‘situation specific’ than hyperactivity among autistic children. Another explanation may be that anxiety presents in atypical ways in autism (e.g. as repetitive or challenging behavior, Bearss et al., 2015; Kerns et al., 2014; Tarver et al., 2021a) and reliance on interpretation may explain lower agreement. However, it is of note that we did not find any association between parent-reported emotional problems and teacher-reported behavioral problems, and vice versa. This suggests emotional problems rated by one informant are not interpreted as behavioral problems by the other.

Parent-teacher agreement on measures of EBPs is modest; young autistic children may show variations in EBPs across settings. A comprehensive assessment of EBPs should always take a cross-setting, multi-informant approach to assist with understanding triggers for EBPs, clinical decision-making and care planning. Young autistic children displaying elevated EBPs at home, may not display this at school, and vice versa. Another key clinical implication is that an assessment of parenting stress should be conducted during care planning for autistic children, as recommended by the National Institute for Health and Care Excellence guidance for assessment of autism and intellectual disability and challenging behavior (National Institute for Health & Care Excellence, 2015, 2016). This is especially the case if discrepancies in parent and teacher reports, or observations of child EBPs are evident. An objective observation of the child in the home environment may aid clinician understanding of whether more parent-reported EBPs is because EBPs are primarily manifesting in the home environment. Given the differences in both the levels of EBPs and the modest agreement between parents and teachers, it may mean that different support for parents and teachers to help them manage child EBPs is required. It is important that discrepancies in informant reports are not dismissed as measurement error or informant bias. Clinicians should consider whether variations in informant report might reflect true situational variation in EBPs.