Introduction
Behavioral teacher training is an effective intervention to reduce children’s attention-deficit/hyperactivity disorder (ADHD) symptoms and related behavioral problems in the classroom (DuPaul et al.,
2012; Evans et al.,
2018; Fabiano et al.,
2009; Veenman et al.,
2016; Ward et al.,
2020). Effect sizes of current training programs generally range from small to medium (DuPaul et al.,
2012; Ward et al.,
2020), thus leaving room for improvement. Insight into which intervention components are effective and which are not may contribute to the development and improvement of behavioral teacher trainings for ADHD (DuPaul et al.,
2020; Schatz et al.,
2020). However, studies on the effectiveness of separate intervention components are scarce.
Behavioral teacher interventions for ADHD generally include training teachers in the use of both antecedent-based techniques (i.e., stimulus-control techniques such as providing structure and clear instructions) and consequent-based techniques (i.e., contingency management techniques such as praise, reward and planned ignoring) (DuPaul et al.,
2022). Teachers are typically taught to combine both sets of techniques, for example by giving a clear instruction to the child to raise his/her hand before speaking and praise the child when doing this (Patterson,
1982). Meta-analytic evidence from behavioral teacher and parent interventions suggests that both antecedent- and consequent-based techniques implemented by teachers and parents are effective to improve children’s ADHD symptoms and oppositional defiant (ODD) behaviors (Gaastra et al.,
2016; Leijten et al.,
2019). However, meta-analysis only allows testing whether intervention effects are larger for interventions that include a particular intervention component (e.g., training teachers in a set of techniques) as compared to interventions that do not include that particular intervention component (Leijten et al.,
2021). Thus each single intervention component is always studied in the context of other intervention components (Lipsey,
2003). Meta-analyses can therefore be used to generate hypotheses about effective intervention components, but whether effect sizes are actually driven by a particular component remains to be studied (Leijten et al.,
2021). In contrast, microtrials are experimental designs that can be used to test hypotheses regarding the effectiveness of single intervention components by testing the effects of relatively brief and focused environmental manipulations, such as single intervention components, on proximal outcomes (Howe et al.,
2010; Leijten et al.,
2015). Such a design allows to study the effectiveness of antecedent- and consequent-based techniques in isolation, which has not been done so far. Therefore, to test the hypotheses about the effectiveness of antecedent- and consequent-based techniques derived from meta-analytic studies (Gaastra et al.,
2016; Leijten et al.,
2019), our study used a microtrial design to examine the effectiveness of implementing antecedent- and consequent-based techniques in reducing the behavioral problems and impairment children with ADHD often experience in the classroom.
In a previous article of our group (Staff et al.,
2021), we analyzed our randomized controlled microtrial using an ecologically momentary assessment (EMA) measure of behavior as outcome measure (Shiffman et al.,
2008). Four preselected individual problem behaviors in a specific situation were assessed, and two of these were directly targeted in the intervention. The behaviors and situations thus differed per child-teacher dyad. Examples were ‘difficulties staying focused during individual seatwork’ or ‘talking excessively during whole group teaching’. Following EMA procedures, the four behaviors were daily assessed in the specific situation, maximizing the ecological validity and minimizing recall and retrospection bias that may be observed using traditional questionnaires based on the Diagnostic and Statistical Manual of Mental Disorders (DSM) (Bentley et al.,
2019; Shiffman et al.,
2008). We showed that antecedent- and consequent-based techniques were equally effective in reducing these four daily teacher-rated problem behaviors in a specific situation. Effects were obtained directly after the intervention (large effects,
d = 0.89, 0.93, respectively), and remained stable up to three months later.
Nevertheless, two important questions remained unanswered, i.e., whether the promising findings on our EMA outcomes are reflected in: 1) broader assessments of ADHD and ODD behaviors, and 2) impairment. Regarding the first question, we were interested whether effects were also obtained if outcomes comprised the full range of DSM-based teacher-rated ADHD and ODD symptoms assessed on a rating scale, i.e., whether effects could also be observed when teachers were asked to report behaviors averaged over the past week and during all situations, rather than during a specific situation during each day. Using traditional DSM-based questionnaires also provides possibilities to compare results with the findings of other behavioral interventions for ADHD. Further, we were interested whether effects were observed by raters who were not involved in treatment delivery and thus less susceptible to social desirability and/or investment bias (Daley et al.,
2014; Sonuga-Barke et al.,
2013). Regarding our second question, as functional impairment is often the primary reason for teachers to seek help (Coles et al.,
2012), we were interested whether effects are also observed in terms of functional impairment.
The aim of the present study was thus to examine the effectiveness of antecedent- and consequent-based techniques on (1) teacher-rated and masked observations of ADHD and ODD behaviors according to DSM-criteria, and (2) teacher-rated functional impairment. Data were collected in our randomized controlled microtrial that tested two short and individualized behavioral teacher interventions focusing on either antecedent- or consequent-based techniques. Based on our previous findings regarding our EMA outcome, we hypothesized that both sets of techniques would be effective compared to a waitlist control condition in reducing ADHD and ODD symptoms as rated by teachers, both immediately after the intervention as well as at three months follow-up. We expected smaller effect sizes compared to our EMA outcomes (Howe et al.,
2010), given that the current measures reflect more distal outcomes. Further, we expected effects to be most pronounced shortly after the intervention compared to three months later (Lee et al.,
2012). For our masked assessments of ADHD and ODD behaviors, structured classroom observations were conducted in a (randomly selected) subsample. Classroom observations have shown to be a valid measure to assess ADHD and ODD in the classroom (Minder et al.,
2018) and to be sensitive to effects of behavioral interventions (Pelham et al.,
2005; Pfiffner et al.,
2013). We expected both sets of techniques to be effective in reducing observed ADHD and ODD behaviors (Pfiffner et al.,
2013). Finally, we expected both sets of techniques to be effective in reducing functional impairment as rated by the teacher (Groenman et al.,
2021).
Discussion
Using a microtrial design, this study was aimed to gain insight into whether previously found effects of antecedent- and consequent-based techniques in teacher training for children with ADHD on EMA outcomes (Staff et al.,
2021), were also reflected in broader assessments. More specifically, we examined the effectiveness of both sets of techniques on teacher ratings that comprise the full range of DSM-criteria for ADHD and ODD behaviors, masked classroom observations of ADHD and ODD behaviors, as well as teacher-rated functional impairment.
Effects on DSM-based teacher-rated ADHD were mostly in line with our previously reported findings (Staff et al.,
2021), and with the broader literature on teacher trainings for ADHD (DuPaul et al.,
2012; Evans et al.,
2018; Fabiano et al.,
2009; Ward et al.,
2020). The previous article showed that both interventions were effective in reducing four daily-rated, individually selected, problem behaviors of the child in specific situations, of which two were directly targeted in the intervention. The current article extends these findings by showing that intervention effects were also present in reductions ADHD symptoms according to DSM-criteria as rated by teachers averaged over the past week and all situations, and in reductions of teacher-rated impairment. Teacher-rated ADHD symptoms in the antecedent and consequent conditions even improved up to levels close to the population based mean (Polderman et al.,
2007), while children in the waitlist condition continued to score one standard deviation above the population mean. As effects were obtained on multiple measures (more and less susceptible to bias) and outcomes, this strongly confirms the effectiveness of the interventions.
No significant differences were observed in the effect sizes of both interventions compared to the waitlist condition. This is in contrast to meta-analytic results showing that effect sizes of teacher programs that include consequent-based interventions are somewhat larger than programs that include antecedent-based interventions (Gaastra et al.,
2016). However, as these meta-analytic findings only present evidence in the context of other intervention components (Lipsey,
2003), our findings support the importance of testing hypotheses using experimental (microtrial) designs in order to draw more firm conclusions on the effectiveness of intervention components (Leijten et al.,
2021). Another explanation for the finding that our antecedent- and consequent-based interventions were both effective compared to waitlist condition with similar effect sizes, may be that antecedent-based interventions included in the meta-analysis by Gaastra et al. (
2016) were mostly general educational accommodations (e.g., extended time) of which the evidence base is limited (Lovett & Nelson,
2020). Furthermore, most of these antecedent-based interventions were not tailored to individual needs of the child, while included consequent-based interventions were. In the current study, both interventions were tailored to individual needs (using the behavioral analysis), which may have increased the relative effectiveness of antecedent-based interventions as compared to consequent-based interventions (Dunlap & Kern,
2018; Harrison et al.,
2019). When comparing the intervention conditions to the waitlist condition, there were even indications that the antecedent-based intervention was somewhat more effective than the consequent-based intervention in reducing teacher-rated inattention symptoms (i.e., medium-sized effect for antecedent-based intervention versus a small to medium-sized effect for consequent-based intervention). A similar study of our group into the effectiveness of both types of intervention components in behavioral parent training for ADHD also found this pattern (Hornstra et al.,
2021). As argued by Hornstra and colleagues, it may be that antecedent-based techniques potentially require less time and effort of teachers to implement during the training as compared to consequent-based techniques, because antecedent-based techniques focus on the prevention of problem behavior and can be implemented regardless of child behavior. In addition, before consequent-based techniques can be effective, children may have to be repeatedly exposed to alternated contingencies in order to adapt their behavior, while antecedent-based techniques may have direct effects (Owen et al.,
2012).
Our findings were in line with studies showing that teacher training has longer term effects over three months (DuPaul et al.,
2012; Evans et al.,
2018; Fabiano et al.,
2009; Ward et al.,
2020). Contrary to our expectations, there were even indications for the three months follow-up that teacher-rated ADHD symptoms further improved, regardless of the techniques used, while such effects were not observed for our proximal outcome (i.e., these effects remained stable from post-intervention to follow-up; Staff et al. (
2021)), and behavior often deteriorates after treatment is withdrawn (Lee et al.,
2012). However, we did not include the waitlist condition at T3 as teachers in this condition were offered treatment after T2, so our results need to be confirmed in future studies.
Further, our findings on masked observations of inattention were consistent with effects obtained with teacher ratings, suggesting that effects on inattention were not affected by possible social desirability and/or investment bias (Daley et al.,
2014; Sonuga-Barke et al.,
2013). Compared to the waitlist condition, observed attention problems decreased in the active conditions, confirming the positive (and protective) effects of the interventions. Intervention effects on masked hyperactivity-impulsivity were in the same direction as teacher ratings although effects did not reach statistical significance. This is likely to be explained by the limited number of subjects included in our masked analyses, reducing power. Observed verbal hyperactivity, however, did not show such a pattern, and even increased in the antecedent condition over time. Although we cannot fully explain this finding, this may be related to the low baseline levels of this behavior in the antecedent condition (5.7%, see Table
2) compared to the other conditions, while at T2, group differences in verbal hyperactivity between conditions were small. Further research in larger samples is needed to conclude on the effectiveness of the sets of techniques on masked outcomes of hyperactivity.
In contrast to ODD behaviors as measured with the daily EMA ratings (Staff et al.,
2021) and meta-analytic results showing effects of behavioral interventions on ODD symptoms (Daley et al.,
2014; Leijten et al.,
2019), we did not observe effects of the specific techniques on teacher-rated ODD symptoms, neither on the short term, nor on the longer term, nor on classroom observations of oppositional defiant behaviors. This may be explained by the current sample in which children had low levels of baseline ODD symptoms, possibly indicating that there was not enough room for improvement on ODD behavior. However, given that we obtained large effects on the daily ratings of oppositional behavior assessed with the proximal EMA measure, one may also argue that a proximal measure such as daily ratings using EMA may be more sensitive to observe effects compared to measures assessing broadly defined ODD behavior.
Although the results of our study are promising, there are limitations to note. First, this study was powered on our primary outcome and therefore power for the secondary outcomes reported here may have been too low (Jakobsen et al.,
2019), possibly leading to small effects being undetected. This seems particularly relevant for antecedent versus consequent comparisons as these are both active conditions. Second, classroom observations were conducted only in a subset of the sample, given the time-intensive nature of coding of the observations, and may have led to undetected small effects. However, the effects obtained for attention problems were robust and provide important corroborative information next to our proximal daily ratings and questionnaire ratings for the effectiveness of both sets of techniques. A third limitation is that we have not quantified teacher implementation of the techniques in the classroom (neither quality or dose), and such it cannot be used as a moderator in the analyses. Fourth, our sample predominantly included children with subthreshold ADHD symptoms and low levels of ODD symptoms. Although our results provide useful information for children with (subthreshold) ADHD, effects may not be generalizable to children with more severe ADHD and/or ODD symptoms. Further, our sample was nearly 100% Caucasian and we lack insight into other relevant child (e.g., parental income) and teacher (e.g., race) factors, which may limit the representativeness or our sample.
Conclusions and Clinical Implications
This randomized controlled microtrial showed that antecedent- and consequent-based techniques are effective in reducing children’s ADHD symptoms in the classroom, as assessed by teacher-rated DSM-based measures of ADHD symptoms and functional impairment, as well as masked observations of inattention. These findings extend our previously obtained results on a proximal EMA outcome (Staff et al.,
2021).
Importantly, the effect sizes of these brief and individualized interventions on our secondary outcomes appear similar to those of full and longer interventions often containing both sets of techniques (DuPaul et al.,
2012; Evans et al.,
2018; Fabiano et al.,
2009; Ward et al.,
2020). As described previously (Staff et al.,
2021), the current interventions were short and individualized and were based on functional behavioral analysis of the child’s problem behavior (FBA; Dunlap & Kern,
2018), which may have added to their effectiveness (Chronis et al.,
2004). Furthermore, the brief interventions seem acceptable and feasible for school based practice as all teachers completed the intervention, the majority of the teachers reported to use the techniques learned at three months follow-up (Staff et al.,
2021), and most of the teachers would recommend the training to colleagues. Such short individualized interventions well meet teachers’ needs (DuPaul et al.,
2019; Egan et al.,
2019; Gaastra et al.,
2020), and fits with current ADHD guidelines suggesting that environmental modifications are regarded as first-line interventions prior to more intensified treatment (Akwa GGZ,
2019; National Institute for Health and Care Excellence,
2018). To increase suitability for schools, both sets of techniques could be combined into one intervention. For example, a brief and individualized intervention combing the effective sets of techniques can be provided to teachers seeking help to cope with the disruptive behavior of an individual student showing ADHD symptoms (e.g., Tier 2 interventions).
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.