Introduction
Adolescents with mild intellectual disabilities or borderline intellectual functioning (MID–BIF
1; IQ between 50 and 85 and limitations in social adaptive skills; American Psychiatric Association,
2013) have a higher risk of developing externalizing problems than adolescents without MID–BIF (Dekker et al.,
2002; Simó-Pinatella et al.,
2019). Besides, their prognosis is poor (Emerson et al.,
2011; Odgers et al.,
2008; Reef et al.,
2011), and they are overrepresented in child welfare and justice systems (Kaal,
2010; Slayter & Springer,
2011; Thompson & Morris,
2016). Therefore, interventions are needed that effectively target externalizing problems in this specific population.
For adolescents without intellectual disabilities, cognitive behavior therapy (CBT) is among the most effective approaches to treat externalizing problems (e.g., McCart et al.,
2006; Sukhodolsky et al.,
2004), with an average effect size of 0.52 (95% confidence interval [0.68, 0.36]; Battagliese et al.,
2015). Over the past years, several CBT protocols have been adapted for adolescents with MID–BIF. These adapted protocols are promising, and seem to result in decreases in teacher reported externalizing problems (Schuiringa et al.,
2017). Evidence regarding the overall effects of these adapted CBT protocols is, however, scarce. A meta-analysis into the overall effectiveness of CBT-based interventions for youth with MID–BIF, identified only 12 eligible studies and concluded that the majority of these studies suffer from multiple methodological limitations (Kok et al.,
2016). Therefore, overall effect sizes of CBT for adolescents with/without intellectual disabilities cannot be directly compared. It is assumed, however, that the overall effects of CBT for adolescents with MID–BIF are smaller than the effects for adolescents without intellectual disabilities (Kok et al.,
2016). It is thus important that the treatment approaches of CBT protocols are matched to the needs of these adolescents. The first aim of the current experimental study is, therefore, to enable finetuning of adolescent focused CBT protocols, by examining the relative effects of two approaches to treat externalizing problems among adolescents with MID–BIF: a cognitive and behavioral approach to emotion regulation training.
Relative Effects of Cognitive Versus Behavioral Treatment Approaches
Historically, behavioral approaches, such as applied behavior analysis, are widely used for adolescents with MID–BIF (Remington,
1998), and for a long time, cognitive difficulties were considered a contraindication for the use of cognitive approaches (Cooney et al.,
2018; Willner,
2006). Recently, this assumption was rebutted, with empirical research showing that individuals with MID–BIF seem to possess the basic cognitive skills that are required to engage in CBT-based treatments (e.g., Cooney et al.,
2018; Oathamshaw & Haddock,
2006; Taylor et al.,
2008). As a result, treatment protocols for adolescents with externalizing behavior and MID–BIF currently consist of a combined cognitive and behavioral approach.
This combined cognitive and behavioral approach may be especially visible for treatment elements that focus on a specific underlying mechanism of externalizing problems: emotion regulation training. Emotion regulation is a multi-modal construct that includes
cognitive strategies (i.e., reappraisal) and
behavioral strategies (i.e., behavioral strategies such as distraction) (Naragon-Gainey et al.,
2017). Likewise, CBT interventions that target emotion regulation also differ in the relative degree to which they use a cognitive or behavioral approach (Menting et al.,
2016). Some interventions focus more on cognitive strategies, and train these strategies through cognitive approaches (i.e., “thought exercises” such as cognitive restructuring), whereas other interventions have a stronger focus on behavioral strategies with behavioral approaches (i.e., “behavioral exercises” such as role-play).
Empirical studies, however, mainly examined the overall effectiveness of these so-called
cocktail treatment packages (Leijten et al.,
2015). Therefore, the debate about the relative effects of cognitive and behavioral training approaches for adolescents with MID–BIF still continues (i.e., McGillivray & Kershaw,
2015; Vereenooghe & Langdon,
2013), with two contrasting views. On the one hand, it has been suggested that individuals with MID–BIF are more likely to benefit from behavioral rather than cognitive approaches (Sturmey,
2004,
2006). Problems with abstract thinking, might hinder the effectiveness of a cognitive approach, because this approach relies on a variety of abstract verbal skills (Sturmey,
2004). Adolescents with MID–BIF might, for example, find it more difficult to use hypothetical thinking, an abstract cognitive instruction that is frequently included in thought exercises. On the other hand, it has been suggested that cognitive approaches may be more effective than behavioral approaches (Lindsay,
2006; McGillivray & Kershaw,
2015). Adolescents with intellectual disabilities might, for example, be more likely to already use behavioral strategies to regulate their emotions, which leaves more room for the improvement of cognitive strategies (te Brinke et al.,
2021).
Although direct empirical evidence about the relative effects of cognitive and behavioral approaches is lacking for adolescents with externalizing problems and MID–BIF, some indications can be drawn from the adult intellectual disability literature. A pilot study into the effectiveness of CBT for anxiety showed that, after intervention termination, the majority of participating adults with mild intellectual disabilities could be rated competent in specific cognitive skills, such as generating alternative thoughts (Roberts & Kwan,
2018). The findings of another pilot study, into the effectiveness of CBT for depression, found that the participating adults with mild intellectual disabilities who received the intervention improved in their ability to infer emotions and thoughts based on various situation–thought–emotion pairings, whereas the participants who received treatment as usual, did not improve (Hartley et al.,
2015). In both of these pilot studies, post-treatment implementation or improvement in behavioral skills was, however, not examined. A study into the differential effectiveness of a cognitive, behavioral, and combined treatment approach for depression among adults with mild intellectual disabilities found no differences between the three approaches (McGillivray & Kershaw,
2015). Although it thus seems that adults with internalizing problems and mild intellectual disabilities may benefit from both cognitive and behavioral approaches, findings from adult internalizing problems cannot simply be generalized to adolescent externalizing behavior, due to maturation effects and differential roles of cognition and behavior in internalizing and externalizing problems. Therefore, the first aim of this study is to test the contrasting hypotheses that for adolescents with externalizing problems and MID–BIF: (1) a cognitive approach is more effective versus (2) a behavioral approach is more effective.
Sequence Effects of Cognitive Versus Behavioral Treatment Approaches
Besides the relative effects of a cognitive and behavioral approach, the current study also examines possible sequence effects of offering either of those two approaches as first or as second. Because CBT protocols for adolescents with MID–BIF typically consists of a combined treatment approach, cognitive and behavioral approaches are currently offered in conjunction across different phases of the treatment, without specifying the most optimal sequence. Thus, little is known with regard to sequence effects.
From a theoretical viewpoint, effects of both sequences seem plausible. Adolescents with MID–BIF often have difficulties to understand abstract cognitive instructions that are currently included in CBT protocols for adolescents with MID–BIF (i.e., hypothetical scenario's, van den Bogaard et al.,
2020). Given these difficulties, it may be that abstract cognitive instructions (i.e., thought exercises, cognitive restructuring, hypothetical scenario’s) are only properly understood after behavioral exercises have made participants familiar with the treatment concepts. Alternatively, it may be that cognitive change (i.e., use of cognitive regulation strategies) only transfers to behavior change when cognitive training is followed by behavioral exercises in different contexts (i.e., role plays to practice regulation strategies). This last possibility would be in accordance to theories of treatment motivation, in which cognitive aspects (considering change) are expected to
precede behavioral aspects (acting on the desired behavior change) (DiClemente & Velasquez,
2002). Given that both sequences are theoretically plausible, the second aim of this study is to examine the contrasting hypotheses that for adolescents with externalizing problems and MID–BIF: (1) a cognitive–behavioral sequence is more effective, versus (2) a behavioral–cognitive sequence is more effective.
Current Study
In the current study, we examine which treatment approach (cognitive versus behavioral) and which sequence (cognitive–behavioral versus behavioral–cognitive) is more effective in decreasing emotion regulation difficulties and externalizing problems among adolescents with MID–BIF, by using a micro-trial design. This is the main outcome paper from this micro-trial. Micro-trials have been described as randomized experiments testing the effects of brief and focused environmental manipulations, such as different treatment modules (Howe et al.,
2010; Lochman et al.,
2019). The environmental manipulation of the current study is an experimental training (the Think Cool Act Cool training; te Brinke et al.,
2018) consisting of two modules: a cognitive module (Think Cool) and a behavioral module (Act Cool). Both modules cover the same general topics in the same number of sessions, from respectively a cognitive or behavioral approach. The protocol focuses on the specific skill emotion regulation, because inadequate emotion regulation is a well-known maintaining factor of externalizing problems (McLaughlin et al.,
2011), and many evidence-based CBT treatments for externalizing problems focus on enhancing emotion regulations skills (Garland et al.,
2008; Menting et al.,
2016).
Participating adolescents received both the Think Cool and Act Cool module, but were randomly assigned to a module sequence condition, which enabled us to examine the following research questions: (1) Which module (Think Cool versus Act Cool) is more effective? (2) Which sequence (Think–Act Cool versus Act–Think Cool) is more effective? Effects of the experimental manipulation were assessed from both a randomized pre-post-measurement perspective, with assessments before and after each of the two experimental modules, and an intensive longitudinal data perspective, with continuing weekly assessments during a baseline period and two experimental phases in which adolescents received the two experimental modules. This assessment method enabled us to examine both inter-individual (between–group) differences and intra-individual (within-person) change during the cognitive and behavioral module.
The Experimental Modules
Participants received the manualized experimental Think Cool Act Cool emotion regulation training for adolescents with MID–BIF (te Brinke et al.,
2018). The training is based on components of evidence-based treatments for adolescents with externalizing problems, such as Coping Power (Lochman et al.,
2015) and Aggression Replacement Training (Currie et al.,
2012), and adjusted to the cognitive capacities of adolescents with MID–BIF (de Wit et al.,
2011). The main adjustments included simplification of language use, longer treatment sessions with fewer skills per session, and use of visual aids.
The training consists of an introduction session and ten individual 60-min sessions (five cognitive sessions “Think Cool” and five behavioral sessions “Act Cool”). Both modules use a three-step approach of regulating emotions but consist of either a cognitive or behavioral approach. Care was taken to ensure that both modules are identical in all other ways, such as dosage, timing, structure and lay out of materials. In both modules, adolescents were instructed to make daily at-home assignments. The content of the modules is displayed in Table
1.
Table 1
Content of the Think Cool Act Cool Plus Emotion Regulation Training
Acquaintance Session | •Participant and clinician get to know each other •Training objectives are explained •Brainstorm about words for anger •Formulate personal training goals | •Participant and clinician get to know each other •Training objectives are explained •Brainstorm about words for anger •Formulate personal training goals |
Session 1/6 | •Make or adjusta an anger thermometer, based on situations, bodily sensations and cognitions •Explain the Think Cool Chain •Practice with regulation strategy “think about something fun” (cognitive distraction) •Introduce at-home assignments | •Make or adjusta an anger thermometer, based on situations, bodily sensations and behaviors •Explain the Act Cool Chain Practice with regulation strategy “do something fun” (behavioral distraction) •Introduce at-home assignments |
Session 2/7 | •Look back and discuss at-home assignments •Practice regulation strategy “talk in your head” (cognitive relaxation) Practice regulation strategy “helping thoughts” (cognitive reappraisal) •Summarize and discuss new at-home assignment | •Look back and discuss at-home assignments •Practice regulation strategy “deep breathing” (behavioral relaxation) •Practice regulation strategy “time out” (behavioral modification) •Summarize and discuss new at-home assignment |
Session 3/8 | •Look back and discuss at-home assignments •Practice to look at a situation from multiple viewpoints •Introduce cognitive problem solving •Practice perspective taking with •Summarize and discuss new at-home assignment | •Look back and discuss at-home assignments •Practice behavioral problem solving skills (set a boundary, ask for help) •Summarize and discuss new at-home assignment |
Session 4/9 | •Look back and discuss at-home assignments •Practice cognitive problem solving •Summarize and discuss new at-home assignment | •Look back and discuss at-home assignments •Practice behavioral problem solving in difficult situations (e.g., disappointments, frustration) Summarize and discuss new at-home assignment |
Session 5/10 | •Look back and discuss at-home assignments •Practice complete Think Cool Chain | •Look back and discuss at-home assignments •Practice complete Act Cool Chain |
Think Cool
In the Think Cool module, participants learn to regulate emotions via a cognitive approach. This module is based on the Think Cool chain, a cognitive approach to emotion regulation that is typically used in current interventions (e.g., Goldstein et al.,
1987; Lochman et al.,
2008). The first step of the chain (Session 1) is to signal anger, with an anger thermometer that is based on situations, feelings, sensations and cognitions (e.g., “they always blame me”). The second step (Session 1 and 2) is to practice three cognitive emotion regulation strategies (cognitive distraction, cognitive relaxation and cognitive reappraisal). The third step (Session 3, 4, and 5) is cognitive problem solving (understand a problem from multiple perspectives, think about possible solutions and possible consequences of these solutions, decide which is the most suitable solution), which is practiced with step-by-step paper-and-pencil exercises.
Act Cool
In the Act Cool module, participants learn to regulate emotion via a behavioral approach, with the Act Cool Chain, consisting of a behavioral approach to emotion regulation that is typically used in current interventions (e.g., Albrecht & Spanjaard,
2011; Goldstein et al.,
1987). The first step of the Act Cool Chain (Session 1), is to signal anger with an anger thermometer, similar to the thermometer that is used in the Think Cool module. However, in the Act Cool module, the thermometer is based on behaviors (e.g., “if I become angry, I raise my voice”) rather than cognitions. The second step (Session 1 and 2) is to practice behavioral emotion regulation strategies (behavioral distraction, behavioral relaxation and time out). The third step (Session 3, 4, and 5) is behavioral problem solving (behavioral role-play exercises). Adolescents learn specific behavioral skills (set a boundary, ask for help) and practice these skills in difficult situations (e.g., disappointments, frustration).
Delivery of the Experimental Training
Attendance
In total, 23 adolescents (54.8% of the total sample) discontinued the experimental training at some point, after following on average three out of the ten sessions (see Fig.
1). The main reasons from discontinuation of the experimental training were no show/resistance and practical/scheduling issues. Treatment discontinuation during the first (χ
2 = 1.67,
p = 0.197) and second (χ
2 = 0.19,
p = 0.662) experimental phase did not differ between modules. Adolescents who continued the training followed on average nine out of the ten sessions (Think Cool
: M = 4.74, SD = 0.56, Act Cool:
M = 4.42, SD = 1.12).
Delivery
The modules were carried-out by 12 experienced clinicians (i.e., clinical psychologists, psychomotor therapists or social workers) who worked at the treatment center. Before the start of the study, all participating clinicians received a two-day training course, guided by the developers of the training manual (te Brinke et al.,
2018). To measure whether the experimental treatment modules were delivered as intended, all sessions were audio taped. Subsequently, 67 sessions (28% of all delivered sessions) were randomly selected and independently coded by four trained research assistants on two main components of treatment integrity: adherence and differentiation (McLeod et al.,
2009). The coding scheme was based on previous studies (e.g., McLeod et al.,
2015; Schuiringa et al.,
2017) and all components were rated on a 4-point scale (1 =
totally not, 2 =
slightly, 3 =
mostly, 4 =
totally). To assess inter-rater reliability, 40% of all coded sessions were coded by two raters. Average scale intra class coefficients (ICCs) were calculated as two-way random coefficients, and the codes of the coder who first rated the session were used in the analyses.
Adherence to Training Manual
To measure treatment adherence, coders rated the degree to which a clinician delivered general session content (e.g., “discussed homework assignment”, four items per session), session-specific content (e.g., “performed helping thoughts exercise”, 6–13 items per session), and session-specific goals (e.g., “adolescent learns to identify problems”, 2–3 items per session). Interrater reliability was excellent (Cicchetti,
1994), ranging from 0.79 (session-specific content) to 0.89 (general content). Treatment adherence may be considered good in both modules, with high average percentages scored as “totally” or “mostly” for general session content (Think Cool:
M% = 81.31, SD = 22.34, Act Cool:
M% = 69.36, SD = 25.76,
F(1,65) = 4.10,
p = 0.047), session-specific content (Think Cool:
M% = 88.48, SD = 17.56, Act Cool:
M% = 75.76, SD = 32.78,
F(1,64) = 3.86,
p = 0.054), and session-specific goals (Think Cool:
M% = 87.88, SD = 22.54, Act Cool:
M% = 71.72, SD = 40.72, SD = 25.76,
F(1,64) = 3.52,
p = 0.050).
Differentiation Between Modules
Treatment differentiation, or the extent to which a treatment module excluded content of the opposing module (i.e., content bleeding) and matched the underlying theory (Schulte et al.,
2009), was measured by coding both positive differentiation (e.g., “there was a focus on cognitions/behavior”, three items per session) and negative differentiation (e.g., “the clinician included content of the opposing module”, two items per session). Interrater reliability for positive differentiation was excellent (0.96) (Cicchetti,
1994). Overall, differentiation between the two modules was high, with trainers displaying high degrees of theoretical focus (positive differentiation scored as “totally”: Think Cool:
M% = 95.96, SD = 11.05, Act Cool:
M% = 73.74, SD = 34.11,
F(1,64) = 12.67,
p = 0.001). Findings for negative differentiation need to be interpreted with caution due to the poor (0.34) agreement between coders (Cicchetti,
1994). Clinicians seemed to include, however, little content of the opposing modules (negative differentiation scored as “totally not”: Think Cool:
M% = 93,94, SD = 16.57, Act Cool:
M% = 98,49, SD = 8.70,
F(1,64) = 1.95,
p = 0.168).
Data Analyses
First, differences between the two conditions at baseline were examined. Subsequently, between–group differences in approach (Think Cool versus Act Cool) and sequence (Think–Act versus Act–Think) were examined with Structural Equation Modeling (SEM) path analyses in Mplus version 8.1, using the MLR estimator (Muthén & Muthén,
2017). The advantage of SEM is that all randomized participants who completed the pretest assessment can be included in the analyses, because a full information approach is used. In addition, the MLR estimator produces standard errors that are robust in the case of a non-normal distribution (Muthén & Muthén,
2017).
To examine the effect of module during the first experimental phase, T2 was regressed on T1 and module (0 = Think Cool, 1 = Act Cool). To examine the effect of module during the second experimental phase, T3 was regressed on T2 and module (0 = Act Cool, 1 = Think Cool). To examine the effect of sequence, T3 was regressed on T1 and sequence (0 = Think–Act, 1 = Act–Think). For all regression analyses, effect size estimates were computed as Cohen’s
d, with a two-step approach, as recommended by Feingold (
2019). First, the pooled standard deviation was calculated (√(SD
2group1 + SD
2group2)/2), and subsequently, Cohen’s
d was calculated by using the MODEL CONSTRAINT option in Mplus. An effect size of
d = 0.2 was considered small,
d = 0.5 medium and
d = 0.8 large (Cohen,
1992).
Within-person change during the Think Cool and Act Cool module was examined with Piecewise Hierarchical Linear Growth Models in HLM8, using the RML estimator (Raudenbush et al.,
2019). Hierarchical models are best suited to examine trajectories of change, because these models take the nested structure of the data into account, with weekly measurements (Level 1) nested within participants (Level 2) (Tasca & Gallop,
2009). Another advantage of this framework is that it handles unbalanced designs efficiently, allowing the number of observations to vary across participants. Weekly emotion regulation and aggression were entered as dependent variables, and models were examined separately for participants in the Think–Act and Act–Think sequence conditions, because we were specifically interested in within-person (Level 1) change. All time coefficients were modeled as random effects, which allows for variation across participants.
First, the stability of emotion regulation difficulties and aggression during the baseline period (Week 1–3) was examined, with univariate linear growth models. If the change during baseline was not significantly different from zero (i.e., baseline stability), weekly baseline assessments were coded as “0” in subsequent models, to model the average level of emotion regulation difficulties and aggression before the start of the experimental manipulation. Subsequent weekly assessments were scaled, to represent the change in outcome across a one-unit change in time during the first (Week 4–10) and second (Week 11–17) experimental phase. To examine within-person change during the cognitive and behavioral module, piecewise linear growth models were examined, in which the slopes of the first and second experimental phase were modeled simultaneously. All analyses were performed according to the intention to treat principle (Kruse et al.,
2002).
Discussion
This experimental study aimed to examine which treatment approach (cognitive versus behavioral) and which treatment sequence (cognitive–behavioral versus behavioral–cognitive) is most effective in decreasing emotion regulation difficulties and externalizing problems among adolescents with MID–BIF. The study was conducted in the context of a residential treatment center, which enabled us to include a sample of difficult-to-reach adolescents. With regard to approach effects, the results suggested that by itself, the cognitive module appeared to be more effective than the behavioral module. Specifically, between–group analyses showed that after the first module, clinical-staff reported lower levels of externalizing problems for the group of adolescents who first received the cognitive module than for the group of adolescents who first received the behavioral module. Within-person analyses partly confirmed this finding, by showing that a stable baseline period was followed by a significant within-person decrease in weekly self-reported emotion regulation difficulties during the cognitive module for adolescents in the cognitive–behavioral sequence. Although findings were somewhat mixed, with significant between–person effects for clinical-staff reported measures (i.e., externalizing problems) but not for self-reported measures, and significant within-person effects for only one out of two self-reported measures (i.e., emotion regulation), these findings show more favorable outcomes for adolescents who followed the cognitive module during the first experimental phase in comparison to adolescents who followed the behavioral module during the first experimental phase. The finding that the cognitive module appeared to be more effective, is in line with theoretical views in which the relevance of cognitive approaches for individuals with intellectual disabilities are emphasized (e.g., Lindsay,
2006; McGillivray & Kershaw,
2015). Besides, the finding that the behavioral module appeared to be less effective in decreasing externalizing problems, is in contrast to the idea that behavioral approaches are more effective for individuals with intellectual disabilities (e.g., Sturmey,
2004,
2006).
A possible explanation for the lack of significant between–person approach effects for self-reported outcomes after the first experimental phase, may be the relatively small sample size. Moreover, it is possible that adolescents with externalizing problems and MID–BIF may find it relatively difficult to reflect on subtle changes in their emotion regulation skills and externalizing problems (te Brinke et al.,
2021). Thus, the mixed effects can be attributed to methodological aspects and characteristics of the target population. It should also be noted that treatment drop-out was relatively high, after both the cognitive and the behavioral modules. On first glance, the relatively high drop-out may seem somewhat surprising, given that the study was conducted in a residential treatment center which eliminates some barriers to treatment such as traveling or timing issues. The relatively high drop-out might therefore be attributed to well-known issues with treatment fidelity in residential settings (i.e., stress experienced by staff due to administrative and organizational barriers; Kocken et al.,
2019), rather than characteristics of this study.
With regard to sequence effects, the results of the current study suggested that it is most effective to include behavioral exercises
after cognitive training. Specifically, at the end of the experimental training, self-reported emotion regulation difficulties were lower in the group of adolescents who received the cognitive–behavioral sequence than in the group of adolescents who received the behavioral–cognitive sequence. Although this sequence effect was not significant for externalizing problems, it should be noted that the between–group differences in both self-reported and staff-reported externalizing problems showed a trend in the same direction. Importantly, this conclusion was also supported by the within-person analyses, as for adolescents who received the behavioral–cognitive sequence, weekly reported emotion regulation difficulties and aggression did not change, whereas for adolescents who received the cognitive–behavioral sequence, there was a decrease in emotion regulation difficulties during the cognitive module. The finding that the cognitive–behavioral sequence appeared to be more effective than the behavioral–cognitive sequence, is in accordance with theories of treatment motivation, in which cognitive aspects (considering change) are expected to precede behavioral aspects (acting on the desired behavior) (DiClemente & Velasquez,
2002).
In addition, some of the findings from the current study also pointed towards a
negative effect of the behavioral–cognitive sequence. When looking at the between–group differences of the second module, we found that emotion regulation difficulties actually
increased after the cognitive module for adolescents who followed the behavioral–cognitive sequence. Thus, the exact same treatment approach may have opposite effects, depending on the order in which it is presented. It is possible that the (ineffective) behavioral module
desensitized adolescents to the (effective) cognitive module. Adolescents may have experienced that behavioral strategies did not seem to work in the first module, which consecutively made them less motivated and more pessimistic about the cognitive strategies in the second module. If so, this suggests that we need to be cautious with iatrogenic effects, that may compromise the effects of further intervention. An alternative explanation may be that behavioral strategies were already used more than cognitive strategies, due to pre-existing preferences of adolescents with MID–BIF (te Brinke et al.,
2021), or previous treatment focus at the residential treatment center, and that as a result, there was less room for improvement in behavioral strategies. Lastly, it is also possible that a behavioral approach is not effective by itself, but only functions as
booster, and thereby fosters the effect of a cognitive approach.
It should be noted, however, that although the current study suggests that the behavioral module was, by itself, less effective than the cognitive module, this does not mean that clinicians should be inclined to omit behavioral techniques in all CBT protocols for adolescents with MID–BIF. The experimental modular training that was examined in the current study focused specifically on enhancing emotion regulation skills among adolescents with externalizing problems. Thus, it cannot be ruled out that a behavioral approach is more effective than a cognitive approach when other underlying mechanisms are targeted (e.g., contingency management for desirable behavior), or when the psychological symptoms of the target group differ (e.g., behavioral activation for adolescent with MID–BIF and depression). Besides, the current study only included adolescent-focused training, whereas a behavioral approach also frequently includes operant conditioning with mediating parents or teachers (i.e., response change of caregivers and teachers). Standing Strong Together, an evidence-based intervention for adolescents with behavioral problems and MID–BIF, includes for example a parent-component, in which several sessions focus on the use of praise and tokens (Schuiringa et al.,
2017). Little is known about the differential effects of these operant behavioral techniques. It could, therefore, be an interesting direction of future micro-trial research to examine the additional effect of behavioral caregiver-oriented approaches, on top of cognitive child-oriented approaches. Moreover, future micro-trials could focus on other mediating mechanisms and/or therapist skills.
Strengths and Limitations
Strengths of the current study include the focus on an understudied, difficult-to-reach population of adolescents with externalizing problems and MID–BIF, the thorough assessment of treatment adherence and differentiation, and the use of multiple assessment methods, which enabled us to examine not only between–group differences, but also within-person change. Between–group differences in externalizing problems were examined from a multi-informant perspective, which might be of additional importance for research among adolescents with MID–BIF, because these adolescents may have difficulties in understanding verbal information, and limited insight into their own behavior (Bramston & Fogarty,
2000; Pavlović et al.,
2013). The addition of within-person analyses of weekly reported emotion regulation difficulties and aggression also strengthens this study, because weekly assessments do not rely on the recall of experiences over longer periods of time. It should, however, be noted that the response rate to the weekly assessments was relatively low (i.e., on average nine out of the 17 weeks), and that for the aggression subscale, the internal consistency was suboptimal. In future research, a validated assessment of weekly problem behaviors may be included. An example of a validated measure is the idiographic Top Problems measure, in which adolescents and/or caregivers identify and repeatedly rate the severity of their top three problems (Weisz et al.,
2011). Such an idiographic approach might potentially increase the response rate of weekly assessments among adolescents with MID–BIF, because it is more personalized.
A limitation of the current study was its small sample size, which might have diminished the power to detect (between–person) differences. This power issue could, potentially, be an explanation for the fact that even though the effect sizes for the differences between the two sequences were large and all in the same direction (i.e., between 0.40 and 0.79), some effects were only significant for adolescent-reported emotion regulation difficulties, and not for externalizing problems. In addition, treatment dropout was high, with more than half of the participants discontinuing treatment prematurely due to resistance and/or practical issues. Treatment dropout is variable among adolescents with MID–BIF (Kok et al.,
2016) and externalizing problems seem to be an important predictor of treatment dropout (de Haan et al.,
2013). Moreover, well-known treatment barriers in residential settings such as repression, high staff turn-over and organizational difficulties might explain the relatively high dropout in the current study (de Valk et al.,
2016; James et al.,
2017). It should be noted that analyses were performed according to the intention to treat principle, and that the degree of missing data was relatively low (i.e., with 81% of adolescents filling out questionnaires at T3). Nevertheless, the dropout is problematic methodologically, as it may have limited the ability to detect differences between the groups. The high treatment drop-out in the present study suggests that a better understanding of reasons for drop-out in residential settings (and ways to prevent it) may be an important priority for future research.
Another limitation of the study was its open design, as adolescents and other informants included in the assessments (i.e., clinical staff) might have been aware of the fact that they received a specific module sequence. We did, however, not explicitly state that we were interested in approach or sequence effects. In addition, the emotion regulation measures were not specifically validated for adolescents with MID–BIF, although research has indicated the internal consistencies of the emotion regulation questionnaires were comparable between a MID–BIF and average intelligence group (te Brinke et al.,
2021). Lastly, participants were recruited through a residential treatment center for adolescents with behavior problems and intellectual disabilities, and the experimental training was added on to their existing treatment. Therefore, we cannot rule out the possibility that participants were already familiar with the therapeutic techniques that were included in the experimental intervention. We do, however, not have reasons to believe that this familiarity might have had a differential impact on one of the two experimental modules, since existing treatments mainly focus on a combination of cognitive and behavioral approaches.
Conclusion and Implications
The most important conclusion of this study is that for adolescents with externalizing behavior problems and MID–BIF, a cognitive approach of an emotion regulation training seems to be more effective than a behavioral approach. Moreover, it may be more effective to include behavioral exercises after cognitive training than vice versa. Although this is the first, relatively small sample experimental study that examined differences between cognitive and behavioral treatment approaches for adolescents with externalizing problems and MID–BIF, these findings may have implications for existing CBT protocols and clinical guidelines. First of all, these findings seem to suggest that when clinicians or treatment developers aim to
enhance the effectiveness of existing CBT protocols for adolescents with externalizing problems and MID–BIF, they preferably start by adjusting the relative focus of the intervention, to make sure that the intervention includes a cognitive approach. Second, guidelines for CBT among adolescents with MID–BIF, in which it is currently stated that within multi-component treatment packages, there should be a stronger focus on behavioral, rather than cognitive approaches (van den Bogaard et al.,
2020), may be adapted, in order to highlight the importance of cognitive approaches for this particular target group. Eventually, these adjustments may lead towards more effective ways to treat externalizing behavior problems among adolescents with MID–BIF.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.