Treatment Approach and Sequence Effects in Cognitive Behavioral Therapy Targeting Emotion Regulation Among Adolescents with Externalizing Problems and Intellectual Disabilities
Auteurs:
Lysanne W. te Brinke, Hilde D. Schuiringa, Ankie T. A. Menting, Maja Deković, Johanna J. Westera, Bram O. de Castro
Over the past years, it has become clear that adapted cognitive behavior therapy can be effective for adolescents with externalizing problems and mild intellectual disabilities or borderline intellectual functioning (MID–BIF). Most adapted treatment protocols consist, however, of a combined cognitive and behavioral approach, even though it is actually unclear which approach is most suitable for these adolescents. This experimental study aimed to examine which treatment approach (cognitive versus behavioral) and which treatment sequence (cognitive–behavioral versus behavioral–cognitive) is most effective.
Methods
Participating adolescents (N = 42, 50% boys, Mage = 15.52, SD = 1.43) consecutively received a cognitive and behavioral emotion regulation training module, but were randomly assigned to a different module sequence condition. Emotion regulation and externalizing problems were measured before and after the modules, and with continuing weekly assessments.
Results
Results indicated that the cognitive module, by itself, was more effective than the behavioral module. In addition, the results indicated that it is most effective to include behavioral exercises after (rather than before) cognitive training.
Conclusions
These findings highlight the importance of cognitive treatment approaches for adolescents with MID–BIF, and show that treatment approaches may have different effects, depending on the order in which they are presented.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Introduction
Adolescents with mild intellectual disabilities or borderline intellectual functioning (MID–BIF1; IQ between 50 and 85 and limitations in social adaptive skills; American Psychiatric Association, 2013) have a higher risk of developing externalizing problems than adolescents without MID–BIF (Dekker et al., 2002; Simó-Pinatella et al., 2019). Besides, their prognosis is poor (Emerson et al., 2011; Odgers et al., 2008; Reef et al., 2011), and they are overrepresented in child welfare and justice systems (Kaal, 2010; Slayter & Springer, 2011; Thompson & Morris, 2016). Therefore, interventions are needed that effectively target externalizing problems in this specific population.
For adolescents without intellectual disabilities, cognitive behavior therapy (CBT) is among the most effective approaches to treat externalizing problems (e.g., McCart et al., 2006; Sukhodolsky et al., 2004), with an average effect size of 0.52 (95% confidence interval [0.68, 0.36]; Battagliese et al., 2015). Over the past years, several CBT protocols have been adapted for adolescents with MID–BIF. These adapted protocols are promising, and seem to result in decreases in teacher reported externalizing problems (Schuiringa et al., 2017). Evidence regarding the overall effects of these adapted CBT protocols is, however, scarce. A meta-analysis into the overall effectiveness of CBT-based interventions for youth with MID–BIF, identified only 12 eligible studies and concluded that the majority of these studies suffer from multiple methodological limitations (Kok et al., 2016). Therefore, overall effect sizes of CBT for adolescents with/without intellectual disabilities cannot be directly compared. It is assumed, however, that the overall effects of CBT for adolescents with MID–BIF are smaller than the effects for adolescents without intellectual disabilities (Kok et al., 2016). It is thus important that the treatment approaches of CBT protocols are matched to the needs of these adolescents. The first aim of the current experimental study is, therefore, to enable finetuning of adolescent focused CBT protocols, by examining the relative effects of two approaches to treat externalizing problems among adolescents with MID–BIF: a cognitive and behavioral approach to emotion regulation training.
Relative Effects of Cognitive Versus Behavioral Treatment Approaches
Historically, behavioral approaches, such as applied behavior analysis, are widely used for adolescents with MID–BIF (Remington, 1998), and for a long time, cognitive difficulties were considered a contraindication for the use of cognitive approaches (Cooney et al., 2018; Willner, 2006). Recently, this assumption was rebutted, with empirical research showing that individuals with MID–BIF seem to possess the basic cognitive skills that are required to engage in CBT-based treatments (e.g., Cooney et al., 2018; Oathamshaw & Haddock, 2006; Taylor et al., 2008). As a result, treatment protocols for adolescents with externalizing behavior and MID–BIF currently consist of a combined cognitive and behavioral approach.
This combined cognitive and behavioral approach may be especially visible for treatment elements that focus on a specific underlying mechanism of externalizing problems: emotion regulation training. Emotion regulation is a multi-modal construct that includes cognitive strategies (i.e., reappraisal) and behavioral strategies (i.e., behavioral strategies such as distraction) (Naragon-Gainey et al., 2017). Likewise, CBT interventions that target emotion regulation also differ in the relative degree to which they use a cognitive or behavioral approach (Menting et al., 2016). Some interventions focus more on cognitive strategies, and train these strategies through cognitive approaches (i.e., “thought exercises” such as cognitive restructuring), whereas other interventions have a stronger focus on behavioral strategies with behavioral approaches (i.e., “behavioral exercises” such as role-play).
Empirical studies, however, mainly examined the overall effectiveness of these so-called cocktail treatment packages (Leijten et al., 2015). Therefore, the debate about the relative effects of cognitive and behavioral training approaches for adolescents with MID–BIF still continues (i.e., McGillivray & Kershaw, 2015; Vereenooghe & Langdon, 2013), with two contrasting views. On the one hand, it has been suggested that individuals with MID–BIF are more likely to benefit from behavioral rather than cognitive approaches (Sturmey, 2004, 2006). Problems with abstract thinking, might hinder the effectiveness of a cognitive approach, because this approach relies on a variety of abstract verbal skills (Sturmey, 2004). Adolescents with MID–BIF might, for example, find it more difficult to use hypothetical thinking, an abstract cognitive instruction that is frequently included in thought exercises. On the other hand, it has been suggested that cognitive approaches may be more effective than behavioral approaches (Lindsay, 2006; McGillivray & Kershaw, 2015). Adolescents with intellectual disabilities might, for example, be more likely to already use behavioral strategies to regulate their emotions, which leaves more room for the improvement of cognitive strategies (te Brinke et al., 2021).
Although direct empirical evidence about the relative effects of cognitive and behavioral approaches is lacking for adolescents with externalizing problems and MID–BIF, some indications can be drawn from the adult intellectual disability literature. A pilot study into the effectiveness of CBT for anxiety showed that, after intervention termination, the majority of participating adults with mild intellectual disabilities could be rated competent in specific cognitive skills, such as generating alternative thoughts (Roberts & Kwan, 2018). The findings of another pilot study, into the effectiveness of CBT for depression, found that the participating adults with mild intellectual disabilities who received the intervention improved in their ability to infer emotions and thoughts based on various situation–thought–emotion pairings, whereas the participants who received treatment as usual, did not improve (Hartley et al., 2015). In both of these pilot studies, post-treatment implementation or improvement in behavioral skills was, however, not examined. A study into the differential effectiveness of a cognitive, behavioral, and combined treatment approach for depression among adults with mild intellectual disabilities found no differences between the three approaches (McGillivray & Kershaw, 2015). Although it thus seems that adults with internalizing problems and mild intellectual disabilities may benefit from both cognitive and behavioral approaches, findings from adult internalizing problems cannot simply be generalized to adolescent externalizing behavior, due to maturation effects and differential roles of cognition and behavior in internalizing and externalizing problems. Therefore, the first aim of this study is to test the contrasting hypotheses that for adolescents with externalizing problems and MID–BIF: (1) a cognitive approach is more effective versus (2) a behavioral approach is more effective.
Sequence Effects of Cognitive Versus Behavioral Treatment Approaches
Besides the relative effects of a cognitive and behavioral approach, the current study also examines possible sequence effects of offering either of those two approaches as first or as second. Because CBT protocols for adolescents with MID–BIF typically consists of a combined treatment approach, cognitive and behavioral approaches are currently offered in conjunction across different phases of the treatment, without specifying the most optimal sequence. Thus, little is known with regard to sequence effects.
From a theoretical viewpoint, effects of both sequences seem plausible. Adolescents with MID–BIF often have difficulties to understand abstract cognitive instructions that are currently included in CBT protocols for adolescents with MID–BIF (i.e., hypothetical scenario's, van den Bogaard et al., 2020). Given these difficulties, it may be that abstract cognitive instructions (i.e., thought exercises, cognitive restructuring, hypothetical scenario’s) are only properly understood after behavioral exercises have made participants familiar with the treatment concepts. Alternatively, it may be that cognitive change (i.e., use of cognitive regulation strategies) only transfers to behavior change when cognitive training is followed by behavioral exercises in different contexts (i.e., role plays to practice regulation strategies). This last possibility would be in accordance to theories of treatment motivation, in which cognitive aspects (considering change) are expected to precede behavioral aspects (acting on the desired behavior change) (DiClemente & Velasquez, 2002). Given that both sequences are theoretically plausible, the second aim of this study is to examine the contrasting hypotheses that for adolescents with externalizing problems and MID–BIF: (1) a cognitive–behavioral sequence is more effective, versus (2) a behavioral–cognitive sequence is more effective.
Current Study
In the current study, we examine which treatment approach (cognitive versus behavioral) and which sequence (cognitive–behavioral versus behavioral–cognitive) is more effective in decreasing emotion regulation difficulties and externalizing problems among adolescents with MID–BIF, by using a micro-trial design. This is the main outcome paper from this micro-trial. Micro-trials have been described as randomized experiments testing the effects of brief and focused environmental manipulations, such as different treatment modules (Howe et al., 2010; Lochman et al., 2019). The environmental manipulation of the current study is an experimental training (the Think Cool Act Cool training; te Brinke et al., 2018) consisting of two modules: a cognitive module (Think Cool) and a behavioral module (Act Cool). Both modules cover the same general topics in the same number of sessions, from respectively a cognitive or behavioral approach. The protocol focuses on the specific skill emotion regulation, because inadequate emotion regulation is a well-known maintaining factor of externalizing problems (McLaughlin et al., 2011), and many evidence-based CBT treatments for externalizing problems focus on enhancing emotion regulations skills (Garland et al., 2008; Menting et al., 2016).
Participating adolescents received both the Think Cool and Act Cool module, but were randomly assigned to a module sequence condition, which enabled us to examine the following research questions: (1) Which module (Think Cool versus Act Cool) is more effective? (2) Which sequence (Think–Act Cool versus Act–Think Cool) is more effective? Effects of the experimental manipulation were assessed from both a randomized pre-post-measurement perspective, with assessments before and after each of the two experimental modules, and an intensive longitudinal data perspective, with continuing weekly assessments during a baseline period and two experimental phases in which adolescents received the two experimental modules. This assessment method enabled us to examine both inter-individual (between–group) differences and intra-individual (within-person) change during the cognitive and behavioral module.
Method
Design
This study used a randomized parallel group micro-trial design with two conditions. Participants were recruited from a residential treatment center for adolescents with intellectual disabilities and behavior problems, and randomly assigned to a modular sequence condition: they either received first the cognitive and then the behavioral module or the reverse sequence. Randomization took place on the individual level by means of computer-generated random numbers. Ethical approval for this study was granted by the independent medical ethics committee of the University Medical Center, Utrecht. The study was pre-registered in the Dutch CCMO trial register (NL61104.041.17).
Eligibility Criteria
Adolescents were selected to participate in the current study when: (1) they scored above the 84th percentile on the externalizing behavior subscale of the Child Behavior Checklist (CBCL Dutch norm scores; Achenbach & Rescorla, 2001; Verhulst & van der Ende, 2001) as reported by clinical-staff, and (2) had a mild intellectual disability (IQ 55–69) or borderline intellectual functioning (IQ 70–84). Exclusion criteria were: (1) severe autism spectrum symptoms, as evidenced by an autism spectrum diagnosis and a score above the 98th percentile on the Autism Spectrum Questionnaire (ASV; van der Ploeg & Scholte, 2014) reported by clinical-staff, and (2) assumed insufficient Dutch language skills and/or visual and auditorily disability to understand questionnaires and the experimental training.
Procedure
First, the treatment center sent out information letters to all adolescents who were eligible to participate in the study according to the on-site research coordinator (i.e., based on practical considerations such as expected length of stay and their age). Informed consent was obtained from adolescents and the parent(s) or legal guardian(s) of adolescents aged 12–15 (for adolescents aged 16–18 consent of a parent/guardian was not required, though their parent(s) or legal guardian(s) were always informed about the study). Second, the residential clinical-staff filled out the screening measures (see measures). In addition, information about the adolescent’s IQ score and possible ASS diagnosis was obtained. If this information was not available, a short IQ-test was administered by a trained research assistant (see measures).
Subsequently, adolescents who met the inclusion criteria participated in three phases of the experiment; a baseline phase, the first experimental phase, and the second experimental phase (see Fig. 1). During the two experimental phases, each lasting for 7 weeks, adolescents received the cognitive and behavioral experimental module, both consisting of five individual sessions. Data collection consisted of both adolescent self-report and clinical-staff reported measurements (T1 = Pretest, T2 = Posttest phase one, T3 = Posttest phase two), and continuing weekly adolescent self-report measurements (3 weeks during baseline, 7 weeks during phase one, 7 weeks during phase two). Adolescent self-reports at T1–T3 were administered in interview format, to ensure comprehension. Weekly measures and clinical-staff reported measures were administered online. At T2 and T3, adolescents received a small monetary reward for filling out the questionnaires.
Fig. 1
Participant flow diagram
×
Participants
The flow of participants is displayed in Fig. 1. In total, 75 adolescents were assessed for eligibility, and 42 adolescents (50% boys) met the inclusion criteria. The participating adolescents were between 12 and 18 years old (Mage = 15.52, SD = 1.43). The majority of participating adolescents (81%) was born in the Netherlands. Of the adolescents that were not born in the Netherlands, 37.5% was born in an East-European country, 25% in a West-European country, 25% in a South-American country, and 12.5% in the Caribbean. However, for 59.5% of the sample, at least one parent was born in a different country than the Netherlands, and therefore, these adolescents were considered of non-Dutch ethnicity (Keij, 2000).
Of the adolescents who filled out the questionnaires at T1, 85.7% also filled out the questionnaires at T2, and 81.0% also filled out the questionnaires at T3 (see Fig. 1). Clinical-staff reported questionnaires were available for 90.5% of the adolescents at T2, and 69.0% of the adolescents at T3. The weekly questionnaire was completed on average on nine out of the 17 measurement weeks (with 381 available data points). Little’s Missing Completely At Random (MCAR) test showed that adolescent T1-T3 measurement data (χ2/df = 1.34, p = 0.234), clinical-staff T1-T3 measurement data (χ2/df = 2.28, p = 0.077), and weekly measurement data (χ2/df = 0.78, p = 0.999) were missing completely at random.
Measures
Screening Measures
Externalizing Problems
Clinical-staff reported externalizing problems were assessed with the broadband externalizing problems scale of the Dutch version of the CBCL (Achenbach & Rescorla, 2001; Verhulst & van der Ende, 2001). This scale consists of 35 items (e.g., “argues a lot”) that are rated on a 3-point scale from 0 (not true) to 2 (very true or often true). During the screening of the current study, Cronbach’s alpha was 0.90 for the externalizing problems scale.
Autism Spectrum Symptoms
Severity of autism spectrum symptoms was measured with the ASV (van der Ploeg & Scholte, 2014). This questionnaire consists of 24 items (e.g., “does not seek eye contact”) that are rated on a 5-point scale from 1 (totally disagree) to 5 (totally agree). Percentile scores were obtained based on the Dutch norm scores (van der Ploeg & Scholte, 2014). Cronbach’s alpha was 0.90 in the current study.
Intelligence
Intelligence was assessed with the Wechsler Intelligence Scale for Children for participants aged 12–16 (WISC; Kort et al., 2005) or the Wechsler Adult Intelligence Scale for participants aged 17–18 (WAIS; Wechsler, 2012). A full-scale IQ score from the adolescents’ clinical file was used in case an intelligence test (WISC or WAIS) was administered within 24 months before the start of the study. If this score was not available, the subtests “Block Design” and “Vocabulary” were administered by research assistants who received extensive training in IQ test administration. Subsequently, total-IQ was estimated with the formula for approximation of Full Scale IQ (FIQ; Silverstein, 1970). FIQ estimates are found to be reliable and strongly correlated with the total-IQ (Hrabok et al., 2014) and have also been used in MID–BIF populations (Schuiringa et al., 2017). In the current study, FIQ estimates were used for 33.3% of the adolescents.
T1–T3 measures
Emotion Regulation Difficulties
Emotion regulation difficulties were measured with a short Dutch version of the Difficulties in Emotion Regulation Scale (DERS; de Castro et al., 2018; Gratz & Roemer, 2004; Neumann et al., 2010). Adolescents rated 15 items (e.g., “when I am upset, I become out of control”) on a 5-point scale from 1 (almost never) tot 5 (almost always). Cronbach’s alpha ranged from 0.89 to 0.91 across measurement moments. The DERS measure is not specifically validated for adolescents with MID–BIF but has been used in this population before (te Brinke et al., 2021).
Externalizing Problems
Externalizing problems were assessed from a multi-informant perspective, with subscales of the Dutch ASEBA-questionnaires (Achenbach & Rescorla, 2001; Verhulst & van der Ende, 2001). Adolescents (Youth Self Report; YSR) and clinical-staff (Child Behavior Checklist; CBCL) completed, respectively the 32 and 35 items of the broadband externalizing scale. For the clinical staff pre-test report, the externalizing behavior screening scores were used, as the screening took place just before the start of the study. Items (e.g., “I fight a lot / fights a lot”) were rated on a 3-point scale from 0 (not true) to 2 (very true or often true). Cronbach’s alpha ranged from 0.83 to 0.85 across measurement moments for adolescent self-reports and from 0.86 to 0.93 for clinical-staff reports. The ASEBA measure has been validated for adolescents with MID–BIF (Douma et al., 2006).
Weekly Measures
Emotion Regulation Difficulties
Weekly self-perceived emotion regulation difficulties were measured with a 3-item scale, that was based on the DERS (Bjureberg et al., 2016). Items (e.g., “how often did you become so angry this week, that you could not control yourself?”) were rated on a 5-point scale with the following answer options: 1 (never), 2 (a few times), 3 (two to three times), 4 (four to five times), 5 (more often, … times). Cronbach’s alpha ranged from 0.75 to 0.79 across measurement phases. The baseline-average score of the weekly emotion regulation scale was significantly positively correlated with the T1-score of the full-scale DERS (see Supplementary materials Table A).
Aggression
Weekly self-perceived aggression (hitting, kicking and swearing) was measured with a 3-item scale, based on the YSR (Achenbach & Rescorla, 2001). Items (e.g., “how often did you hit someone this week?”) were rated on the same 5-point scale as the weekly emotion regulation measure. Cronbach’s alpha ranged from 0.47 to 0.62 across measurement phases. The baseline-average of the weekly aggressive behavior scale correlated significantly in the expected direction with the T1-score of the YSR (see Supplementary materials Table A).
The Experimental Modules
Participants received the manualized experimental Think Cool Act Cool emotion regulation training for adolescents with MID–BIF (te Brinke et al., 2018). The training is based on components of evidence-based treatments for adolescents with externalizing problems, such as Coping Power (Lochman et al., 2015) and Aggression Replacement Training (Currie et al., 2012), and adjusted to the cognitive capacities of adolescents with MID–BIF (de Wit et al., 2011). The main adjustments included simplification of language use, longer treatment sessions with fewer skills per session, and use of visual aids.
The training consists of an introduction session and ten individual 60-min sessions (five cognitive sessions “Think Cool” and five behavioral sessions “Act Cool”). Both modules use a three-step approach of regulating emotions but consist of either a cognitive or behavioral approach. Care was taken to ensure that both modules are identical in all other ways, such as dosage, timing, structure and lay out of materials. In both modules, adolescents were instructed to make daily at-home assignments. The content of the modules is displayed in Table 1.
Table 1
Content of the Think Cool Act Cool Plus Emotion Regulation Training
Session
Session components
Think Cool module
Session components
Act Cool module
Acquaintance Session
•Participant and clinician get to know each other
•Training objectives are explained
•Brainstorm about words for anger
•Formulate personal training goals
•Participant and clinician get to know each other
•Training objectives are explained
•Brainstorm about words for anger
•Formulate personal training goals
Session 1/6
•Make or adjusta an anger thermometer, based on situations, bodily sensations and cognitions
•Explain the Think Cool Chain
•Practice with regulation strategy “think about something fun” (cognitive distraction)
•Introduce at-home assignments
•Make or adjusta an anger thermometer, based on situations, bodily sensations and behaviors
•Explain the Act Cool Chain
Practice with regulation strategy “do something fun” (behavioral distraction)
•Introduce at-home assignments
Session 2/7
•Look back and discuss at-home assignments
•Practice regulation strategy “talk in your head” (cognitive relaxation)
Practice regulation strategy “helping thoughts” (cognitive reappraisal)
•Practice to look at a situation from multiple viewpoints
•Introduce cognitive problem solving
•Practice perspective taking with
•Summarize and discuss new at-home assignment
•Look back and discuss at-home assignments
•Practice behavioral problem solving skills (set a boundary, ask for help)
•Summarize and discuss new at-home assignment
Session 4/9
•Look back and discuss at-home assignments
•Practice cognitive problem solving
•Summarize and discuss new at-home assignment
•Look back and discuss at-home assignments
•Practice behavioral problem solving in difficult situations (e.g., disappointments, frustration)
Summarize and discuss new at-home assignment
Session 5/10
•Look back and discuss at-home assignments
•Practice complete Think Cool Chain
•Look back and discuss at-home assignments
•Practice complete Act Cool Chain
aDuring the first session of the second module, the existing thermometer is adjusted. Therefore, the column “cognitions/behaviors” from the thermometer that was developed in the first session of the first module, is removed and a new column is added. Besides this, the sessions are the same, irrespective of the sequence in which the modules are followed
Think Cool
In the Think Cool module, participants learn to regulate emotions via a cognitive approach. This module is based on the Think Cool chain, a cognitive approach to emotion regulation that is typically used in current interventions (e.g., Goldstein et al., 1987; Lochman et al., 2008). The first step of the chain (Session 1) is to signal anger, with an anger thermometer that is based on situations, feelings, sensations and cognitions (e.g., “they always blame me”). The second step (Session 1 and 2) is to practice three cognitive emotion regulation strategies (cognitive distraction, cognitive relaxation and cognitive reappraisal). The third step (Session 3, 4, and 5) is cognitive problem solving (understand a problem from multiple perspectives, think about possible solutions and possible consequences of these solutions, decide which is the most suitable solution), which is practiced with step-by-step paper-and-pencil exercises.
Act Cool
In the Act Cool module, participants learn to regulate emotion via a behavioral approach, with the Act Cool Chain, consisting of a behavioral approach to emotion regulation that is typically used in current interventions (e.g., Albrecht & Spanjaard, 2011; Goldstein et al., 1987). The first step of the Act Cool Chain (Session 1), is to signal anger with an anger thermometer, similar to the thermometer that is used in the Think Cool module. However, in the Act Cool module, the thermometer is based on behaviors (e.g., “if I become angry, I raise my voice”) rather than cognitions. The second step (Session 1 and 2) is to practice behavioral emotion regulation strategies (behavioral distraction, behavioral relaxation and time out). The third step (Session 3, 4, and 5) is behavioral problem solving (behavioral role-play exercises). Adolescents learn specific behavioral skills (set a boundary, ask for help) and practice these skills in difficult situations (e.g., disappointments, frustration).
Delivery of the Experimental Training
Attendance
In total, 23 adolescents (54.8% of the total sample) discontinued the experimental training at some point, after following on average three out of the ten sessions (see Fig. 1). The main reasons from discontinuation of the experimental training were no show/resistance and practical/scheduling issues. Treatment discontinuation during the first (χ2 = 1.67, p = 0.197) and second (χ2 = 0.19, p = 0.662) experimental phase did not differ between modules. Adolescents who continued the training followed on average nine out of the ten sessions (Think Cool: M = 4.74, SD = 0.56, Act Cool: M = 4.42, SD = 1.12).
Delivery
The modules were carried-out by 12 experienced clinicians (i.e., clinical psychologists, psychomotor therapists or social workers) who worked at the treatment center. Before the start of the study, all participating clinicians received a two-day training course, guided by the developers of the training manual (te Brinke et al., 2018). To measure whether the experimental treatment modules were delivered as intended, all sessions were audio taped. Subsequently, 67 sessions (28% of all delivered sessions) were randomly selected and independently coded by four trained research assistants on two main components of treatment integrity: adherence and differentiation (McLeod et al., 2009). The coding scheme was based on previous studies (e.g., McLeod et al., 2015; Schuiringa et al., 2017) and all components were rated on a 4-point scale (1 = totally not, 2 = slightly, 3 = mostly, 4 = totally). To assess inter-rater reliability, 40% of all coded sessions were coded by two raters. Average scale intra class coefficients (ICCs) were calculated as two-way random coefficients, and the codes of the coder who first rated the session were used in the analyses.
Adherence to Training Manual
To measure treatment adherence, coders rated the degree to which a clinician delivered general session content (e.g., “discussed homework assignment”, four items per session), session-specific content (e.g., “performed helping thoughts exercise”, 6–13 items per session), and session-specific goals (e.g., “adolescent learns to identify problems”, 2–3 items per session). Interrater reliability was excellent (Cicchetti, 1994), ranging from 0.79 (session-specific content) to 0.89 (general content). Treatment adherence may be considered good in both modules, with high average percentages scored as “totally” or “mostly” for general session content (Think Cool: M% = 81.31, SD = 22.34, Act Cool: M% = 69.36, SD = 25.76, F(1,65) = 4.10, p = 0.047), session-specific content (Think Cool: M% = 88.48, SD = 17.56, Act Cool: M% = 75.76, SD = 32.78, F(1,64) = 3.86, p = 0.054), and session-specific goals (Think Cool: M% = 87.88, SD = 22.54, Act Cool: M% = 71.72, SD = 40.72, SD = 25.76, F(1,64) = 3.52, p = 0.050).
Differentiation Between Modules
Treatment differentiation, or the extent to which a treatment module excluded content of the opposing module (i.e., content bleeding) and matched the underlying theory (Schulte et al., 2009), was measured by coding both positive differentiation (e.g., “there was a focus on cognitions/behavior”, three items per session) and negative differentiation (e.g., “the clinician included content of the opposing module”, two items per session). Interrater reliability for positive differentiation was excellent (0.96) (Cicchetti, 1994). Overall, differentiation between the two modules was high, with trainers displaying high degrees of theoretical focus (positive differentiation scored as “totally”: Think Cool: M% = 95.96, SD = 11.05, Act Cool: M% = 73.74, SD = 34.11, F(1,64) = 12.67, p = 0.001). Findings for negative differentiation need to be interpreted with caution due to the poor (0.34) agreement between coders (Cicchetti, 1994). Clinicians seemed to include, however, little content of the opposing modules (negative differentiation scored as “totally not”: Think Cool: M% = 93,94, SD = 16.57, Act Cool: M% = 98,49, SD = 8.70, F(1,64) = 1.95, p = 0.168).
Data Analyses
First, differences between the two conditions at baseline were examined. Subsequently, between–group differences in approach (Think Cool versus Act Cool) and sequence (Think–Act versus Act–Think) were examined with Structural Equation Modeling (SEM) path analyses in Mplus version 8.1, using the MLR estimator (Muthén & Muthén, 2017). The advantage of SEM is that all randomized participants who completed the pretest assessment can be included in the analyses, because a full information approach is used. In addition, the MLR estimator produces standard errors that are robust in the case of a non-normal distribution (Muthén & Muthén, 2017).
To examine the effect of module during the first experimental phase, T2 was regressed on T1 and module (0 = Think Cool, 1 = Act Cool). To examine the effect of module during the second experimental phase, T3 was regressed on T2 and module (0 = Act Cool, 1 = Think Cool). To examine the effect of sequence, T3 was regressed on T1 and sequence (0 = Think–Act, 1 = Act–Think). For all regression analyses, effect size estimates were computed as Cohen’s d, with a two-step approach, as recommended by Feingold (2019). First, the pooled standard deviation was calculated (√(SD2group1 + SD2group2)/2), and subsequently, Cohen’s d was calculated by using the MODEL CONSTRAINT option in Mplus. An effect size of d = 0.2 was considered small, d = 0.5 medium and d = 0.8 large (Cohen, 1992).
Within-person change during the Think Cool and Act Cool module was examined with Piecewise Hierarchical Linear Growth Models in HLM8, using the RML estimator (Raudenbush et al., 2019). Hierarchical models are best suited to examine trajectories of change, because these models take the nested structure of the data into account, with weekly measurements (Level 1) nested within participants (Level 2) (Tasca & Gallop, 2009). Another advantage of this framework is that it handles unbalanced designs efficiently, allowing the number of observations to vary across participants. Weekly emotion regulation and aggression were entered as dependent variables, and models were examined separately for participants in the Think–Act and Act–Think sequence conditions, because we were specifically interested in within-person (Level 1) change. All time coefficients were modeled as random effects, which allows for variation across participants.
First, the stability of emotion regulation difficulties and aggression during the baseline period (Week 1–3) was examined, with univariate linear growth models. If the change during baseline was not significantly different from zero (i.e., baseline stability), weekly baseline assessments were coded as “0” in subsequent models, to model the average level of emotion regulation difficulties and aggression before the start of the experimental manipulation. Subsequent weekly assessments were scaled, to represent the change in outcome across a one-unit change in time during the first (Week 4–10) and second (Week 11–17) experimental phase. To examine within-person change during the cognitive and behavioral module, piecewise linear growth models were examined, in which the slopes of the first and second experimental phase were modeled simultaneously. All analyses were performed according to the intention to treat principle (Kruse et al., 2002).
Results
Preliminary Analyses
Demographic and screening variables are displayed in Table 2, mean scores of study variables in Table 3, and correlations between study variables in Table A (supplementary materials). The participants in the two sequence conditions did not differ significantly in demographic or screening variables at pretest (Table 2), and there were no significant differences at T1 in difficulties in emotion regulation [F(1,40) = 1.24, p = 0.273], self-reported externalizing problems [F(1,40) = 0.01, p = 0.963], and clinical-staff reported externalizing problems [F(1,40) = 0.01, p = 0.983]. In addition, the average scores of the weekly measure during baseline did not differ between the two conditions [weekly emotion regulation difficulties: F(1,40) = 0.35, p = 0.556, weekly aggression: F(1,40) = 0.04, p = 0.842].
Table 2
Means, standard deviations and group differences for demographic and screening variables
Think–Act (n = 20)
Act–Think (n = 22)
F/χ
p
M
SD
M
SD
Externalizing (T-score)
71.60
5.74
71.68
5.76
0.01
.964
Autism symptoms (sum)
78.65
11.97
79.18
15.06
0.01
.964
IQ-score
74.80
7.40
76.45
7.40
0.66
.421
Age
15.65
1.09
15.41
1.71
0.29
.593
Gender (% male)
50.00
50.00
0.00
.999
Ethnicity (% non-Dutch)
60.00
59.10
0.01
.952
p = significance of difference between sequence conditions at baseline
Table 3
Means and standard deviations (in brackets) of emotion regulation and externalizing problems for the two intervention sequence groups at three measurement moments
Pre-test (T1)
In-between test (T2)
Post-test (T3)
Adolescent self-reported
ER difficulties
1. Think–Act
2.36 (0.74)
2.11 (0.63)
2.02 (0.66)
2. Act–Think
2.11 (0.74)
2.13 (0.74)
2.40 (0.69)
Externalizing problems
1. Think–Act
0.53 (0.25)
0.44 (0.24)
0.38 (0.21)
2. Act–Think
0.52 (0.26)
0.44 (0.20)
0.47 (0.23)
Clinical-staff reported
Externalizing problems
1. Think–Act
0.85 (0.28)
0.62 (0.30)
0.53 (0.28)
2. Act–Think
0.85 (0.28)
0.84 (0.37)
0.74 (0.31)
Between–Group Differences
The results of path analyses are displayed in Table 4, and a graphical display of the means is shown in Fig. 2.
Table 4
Between–person effects of the experimental manipulation on emotion regulation difficulties and externalizing problems during the first experimental phase (effect of module 0 = Think, 1 = Act on T2 with T1 as covariate), the second experimental phase (effect of module 0 = Act, 1 = Think on T3 with T2 as covariate), and the total experimental phase (effect of sequence 0 = Think–Act, 1 = Act–Think on T3 with T1 as covariate)
First experimental phase
Second experimental phase
Total experimental phase
β [95% CI]
p
d
β [95% CI]
p
d
β [95% CI]
p
d
Adolescent self-reported
ER difficulties
0.03 [− 0.19, 0.25]
.321
0.07
0.26 [0.04, 0.49]
.022
0.52
0.30 [0.09, 0.51]
.004
0.63
Externalizing problems
− 0.03 [− 0.22, 0.17]
.773
− 0.06
0.21 [− 0.01, 0.19]
.063
0.42
0.19 [− 0.01, 0.39]
.058
0.40
Clinical-staff reported
Externalizing problems
0.26 [0.04, 0.48]
.020
0.53
0.16 [− .0.08, 0.40]
.184
0.44
0.29 [− 0.01, 0.58]
.057
0.79
Fig. 2
Mean scores at T1 (pretest), T2 (posttest phase one), and T3 (posttest phase two)
×
Effects of the First Module
There was a significant effect with a medium effect size of the first module on clinical-staff reported externalizing behavior problems (Table 4). Examination of the means (see panel C of Fig. 2) shows that the decrease in clinical-staff reported externalizing problems from T1 to T2 was larger in the group of adolescents who first received the Think Cool module, than in the group of adolescents who first received the Act Cool module. Thus, during the first experimental phase, the cognitive module was significantly more effective in decreasing clinical-staff reported externalizing problems than the behavioral module. For the adolescent-reported outcome variables, this effect was, however, not found.
Effects of the Second Module
There was a significant effect with a medium effect size of the second module on difficulties in emotion regulation (Table 4). Emotion regulation difficulties increased from T2 to T3 for adolescents who received the Think Cool module after the Act Cool module, whereas for the group of adolescents who received the Act Cool module after the Think Cool module, emotion regulation difficulties (slightly) decreased from T2 to T3 (see panel A of Fig. 2). Thus, during the second experimental phase, there appeared to be a negative effect of receiving the cognitive module after the behavioral module. For adolescent- and clinical-staff reported externalizing problems, this effect was, however, not found.
Effects of Sequence
There was a significant effect of sequence on adolescent-reported emotion regulation difficulties, with a medium effect size (Table 4). Emotion regulation difficulties decreased from T1 to T3 for adolescents who received the Think–Act sequence, whereas emotion regulation difficulties increased for the group of adolescents who received the Act–Think sequence (see panel A of Fig. 2). Thus, for adolescent-reported emotion regulation difficulties, the cognitive–behavioral sequence appeared to be significantly more effective than the behavioral–cognitive sequence. For adolescent- and clinical-staff reported externalizing problems, this effect of sequence was, however, not found.
Within-Person Change
The results of the univariate and piecewise growth models are displayed in Table 5, and the mean scores of the weekly measure in Table B (Supplementary materials). Baseline stability was established for both emotion regulation difficulties and aggression in both sequences.
Table 5
Within-person fixed effects of change in weekly emotion regulation difficulties and aggression during baseline (univariate Model A) and the Think Cool and Act Cool module (piecewise Model B)
Weekly ER difficulties
Weekly aggression
Coëfficiënt
SE
t ratio
p
Coëfficiënt
SE
t ratio
p
Sequence group Think–Act
Model A: Baseline slope
− 0.05
0.08
− 0.69
.498
0.05
0.06
0.78
.446
Model B: Slope Think
− 0.05
0.02
− 2.38
.028
0.02
0.03
0.29
.775
Model B: Slope Act
0.05
0.03
1.83
.083
− 0.04
0.03
− 1.14
.268
Sequence group Act–Think
Model A: Baseline slope
− 0.07
0.10
− 0.71
.487
− 0.12
0.07
− 1.72
.100
Model B: Slope Act
− 0.03
0.02
− 1.19
.249
0.01
0.02
0.70
.494
Model B: Slope Think
0.02
0.03
0.74
.466
− 0.03
0.02
− 1.32
.202
Think–Act Sequence
There was a significant within-person decrease in emotion regulation difficulties during the Think module, whereas the slope of emotion regulation difficulties was not significantly different from zero during the Act module (Table 5). The slopes of aggression were not significantly different from zero. Thus, for adolescents in the Think–Act sequence condition, emotion regulation difficulties decreased during the Think module, and subsequently stabilized during the Act module, whereas aggression stayed stable.
Act–Think Sequence
The slopes of emotion regulation difficulties and aggression were not significantly different from zero during the Act and Think module (Table 5). Thus, for adolescents in the Act–Think sequence condition, weekly emotion regulation difficulties and aggression stayed stable.
Discussion
This experimental study aimed to examine which treatment approach (cognitive versus behavioral) and which treatment sequence (cognitive–behavioral versus behavioral–cognitive) is most effective in decreasing emotion regulation difficulties and externalizing problems among adolescents with MID–BIF. The study was conducted in the context of a residential treatment center, which enabled us to include a sample of difficult-to-reach adolescents. With regard to approach effects, the results suggested that by itself, the cognitive module appeared to be more effective than the behavioral module. Specifically, between–group analyses showed that after the first module, clinical-staff reported lower levels of externalizing problems for the group of adolescents who first received the cognitive module than for the group of adolescents who first received the behavioral module. Within-person analyses partly confirmed this finding, by showing that a stable baseline period was followed by a significant within-person decrease in weekly self-reported emotion regulation difficulties during the cognitive module for adolescents in the cognitive–behavioral sequence. Although findings were somewhat mixed, with significant between–person effects for clinical-staff reported measures (i.e., externalizing problems) but not for self-reported measures, and significant within-person effects for only one out of two self-reported measures (i.e., emotion regulation), these findings show more favorable outcomes for adolescents who followed the cognitive module during the first experimental phase in comparison to adolescents who followed the behavioral module during the first experimental phase. The finding that the cognitive module appeared to be more effective, is in line with theoretical views in which the relevance of cognitive approaches for individuals with intellectual disabilities are emphasized (e.g., Lindsay, 2006; McGillivray & Kershaw, 2015). Besides, the finding that the behavioral module appeared to be less effective in decreasing externalizing problems, is in contrast to the idea that behavioral approaches are more effective for individuals with intellectual disabilities (e.g., Sturmey, 2004, 2006).
A possible explanation for the lack of significant between–person approach effects for self-reported outcomes after the first experimental phase, may be the relatively small sample size. Moreover, it is possible that adolescents with externalizing problems and MID–BIF may find it relatively difficult to reflect on subtle changes in their emotion regulation skills and externalizing problems (te Brinke et al., 2021). Thus, the mixed effects can be attributed to methodological aspects and characteristics of the target population. It should also be noted that treatment drop-out was relatively high, after both the cognitive and the behavioral modules. On first glance, the relatively high drop-out may seem somewhat surprising, given that the study was conducted in a residential treatment center which eliminates some barriers to treatment such as traveling or timing issues. The relatively high drop-out might therefore be attributed to well-known issues with treatment fidelity in residential settings (i.e., stress experienced by staff due to administrative and organizational barriers; Kocken et al., 2019), rather than characteristics of this study.
With regard to sequence effects, the results of the current study suggested that it is most effective to include behavioral exercises after cognitive training. Specifically, at the end of the experimental training, self-reported emotion regulation difficulties were lower in the group of adolescents who received the cognitive–behavioral sequence than in the group of adolescents who received the behavioral–cognitive sequence. Although this sequence effect was not significant for externalizing problems, it should be noted that the between–group differences in both self-reported and staff-reported externalizing problems showed a trend in the same direction. Importantly, this conclusion was also supported by the within-person analyses, as for adolescents who received the behavioral–cognitive sequence, weekly reported emotion regulation difficulties and aggression did not change, whereas for adolescents who received the cognitive–behavioral sequence, there was a decrease in emotion regulation difficulties during the cognitive module. The finding that the cognitive–behavioral sequence appeared to be more effective than the behavioral–cognitive sequence, is in accordance with theories of treatment motivation, in which cognitive aspects (considering change) are expected to precede behavioral aspects (acting on the desired behavior) (DiClemente & Velasquez, 2002).
In addition, some of the findings from the current study also pointed towards a negative effect of the behavioral–cognitive sequence. When looking at the between–group differences of the second module, we found that emotion regulation difficulties actually increased after the cognitive module for adolescents who followed the behavioral–cognitive sequence. Thus, the exact same treatment approach may have opposite effects, depending on the order in which it is presented. It is possible that the (ineffective) behavioral module desensitized adolescents to the (effective) cognitive module. Adolescents may have experienced that behavioral strategies did not seem to work in the first module, which consecutively made them less motivated and more pessimistic about the cognitive strategies in the second module. If so, this suggests that we need to be cautious with iatrogenic effects, that may compromise the effects of further intervention. An alternative explanation may be that behavioral strategies were already used more than cognitive strategies, due to pre-existing preferences of adolescents with MID–BIF (te Brinke et al., 2021), or previous treatment focus at the residential treatment center, and that as a result, there was less room for improvement in behavioral strategies. Lastly, it is also possible that a behavioral approach is not effective by itself, but only functions as booster, and thereby fosters the effect of a cognitive approach.
It should be noted, however, that although the current study suggests that the behavioral module was, by itself, less effective than the cognitive module, this does not mean that clinicians should be inclined to omit behavioral techniques in all CBT protocols for adolescents with MID–BIF. The experimental modular training that was examined in the current study focused specifically on enhancing emotion regulation skills among adolescents with externalizing problems. Thus, it cannot be ruled out that a behavioral approach is more effective than a cognitive approach when other underlying mechanisms are targeted (e.g., contingency management for desirable behavior), or when the psychological symptoms of the target group differ (e.g., behavioral activation for adolescent with MID–BIF and depression). Besides, the current study only included adolescent-focused training, whereas a behavioral approach also frequently includes operant conditioning with mediating parents or teachers (i.e., response change of caregivers and teachers). Standing Strong Together, an evidence-based intervention for adolescents with behavioral problems and MID–BIF, includes for example a parent-component, in which several sessions focus on the use of praise and tokens (Schuiringa et al., 2017). Little is known about the differential effects of these operant behavioral techniques. It could, therefore, be an interesting direction of future micro-trial research to examine the additional effect of behavioral caregiver-oriented approaches, on top of cognitive child-oriented approaches. Moreover, future micro-trials could focus on other mediating mechanisms and/or therapist skills.
Strengths and Limitations
Strengths of the current study include the focus on an understudied, difficult-to-reach population of adolescents with externalizing problems and MID–BIF, the thorough assessment of treatment adherence and differentiation, and the use of multiple assessment methods, which enabled us to examine not only between–group differences, but also within-person change. Between–group differences in externalizing problems were examined from a multi-informant perspective, which might be of additional importance for research among adolescents with MID–BIF, because these adolescents may have difficulties in understanding verbal information, and limited insight into their own behavior (Bramston & Fogarty, 2000; Pavlović et al., 2013). The addition of within-person analyses of weekly reported emotion regulation difficulties and aggression also strengthens this study, because weekly assessments do not rely on the recall of experiences over longer periods of time. It should, however, be noted that the response rate to the weekly assessments was relatively low (i.e., on average nine out of the 17 weeks), and that for the aggression subscale, the internal consistency was suboptimal. In future research, a validated assessment of weekly problem behaviors may be included. An example of a validated measure is the idiographic Top Problems measure, in which adolescents and/or caregivers identify and repeatedly rate the severity of their top three problems (Weisz et al., 2011). Such an idiographic approach might potentially increase the response rate of weekly assessments among adolescents with MID–BIF, because it is more personalized.
A limitation of the current study was its small sample size, which might have diminished the power to detect (between–person) differences. This power issue could, potentially, be an explanation for the fact that even though the effect sizes for the differences between the two sequences were large and all in the same direction (i.e., between 0.40 and 0.79), some effects were only significant for adolescent-reported emotion regulation difficulties, and not for externalizing problems. In addition, treatment dropout was high, with more than half of the participants discontinuing treatment prematurely due to resistance and/or practical issues. Treatment dropout is variable among adolescents with MID–BIF (Kok et al., 2016) and externalizing problems seem to be an important predictor of treatment dropout (de Haan et al., 2013). Moreover, well-known treatment barriers in residential settings such as repression, high staff turn-over and organizational difficulties might explain the relatively high dropout in the current study (de Valk et al., 2016; James et al., 2017). It should be noted that analyses were performed according to the intention to treat principle, and that the degree of missing data was relatively low (i.e., with 81% of adolescents filling out questionnaires at T3). Nevertheless, the dropout is problematic methodologically, as it may have limited the ability to detect differences between the groups. The high treatment drop-out in the present study suggests that a better understanding of reasons for drop-out in residential settings (and ways to prevent it) may be an important priority for future research.
Another limitation of the study was its open design, as adolescents and other informants included in the assessments (i.e., clinical staff) might have been aware of the fact that they received a specific module sequence. We did, however, not explicitly state that we were interested in approach or sequence effects. In addition, the emotion regulation measures were not specifically validated for adolescents with MID–BIF, although research has indicated the internal consistencies of the emotion regulation questionnaires were comparable between a MID–BIF and average intelligence group (te Brinke et al., 2021). Lastly, participants were recruited through a residential treatment center for adolescents with behavior problems and intellectual disabilities, and the experimental training was added on to their existing treatment. Therefore, we cannot rule out the possibility that participants were already familiar with the therapeutic techniques that were included in the experimental intervention. We do, however, not have reasons to believe that this familiarity might have had a differential impact on one of the two experimental modules, since existing treatments mainly focus on a combination of cognitive and behavioral approaches.
Conclusion and Implications
The most important conclusion of this study is that for adolescents with externalizing behavior problems and MID–BIF, a cognitive approach of an emotion regulation training seems to be more effective than a behavioral approach. Moreover, it may be more effective to include behavioral exercises after cognitive training than vice versa. Although this is the first, relatively small sample experimental study that examined differences between cognitive and behavioral treatment approaches for adolescents with externalizing problems and MID–BIF, these findings may have implications for existing CBT protocols and clinical guidelines. First of all, these findings seem to suggest that when clinicians or treatment developers aim to enhance the effectiveness of existing CBT protocols for adolescents with externalizing problems and MID–BIF, they preferably start by adjusting the relative focus of the intervention, to make sure that the intervention includes a cognitive approach. Second, guidelines for CBT among adolescents with MID–BIF, in which it is currently stated that within multi-component treatment packages, there should be a stronger focus on behavioral, rather than cognitive approaches (van den Bogaard et al., 2020), may be adapted, in order to highlight the importance of cognitive approaches for this particular target group. Eventually, these adjustments may lead towards more effective ways to treat externalizing behavior problems among adolescents with MID–BIF.
Acknowledgements
The authors would like to express their gratitude to the members of the ExtrAct consortium for their help in setting up this study. In particular, the authors thank Gonnie Albrecht and Prof. Dr. Walter Matthys for their help in designing the experimental training protocols, and Jessica Muda for her help during the data collection.
Declarations
Conflict of Interest
Lysanne W. te Brinke, Hilde D. Schuiringa, Ankie T. A. Menting, Maja Deković, Johanna J. Westera and Bram O. de Castro declare that they have no conflict of interest.
Ethical Approval
This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of the University Medical Centre Utrecht.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Animal Rights
No animal studies were carried out by the authors for this article.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Met BSL Psychologie Totaal blijf je als professional steeds op de hoogte van de nieuwste ontwikkelingen binnen jouw vak. Met het online abonnement heb je toegang tot een groot aantal boeken, protocollen, vaktijdschriften en e-learnings op het gebied van psychologie en psychiatrie. Zo kun je op je gemak en wanneer het jou het beste uitkomt verdiepen in jouw vakgebied.
In the Netherlands, people with mild intellectual disabilities (MID; IQ 55–69) and borderline intellectual functioning (BIF; IQ between 70 and 84) are generally treated by the same healthcare centers. Consistent with other Dutch studies (e.g., Seelen-de Lang et al., 2019), these two groups are therefore addressed together in the current study.
Treatment Approach and Sequence Effects in Cognitive Behavioral Therapy Targeting Emotion Regulation Among Adolescents with Externalizing Problems and Intellectual Disabilities
Auteurs
Lysanne W. te Brinke Hilde D. Schuiringa Ankie T. A. Menting Maja Deković Johanna J. Westera Bram O. de Castro
