Introduction
The most intensive form of interventions for youth is residential care, which is often seen as a last resort solution for youths who have not responded well to previous treatment programs. Residential care is an out-of-home placement that typically provides 24-h care and offers mental health services with the goal of preparing youths to re-enter society (Whittaker et al.
2015). Such an invasive intervention is restricted to serve only those most in need, which includes youths with severe, complex behavioral and emotional problems. Their problem behavior is often combined with psychiatric disorders and/or intellectual disabilities (ID; Frensch and Cameron
2002), and up to 90% has been exposed to traumatic experiences such as neglect and/or abuse (Briggs et al.
2012). Thus, youths in residential care often show many risk factors and few protective factors (e.g., supportive caregivers, structured home settings) for behavioral and emotional problems, which contributes to the development and maintenance of these problems (Pollard and Hawkins
1999; Steinberg and Avenevoli
2000).
Residential care has a twofold purpose: first, to provide a safe and structured living environment for the youths in their group homes within the institution, and second, to offer intensive treatment to target problem behavior. At the group homes, group home workers are substitute caregivers who model appropriate behavior, provide support, and encourage youths to use the strategies they have learned in therapy. Treatment usually consists of evidence-based interventions to reduce problem behavior and teach youths adaptive alternative behaviors (McCurdy and McIntyre
2004; Kok et al.
1991). However, despite all efforts, residential treatment remains only moderately effective and approximately 25% of the youths leave care prematurely (Harder et al.
2006). A meta-analysis on the effectiveness of residential treatment showed an average effect size of
d = .36 in the reduction of behavior problems for residential care with evidence-based treatment compared to standard residential care (group home care without specific treatment; de Swart et al.
2012). That meta-analysis highlights the importance of providing youths with treatment during their stay in the institution, but despite these promising outcomes, the average effect size of evidence-based treatment in residential care remains modest (Cohen
1988).
There are in particular two concomitant factors that may impede treatment effectiveness in residential care. The first one is that residential treatment focuses mainly on externalizing problems and tends to overlook internalizing symptoms, while most youths also exhibit co-occurring internalizing symptoms (Granic
2014). Comorbidity rates are in particular high for anxiety – approximately half of all youths in residential care shows clinically elevated levels of anxiety (Connor et al.
2004). These anxious feelings direct youths’ focus towards potential threatening stimuli, which may result in direct, impulsive aggression as a defense to these – real or perceived – threats (Vitaro et al.
2006). Even when youths do not show direct aggression, they tend to amplify their anxious feelings and maintain their state of arousal, rather than applying successful emotion-regulation strategies. This sustained attention to potential threats exhausts youths, and may eventually lead to externalizing behavior through an indirect route – the loss of inhibitory control (Granic
2014). Notably, for both the direct and the indirect route to aggression, anxiety is hypothesized as the eliciting mechanism that leads to youths’ inability to regulate their emotions. Thus, by ignoring anxiety in residential treatment, we may treat the symptoms while avoiding the causes of the behavior problems.
Another factor that may account for the modest effect sizes is that residential treatment usually consists of interventions based on cognitive-behavioral therapy (CBT) principles (de Lange et al.
2013). Although CBT is among the most effective forms of treatment in residential care (Garrido and Morales
2007; de Swart et al.
2012), it has some overarching limitations. These are limitations regarding the delivery model of CBT, not the principles themselves (Kazdin and Blase
2011; Kazdin and Rabbitt
2013). Not only should residential treatment focus on both internalizing and externalizing problems to improve its effectiveness, it should also be delivered in a way that is targeted towards youths’ needs.
Youths in residential care are often characterized by a lack of motivation to change their behavior (van Binsbergen
2003). Poor motivation is the key predictor for both low treatment effectiveness (Harder et al.
2012) and treatment dropout (Harder et al.
2011). In order to ensure that these youths do not leave care prematurely or drop out of intervention programs, these programs need to be engaging for youths. A new approach and potential solution for the delivery of youth interventions is the use of videogames. Whereas CBT depends largely upon imparting psychoeducational information, a didactic style of learning that contains few elements that are intrinsically motivating, videogames are able to deliver evidence-based techniques in an appealing context and make use of youths’ intrinsic motivation to engage them into treatment.
Also, psychoeducational CBT offers knowledge with limited opportunities to practice. Due to this gap between knowledge and behavior, the generalizability of CBT is limited. Youths usually know about appropriate, prosocial behavior, but in their everyday lives they often act impulsively and based on their emotions. Although CBT often incorporates exercises such as role-playing (Kendall et al.
2003), this rarely manages to provoke genuine emotions. Videogames, however, provide youths with the opportunity to learn by doing instead of memorizing (Vygotsky
1978), and are better able to elicit authentically emotional experiences. Youths are provided with an in-game environment where acquired techniques and strategies can be practiced until they are automatized and ideally can be generalized outside the game (Granic et al.
2014). The repetitive nature of gameplay fosters long-term learning (e.g., Rosas et al.
2003). In particular games that implement biofeedback may promote self-regulation skills and foster generalization of learned behaviors to youths’ daily lives (Yucha and Mongomery
2008). A videogame named
RAGE-Control has successfully been integrated into a traditional CBT-based intervention to improve emotion-regulation among youths in residential care.
RAGE-Control was effectively used to practice and strengthen the techniques learned during therapy sessions with a therapist (Ducharme et al.
2012; Kahn et al.
2013).
Another factor that impedes the effectiveness of CBT is that 20–25% of all youths in residential care is diagnosed with ID (van Nieuwenhuijzen
2010). These youths have limited social, emotional, and cognitive capacities (Magiati et al.
2014), while learning about cognitive biases and the links between feelings, thoughts, and behavior requires high-level processing. Playing a videogame, on the other hand, usually requires less cognitive load. Youths with ID might benefit more from experience-based interventions such as videogames compared to verbally based interventions such as CBT (Association for the Treatment of Sexual Abusers
2015). Treatment should be tailored to the specific needs of these youths and requires more simplistic language, smaller learning steps, and more emphasis on generalization to their real lives than conventional programs offer (Didden
2006). At the time, there is little evidence that conventional CBT alone has any beneficial effects on youths with ID (Sturney and Hamelin
2014; Taylor
2002). Although traditional CBT may require some adaptations for youths with ID, its principles have successfully been adjusted for their treatment. CBT components were made more concrete by implementing exposure and using relaxation techniques (i.e., deep breathing, muscle relaxation) to promote self-regulation among youths with ID (Shenk and Brown
2007).
The present study tested the initial effectiveness of
Dojo, a biofeedback videogame intervention (developed by GameDesk, Los Angeles, CA), which targets the emotion-regulation problems that are hypothesized to underlie both anxiety and externalizing problems. The game consists of three in-game rooms (fear, frustration, and anger), each with one or two tutorials and an emotion-evoking mini game. The tutorials teach CBT-based relaxation techniques as deep-breathing techniques, progressive muscle relaxation, positive thinking, and guided imagery (Albano and Kendall
2002; Glancy and Saini
2005; Rapee et al.
2000; Sukhodolsky and Scahill
2012; Weisz and Kazdin
2010). The mini games are designed to trigger the emotion in question and challenge the youths to practice the newly acquired relaxation strategies in the in-game environment. This way, youths are playfully trained how to cope with their emotions. While playing the games, the players heart rate is monitored through biofeedback hardware and displayed on the screen, thus providing the player with real-time feedback on their stress levels. Controlling physiological reactions is required for success in the game, which encourages players to effectively regulate their emotions by using players’ desire to perform well in the game. For a more detailed description of the game, see Schuurmans et al. (
2015). A recent pilot study demonstrated the feasibility and potential of
Dojo as an intervention for a high-risk adolescent target population (Schuurmans et al.
2015). User evaluations and self-reported compliance for the tutorials were high, and initial outcome results on reductions in anxiety and externalizing problems were promising.
The present study was designed as an initial randomized controlled trial (RCT) to test the effectiveness of Dojo as an intervention for youths with clinical anxiety and externalizing problems in residential care. We hypothesized that participants who played Dojo as an addition to their treatment as usual (TAU) would show reduced levels of anxiety and externalizing problems compared to participants who received TAU alone. First, we focused on the main outcomes of the trial, the immediate posttreatment effects on symptoms of anxiety and externalizing problems. Then, we examined intervention outcomes at 4-months follow-up.
Discussion
Key Findings
The current study utilized an initial RCT to test the effectiveness of Dojo as an intervention for high-risk youths with clinical anxiety and externalizing problems in residential institutions. We expected that eight sessions of Dojo gameplay would lead to reduced levels of anxiety and externalizing problems; these hypotheses were partly supported. Youths who played Dojo, compared with youths in the control condition, showed reductions in self-reported anxiety and externalizing problems at posttreatment, and mentor-reported anxiety at both posttreatment and follow-up. Contrary to our expectations, results showed no differences between conditions in self-reported anxiety and externalizing problems at follow-up, and mentor-reported externalizing problems at both posttreatment and follow-up. These findings indicate that the intervention resulted in an immediate reduction of anxiety, and from youths’ own perspective, also externalizing problems.
We have to be cautious with our conclusions regarding these outcomes, given the small sample size and subsequent low power in this study may have resulted in less robust results. The mentor-reported follow-up results have to be interpreted cautiously due to the high attrition rates. This was due to participants leaving the institutions. Participants were either discharged from the institutions to return home, because they showed improvements in their behavior (
n = 3), they were replaced to other, secured institutions, because their behavior problems deteriorated (
n = 6), or they refused further treatment and left the institutions without being discharged (
n = 1). Whereas all participants themselves were willing to complete the last interview for this study, even when they left our institutions, we were unable to obtain mentor-reports for these participants. This non-random missing mentor-report data at follow-up may have resulted in an unrepresentative sample (Graham
2009). Nevertheless, we decided to report these results, but we have to refrain from drawing firm conclusions from this measurement.
Although youths themselves report a decrease in externalizing behavior at posttreatment, their mentors do not report any effect on externalizing problems. These variations in outcomes might be caused by a difference in perception between youths and their mentors. Other studies that were conducted in residential institutions have reported comparable results, with substantial disagreement between youths’ self-reports and mentor-reports, in particular for outcomes on externalizing problems (Bastiaansen et al.
2004; Grills and Ollendick
2003; Nijhof et al.
2011). It may be that group home workers are more critical in the assessment of youths’ behavioral progress compared with youths themselves, precluding them from noticing changes (Knorth et al.
2008).
Our results suggest that
Dojo has a larger effect on anxiety than on externalizing problems. When we compare
Dojo with conventional interventions that target anxiety and externalizing problems, there is a larger overlap with traditional components of anxiety treatment (i.e., skills training such as affect recognition, relaxation, and exposure; e.g., Kendall et al.
2003) than with aggression treatment (i.e. teaching prosocial behavior, anger control, and moral reasoning; Goldstein and Glick
1987). Although
Dojo has one room that specifically targets anger, the main idea of the game is to teach youths to recognize their emotional and physical arousal, and to control this by practicing relaxation techniques.
Surprisingly, youths’ immediate decrease in self-reported anxiety and externalizing problems was not maintained at follow-up, only mentor-report anxiety showed an effect at both posttreatment and follow-up. This suggests that although
Dojo could be an effective way to decrease anxiety and externalizing problems, from the youths’ perspective, the intervention may not help them to cope with future anxiety-provoking situations. In order to maintain immediate posttreatment effects, it may be necessary to provide youths with a ‘booster session,’ as done by
The Growth Factory, a computerized mindset intervention developed for youths in residential care (Helmond et al.
2014).
Finally, we would like to discuss our results in the light of the findings of Scholten et al. (
2016), who also tested
Dojo, but as a method of prevention in a non-clinical sample of adolescents at risk for anxiety. They compared
Dojo to the commercial videogame
Rayman, and found equal reductions in anxiety for both conditions. This could mean that both videogames were equally effective in reducing anxiety, or that neither was effective. This question remains unanswered due to the lack of an inactive control group, but even when we assume that
Dojo did not have an effect in their study, there are some explanations for the different results in our study. It could be due to the substantial differences between the two target populations. It may be that
Dojo does work as an intervention for a clinical population with and without ID, but not as prevention for non-clinical adolescents. Another possibility is the difference in the ways
Dojo was delivered. Scholten et al. (
2016) allowed youths to play freely with minimal supervision, which made it possible for youths to skip the relaxation tutorials – which are hypothesized as
Dojo’s working mechanisms – and play the mini games only.
Strengths and Limitations
To our knowledge the current study is the first to examine a videogame intervention in a residential treatment setting with youths with and without ID. Research in this clinical, high-risk context is critical to establish intervention effectiveness and requires minimal translation to implement its results in these settings. Attrition in high-risk samples is usually high, but we lost only two participants in each condition from baseline to posttreatment. Both participants in the Dojo condition decided to quit the sessions due to scheduling problems: the sessions had to be scheduled after school hours and interfered with their part time job and/or leisure activity appointments. One participant asked for the opportunity to quit the study but start again with the Dojo sessions after the summer, since he would have more spare time by then. This indicates that although he quit the study, it was not because he was not motivated to play Dojo. Youths’ positive game evaluations demonstrated that they liked the game and they reported high compliance during the relaxation tutorials. Moreover, not only was treatment fidelity high – all participants completed the eight scheduled sessions – all youths in the control condition still wanted to play Dojo after the follow-up measurement. They did not receive an incentive for this, which indicated that they were intrinsically motivated to play. This suggests that we met our goal as for engaging this hard-to-motivate population into treatment.
The biggest limitation of the present study is its small sample size. Its results should be interpreted with caution, since these were not robust. Outcomes changed depending on whether or not the missing data were imputed. This study was not powered to definitively test the effectiveness of Dojo. In the future, rigorously designed and adequately powered RCTs could establish the effectiveness of Dojo on anxiety and externalizing problems and examine potential mediating mechanisms.
Participants in the
Dojo condition received the intervention as an addition to TAU, while the control condition only received TAU. This means that participants in the
Dojo condition received extra individual attention compared to the controls. Although active control groups are more rigorous, these are only superior when participants in both conditions have the same type of attention and the same expectations of improvement (Boot et al.
2013). Thus, optimal control would be a videogame that is comparable, but does not include the working mechanisms. This was impossible to achieve for us, since the study was conducted within institutions which had restrictions for casual videogame play. Clinicians did not agree on implementing a condition in which youths were allowed to play a videogame that was not expected to lead to mental health benefits. Moreover, the primary purpose of this study was to test the effectiveness of
Dojo as a beneficial addition to regular treatment, not to determine its superiority to another form of treatment, which makes TAU a valid control condition (Freedland et al.
2011).
Our study design included not only a posttreatment measurement to assess immediate intervention effects, but also a 4-months follow-up measurement to evaluate whether effects were maintained over time. While attrition rates in these high-risk populations are in particular high for follow-up measurements, all participants who were included at posttreatment, also completed the self-report measurement at follow-up. However, we lost a substantial amount of the mentor-reports at follow-up, for reasons explained above.
The gameplay sessions were supervised by the first author and two research assistants, while an ideal clinical study design does not include the researcher’s involvement in the intervention sessions. Given limited funds and personnel time, it was not feasible to hire an additional research assistant blind to the study goals. Gameplay supervision was done following a standardized protocol, to prevent possible supervisor effects. Moreover, since Dojo teaches the emotion-regulation techniques within the game, the only task for supervisors was to answer any questions and to ensure that the youths followed the instructions (e.g., completing the tutorials before starting with the game).
Although this study showed the potential of
Dojo as a form of treatment, we do not propose
Dojo as a stand-alone intervention that is able to replace interventions already in use.
Dojo has advantages compared to conventional treatment, but also possible disadvantages. For example, an important element of traditional therapy that may have positive effects is therapeutic alliance (Shirk and Karver
2003), something that is missing for
Dojo.