Anxiety disorders are the most prevalent mental health problems in childhood. Engaging, adequate, and appropriate prevention programs are needed. Applied games form a potential alternative delivery model and recent evidence suggests that they could be effective. The present randomized controlled non-inferiority trial investigated the beneficial effects of the applied game MindLight compared to cognitive behavioral therapy (CBT) on mental health outcomes associated with anxiety symptoms: internalizing problems, externalizing problems, and self-efficacy. In addition, we examined who benefitted most from both programs and analyzed baseline levels of anxiety, maternal mental health problems, and self-efficacy as predictors of changes in anxiety symptoms. After being screened for elevated anxiety, 174 selected children (8–12-year-old) were randomized to play MindLight or to receive a prevention program based on CBT. Study variables were assessed before and after the intervention, and at 3- and 6-months follow-up. Intention-to-treat analyses showed a significant reduction in mother-reported internalizing and externalizing problems and an increase in self-efficacy. Importantly, the magnitude of change did not differ between intervention groups. Non-inferiority analyses showed that MindLight was as effective as CBT in affecting internalizing problems and self-efficacy. However, CBT was more effective in decreasing externalizing symptoms than MindLight. Furthermore, baseline anxiety levels, maternal mental health problems, and self-efficacy did not influence the change of anxiety symptoms over time. Applied games, specifically theory-based games such as MindLight, hold potential as effective interventions for not only targeting anxiety symptoms, but also more general mental health outcomes.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Approximately one in five children has an anxiety disorder (Beesdo et al. 2009) and even more children suffer from subclinical levels of anxiety, with prevalence rates up to 49% (Muris et al. 2000a). Compared to children with low levels of anxiety, these children perform worse at school (Owens et al. 2012), and have a lower general quality of life (Ramsawh and Chavira 2016). In addition, children with elevated levels of anxiety have more depressive feelings (Lavigne et al. 2015), more difficulties in relationships with their peers (Hoglund and Chisholm 2014), express more conduct problems (Priddis et al. 2014; Kidwell et al. 2016), and show lower levels of self-efficacy (Mathews et al. 2016; Niditch and Varela 2012; O’Neal and Cotten 2016) than children with low levels of anxiety. Left untreated, anxiety symptoms show a disabling and chronic course (Asselmann and Beesdo-Baum 2015). Therefore, effective, accessible and engaging prevention programs are needed that are implemented before full-blown anxiety disorders develop (World Health Organization 2012).
Decades of research has led to the development of several anxiety prevention programs (e.g., Van Starrenbrug’s adaptation of Kendall’s Coping Cat, Barrett’s FRIENDS for Life and Rapee’s Cool Little Kids programs). However, conventional programs face several obstacles that hamper their implementation. Specifically, stigma (Salloum et al. 2016; Mukolo and Heflinger 2011) and program costs (Salloum et al. 2016) impede parents and children from seeking help. Furthermore, conventional programs are moderately effective as shown in various meta-analyses (Stockings et al. 2016; Mychailyszyn et al. 2012; Fisak et al. 2011; Teubert and Pinquart 2011) and drop-out rates are high (i.e., 28 up to 75%; de Haan et al. 2013), possibly because programs are not engaging, adequate, nor appropriate (World Health Organization 2012). These obstacles call for a reconsideration of current group-based and clinical expert-led delivery models of prevention programs (Kazdin 2015). To overcome those barriers, applied games have recently been put forward as an alternative delivery model of therapeutic techniques used in prevention programs (Kazdin 2015). In contrast to current services, games might be cheaper than therapists, easily accessible, engaging and not stigmatizing (Granic et al. 2014).
In the past, we have tested the effectiveness of the applied game MindLight in two randomized controlled indicated prevention trials (RCTs; Schoneveld et al. 2016, 2018). MindLight is an applied game designed for children with anxiety symptoms. The game uses several evidence-based techniques, informed by cognitive behavioral therapy (CBT): exposure (Kendall et al. 2005), attention bias modification (Bar-Haim et al. 2011) and neurofeedback (Price and Budzynski 2009). These techniques are embedded in a horror-themed survival game that trains children to cope with anxious feelings. First, anxiety symptoms reported by both children and parents (primary outcomes) were examined in one RCT (Schoneveld et al. 2016). It was found that MindLight was as effective as a commercial video game (i.e., Max and the Magic Marker, an award-winning exploratory puzzle game) in 8–12-year-olds with elevated levels of anxiety symptoms. Anxiety symptoms decreased after game play and up to 3 months later. The second RCT was designed to more rigorously examine the effectiveness of MindLight by comparing the applied game with the first-line treatment of choice for anxiety symptoms: CBT (James et al. 2015). Children who played MindLight showed the same decrease in anxiety symptoms as children who received CBT (Schoneveld et al. 2018). Importantly, the magnitude of improvement was equal across MindLight and both comparison groups (i.e., commercial game and CBT). In addition, children’s age and gender did not moderate effectiveness in both RCTs.
In light of these initial positive effects of MindLight on anxiety symptoms, the next steps are to investigate whether MindLight also has beneficial effects on other outcomes associated with anxiety symptoms, and for whom MindLight might be most effective. Children who experience elevated levels of anxiety often also suffer from depressive feelings (Lavigne et al. 2015), have difficulties in relationships with their peers (Hoglund and Chisholm 2014), express more externalizing symptoms such as conduct problems (Priddis et al. 2014; Kidwell et al. 2016), and are generally characterized by low levels of self-efficacy (Mathews et al. 2016; Niditch and Varela 2012; O’Neal and Cotten 2016). Given the debilitating effect of these problems on the lives and further development of these children, and the fact that they co-occur as well as contribute to further increases in anxiety, it seems important to investigate whether anxiety prevention programs also have a beneficial effect on those domains.
Previous studies assessing changes in internalizing problems after anxiety prevention programs found that pre-school aged children with an anxiety disorder decreased in internalizing behavior problems from pre- to post-test after receiving an internet-based, therapist assisted, parent-focused, CBT program (Donovan and March 2014). In addition, Morgan et al. (2016) found that highly inhibited children between the age of 3 and 6 years improved significantly in emotional symptoms during an online version of the parenting group program Cool Little Kids. Last, a meta-analytic review showed that interventions targeting anxiety in youth showed significant effects on depressive symptoms for treatment and universal prevention programs, but not in targeted prevention programs (Garber et al. 2016). Thus, it seems that anxiety prevention programs are able to improve other internalizing problems as well.
Whether anxiety prevention programs also have beneficial effects on externalizing problems is unclear, but there are reasons to believe this may be so. Research on the comorbidity between anxiety and aggression could be informative. Two recent reviews about the often-found comorbidity between anxiety and aggression focus on attention control (Fraire and Ollendick 2013; Granic 2014). Anxious children pay more attention to potential threats in their environment (i.e., attentional bias; Bar-Haim et al. 2011) and have less processing capacity left to focus and sustain attention on other stimuli (i.e., attentional control; Fraire and Ollendick 2013; Reinholdt-Dunne et al. 2013; Eysenck et al. 2007). This vigilant focus on the potential negative aspect of the environment might consume most of the available resources. As a result, anxious children may have difficulties inhibiting their impulses and act out and behave aggressively (Granic 2014). Thus, when anxiety symptoms are decreasing as a result of the prevention program this might free up cognitive resources to better regulate impulses (Hadwin and Richards 2016) and consequently externalizing problems might decrease.
Last, previous research has not focused on changes in self-efficacy following anxiety prevention programs, but it seems important to consider as well (Muris 2001; Muris et al. 2009). Self-efficacy has been theorized to play a key role in the etiology and maintenance of anxiety and refers to the belief in one’s ability to produce a desired behavior (Bandura 1997; Maric et al. 2011). More specifically, self-efficacy in the social domain (i.e., perceived capability to deal with social situations), academic domain (i.e., perceived capability to master academic affairs), and emotional domain (i.e., perceived capability to cope with negative emotions) seem relevant for the study of anxiety in youth (Muris 2001). In the literature focusing on intervention and treatment, one study assessed self-efficacy in a CBT-program for school-refusing children and found improvements in children’s self-efficacy for school situations (i.e., perceived ability to cope with anxiety-provoking situations at school; King et al. 1998). Another study found improvements in self-efficacy for anxiety management in youth with anxiety disorders following CBT (Suveg et al. 2009). These findings suggest that it may be important to examine changes in self-efficacy following anxiety prevention programs.
Next to investigating whether MindLight has beneficial effects on other outcomes associated with anxiety symptoms, another important question pertains to the idea that individuals respond differently to prevention programs. There may be important predictors of efficacy to consider. Past research in CBT-based anxiety prevention programs for children has identified several potential baseline predictors, such as baseline anxiety, maternal mental health problems, and self-efficacy. Research investigating the effect of anxiety symptoms at baseline on the response to a prevention/intervention program is inconclusive. One study found that higher levels of baseline anxiety were related to greater decreases in anxiety after an indicated CBT-based program (van Starrenburg et al. 2017). However, another study found that children with clinical anxiety show a more limited response to pain-focused CBT than children with subclinical levels of anxiety (Cunningham et al. 2016), suggesting that anxiety symptoms at baseline adversely impact the intervention response.
Further, parents of children with mental health problems often have mental health issues themselves (Powdthavee and Vignoles 2008; Goodman et al. 2011). Research has shown that parental problems with mental health can be genetically transmitted (Lubke et al. 2016). Alternatively, parental problems can also impact children’s mental well-being through distortions in parenting, for example harsh discipline (Gershoff 2002) or a controlling parenting style (Chorpita et al. 1998). For anxiety specifically, more controlling parents diminish children’s sense of personal control, thereby contributing to increases in anxiety (Chorpita et al. 1998). Last, parental modeling of anxious behaviors and cognitions may also contribute to children’s anxiety (Fisak and Grills-Taquechel 2007).
Studies investigating the effect of maternal mental health problems on intervention effectiveness show inconsistent results. One study on the effect of maternal depression on posttraumatic stress treatment in children found that maternal depression was associated with increasing posttraumatic stress symptoms in children (Weems and Scheeringa 2013), especially for children with higher baseline levels of these symptoms (Nixon et al. 2012). However, another study showed that maternal psychopathology did not predict the effect of a depression and anxiety prevention program for adolescents with parents with mental health problems (Rasing et al. 2020).
There are no studies that have directly examined the role of self-efficacy on anxiety prevention effects. Therefore, it is unclear whether children with high or low levels of self-efficacy respond differently to anxiety prevention programs. A recent meta-analysis of 155 experimental trials found, however, that self-efficacy has a causal effect on health-related intentions and behavior (Sheeran et al. 2016), indicating that when people believe that they can execute the relevant action, they are more likely to change their health intention and behavior.
The current study is the second study out of a non-inferiority RCT comparing MindLight and CBT (Schoneveld et al. 2018) and describes its secondary outcome results. The first aim of the present study was to report the effect of MindLight on mental health outcomes of children with elevated levels of anxiety symptoms. Specifically, changes in internalizing problems, externalizing problems, and self-efficacy were investigated. Based on previous research, we hypothesized that MindLight and CBT would be equally effective in decreasing internalizing and externalizing problems, and in increasing self-efficacy. The second aim was to assess possible predictors of MindLight and CBT anxiety outcome: baseline anxiety symptoms, maternal mental health problems, and self-efficacy. We hypothesized that baseline anxiety would, and maternal mental health problems would not predict changes in anxiety symptoms. We tentatively hypothesized that higher baseline levels of self-efficacy would predict a larger decrease in anxiety symptoms.
Method
Participants
A total of 174 children were randomly assigned to MindLight (n = 86) or CBT (n = 88; see Fig. 1 for a flowchart of participants throughout the trial). The target total sample size was 135 children. Details about the sample size calculation can be found in Schoneveld et al. (2018). At pre-test, children ranged from 7 to 12 years old (M = 9.97, SD = 1.16) and 59.2% were girls. Mothers were between 28 and 49 years old (M = 41.13, SD = 3.67) at pre-test, the majority being Dutch (87.9%). No differences were found between the MindLight and the CBT group on pre-test anxiety symptoms, age, sex, and weekly game time (i.e., number of hours spent playing video games per week; see Schoneveld et al. 2018).
Fig. 1
Flowchart of participants through trial
×
Procedure
The study was designed as a randomized, multicenter non-inferiority study with two parallel intervention arms: MindLight and CBT. Between January and September 2015, all children with active parental consent in grades 3–6 from eight primary schools in the southeast part of the Netherlands (N = 791) were first screened on anxiety symptoms with the child version of the Spence Children’s Anxiety Scale (SCAS; Spence 1998). Screening took place in the classroom during school hours in the presence of one or two members of the research team. Parent reports were not included in the screening for practical reasons, but also because self-report questionnaires for anxiety are considered more reliable than parent reports (Hourigan et al. 2011; Lagattuta et al. 2012; Lahikainen et al. 2006). Children were eligible if either at least two SCAS subscales (excluding the obsessive compulsive disorder subscale) or the total SCAS score, was 1 SD or more above the mean found in a large normative sample (Muris et al. 2000a). Parents of the 221 (27.9%) eligible children were contacted by phone to assess exclusion criteria and invite them and their child(ren) to participate. Children were excluded if they already received anxiety treatment or if they were diagnosed with either obsessive-compulsive disorder, posttraumatic stress disorder or autism spectrum disorder. These children were excluded because obsessive-compulsive disorder and posttraumatic stress disorder are no longer considered an anxiety disorder in the DSM-V, and children with these disorders (including autism spectrum disorder) may benefit from a more specific or specialized treatment than the prevention programs in the current study (e.g., Barrett et al. 2008; Nauta and Scholing 2007). Parents of 174 (78.7%) children gave initial verbal consent; written informed consent was obtained from parents at pre-test, a week prior to the intervention. Children and their mothers then filled out the questionnaires at school or online respectively (i.e., pre-test). Next, children were randomized within school and younger/older age groups, and stratified by sex and grade to participate in the MindLight or CBT group (further details of the randomization procedure are available elsewhere; see Schoneveld et al. 2018). Children and mothers were assessed again at completion of the intervention (post-test), and 3- and 6-months post-intervention completion (i.e., 3- and 6-months follow-up).
MindLight
MindLight is a 3D third-person neurofeedback video game designed by a multidisciplinary team of researchers from the PlayNice Institute and game designers from GainPlay Studio. The game starts with a little boy named Arty who is left at the doorstep of his grandmother’s scary mansion. In his bedroom, he finds Teru, a magical glowing hat that faces him with the task of saving grandmother from the evil forces that have possessed her and the house. Teru teaches Arty (and the player) to change his state of mind and thereby overcome his fears. The player controls the movement of Arty using a Microsoft Xbox 360 controller and (s)he controls Teru’s light via the Neurosky one-channel dry-sensor EEG headset (i.e., MindWave). Several evidence-based, theoretically grounded strategies for decreasing anxiety were translated into game mechanics: neurofeedback, exposure, and attention bias modification. The game mechanic associated with neurofeedback involves the amount of light that shines from the player character’s (Teru’s) magical hat. The EEG headset that the player wears measures the real-time relaxation of the player, which in turn controls the amount of light in the game environment. When the player becomes more relaxed, the light becomes brighter. The exposure game mechanic involves fear events (i.e., fearful obstacles) that need to be approached in order to play through the game. By shining one’s light on the fear events they can be chased away or uncovered. Uncovered fear events will either turn into a friendly kitten or an animal/benign object which rewards the player with a coin that is needed to unlock the (attention bias modification) puzzles. In these puzzles, the player learns to focus on and attend to portraits of happy faces rather than threatening faces. To complete the puzzle, the player needs to shine one’s light on the happy faces, which eventually will turn the light back on in that particular room. For a more elaborate description of MindLight, see previous papers on the applied game (Schoneveld et al. 2016, 2018; Wijnhoven et al. 2015; Wols et al. 2018).
Children played MindLight in 1-h sessions at school, after regular school hours, once a week, for 6 weeks. Children played the game individually but they were seated in a room with about five to ten other children. Children were seated one table away from each other and used earplugs to hear the game sound and to diminish distraction. Master’s degree students gave instructions about MindLight and supervised the groups.
CBT
The CBT program used in the current study was an adaptation of Kendall’s Coping Cat (Flannery-Schroeder et al. 2005; van Starrenburg et al. 2017). It is one of the few CBT prevention programs for children with elevated anxiety symptoms that focuses on anxiety-specific symptoms, emphasizes exposure and is freely accessible (van Starrenburg et al. 2017). Specifically, a shortened eight-session (9 h) Dutch version of the indicated prevention group-based version of Van Starrenburg et al. (2017) was given. In this program, children are taught both cognitive (i.e., cognitive restructuring) and behavioral techniques (i.e., relaxation training and exposure). Children received two 1.5-h sessions and six 1-h sessions at school, after regular school hours, once a week, for 8 weeks. Groups consisted of four to seven children and were led by two CBT trainers (Schoneveld et al. 2018). Parents were informed about the progress of their child halfway through the program and after the last session via e-mail.
Measures
Anxiety symptoms
Children’s anxiety symptoms were measured with the child (45 items) and mother (38 items) versions of the SCAS (Spence 1998). To reduce negative response bias, the child version includes seven positive filler items. All items were rated on a four-point scale: 0 = never, 1 = sometimes, 2 = often, 3 = always. Good convergent validity (Brown-Jacobsen et al. 2011; Muris et al. 2000b) and reliability (Whiteside and Brown 2008) are demonstrated for both the child and the mother version. In our sample, Cronbach’s alphas were 0.90–0.93 for the child version and 0.80–0.84 for the mother version across all time points. Two outcome variables were computed: total anxiety child and total anxiety mother, which are the overall means (with the exception of filler items).
Self-efficacy
Children’s self-efficacy was measured with the self-report version of the Self-Efficacy Questionnaire for Children (SEQ-C; Muris 2001). The 24 items were rated on a five-point scale (0 = very bad, 1 = pretty bad, 2 = not good, not bad, 3 = pretty good, 4 = very good) and represented three domains of self-efficacy: (1) social self-efficacy: perceived capability for assertiveness and peer relationships, (2) academic self-efficacy: perceived capability to fulfill academic expectations, to master academic subjects, and to manage one’s own learning behavior and (3) emotional self-efficacy: perceived capability to cope with negative emotions. The SEQ-C shows satisfactory internal consistency, reliability and validity (Muris 2001). In our sample, Cronbach’s alphas were between 0.73–0.84 for academic self-efficacy, between 0.64–0.75 for social self-efficacy, and between 0.79–0.88 for emotional self-efficacy across all time points. Three outcome variables were computed: social, academic and emotional self-efficacy.
Internalizing and externalizing problems
Children’s internalizing and externalizing problems were measured with the mother version of the Strengths and Difficulties Questionnaire (SDQ; Goodman 1997; Stone et al. 2010). The SDQ includes 25 items divided in three subscales: (1) internalizing problems: emotional symptoms and peer relationship problems; (2) externalizing problems: conduct problems and hyperactivity/inattention symptoms; (3) prosocial behavior. We used this three subscales division, because it is more valid than the original five subscales division in a low-risk (i.e., without disorders) sample (Goodman et al. 2010). We only used the first two subscales in this study. All items were rated on a three-point scale: 0 = not true, 1 = somewhat true, 2 = certainly true. The two subscales showed good convergent and discriminant validity, and good internal reliability (Goodman et al. 2010). In our sample, Cronbach’s alphas were between 0.72–0.75 for internalizing problems and between 0.75–0.79 for externalizing problems across all time points. Two outcome variables were computed: internalizing problems and externalizing problems.
Maternal mental health problems: depression, anxiety, and stress
Maternal mental health problems were assessed with the 21 items version of the Depression Anxiety Stress Scales (DASS-21; Antony et al. 1998). The 21 items, all covering negative feelings, were rated on a four-point scale (0 = not at all, 1 = sometimes, 2 = often, 3 = usually) and represented three subscales: depression, anxiety, and stress. The subscales show good concurrent validity and reliability (Antony et al. 1998). In our sample, Cronbach’s alphas were 0.89 for depression, 0.81 for anxiety and 0.87 for stress at pre-test. Three outcome predictor variables were computed: maternal depression, maternal anxiety and maternal stress.
Data Analyses
A t-test and a χ2-test were performed in IBM SPSS Statistics 23 to assess whether randomization was successful for sex and age. To test non-inferiority, we used a two-sided confidence interval (CI) approach. The idea behind this approach is that if the upper bound of the CI for the difference in mean change in secondary outcomes is below the margin of non-inferiority, MindLight is non-inferior to CBT. Based on a previous indicated anxiety prevention trial (van Starrenburg et al. 2017), the margin of non-inferiority was set at 0.38 SEQ points for social, emotional and academic self-efficacy, at 1.11 SDQ points for internalizing problems, and at 0.90 SDQ points for externalizing symptoms. These differences correspond to 0.5 SD of the change in emotional self-efficacy (M = −0.28, SD = 0.76), internalizing symptoms (M = 1.50, SD = 2.22), and externalizing symptoms (M = 1.10, SD = 1.80) at post-test in children in the CBT condition of the van Starrenburg et al. (2017) trial.
To further examine the effectiveness of MindLight on the secondary outcomes (i.e., social, emotional, and academic self-efficacy, internalizing and externalizing problems), Latent Growth Curve Modelling (LGCM) was performed using Mplus 7.2 (Muthén and Muthén 1998‐2012). First, we estimated the initial model based on the four time points (i.e., pre-test, post-test, 3- and 6-months FU) without any predictors or control variables. Second, we tested whether condition predicted initial levels of outcomes (i.e., intercept) and/or rates of change in outcomes (i.e., slope). For our second aim, predictors (i.e., baseline anxiety symptoms, maternal mental health problems and self-efficacy) of the effectiveness of MindLight and CBT on anxiety symptoms were also assessed with LGCM. The effectiveness on anxiety symptoms was already reported in Schoneveld et al. (2018). For the current analyses, we started with the quadratic growth model of anxiety symptoms found in Schoneveld et al. and added the outcome predictors measured at pre-test (i.e., social, emotional, and academic self-efficacy) next to condition as predictors. In addition, we added maternal depression, anxiety and stress, and baseline anxiety levels as predictors. This was done separately for the model based on child-reported and mother-reported total anxiety symptoms.
To determine model fit, we used the Chi-square an p value, the Comparative Fit Index (CFI, critical value ≥ 0.95), Tucker Lewis Index (TLI, critical value ≥ 0.95) and the Root Mean Squared Estimate of Approximation (RMSEA, critical value ≤ 0.06) (Hu and Bentler 1999). The default maximum likelihood estimator was used. Attrition analyses were conducted but no systematic relationships were found between baseline covariates and missingness. The model was estimated using all available data.
Results
Descriptive Statistics
Means and SDs of all study variables at all-time points separately for condition are shown in Table 1. Randomization was successful. Details of the randomization results are available elsewhere (see Schoneveld et al. 2018). In addition, no differences were found on the outcome predictors: baseline anxiety symptoms child report, baseline anxiety symptoms mother report, maternal depression, maternal anxiety, and maternal stress.
Table 1
Means, sums, standard deviations, minimum, maximum, skewness, and kurtosis of study variables separately for intervention programs and for time point
MindLight
CBT
Measure
M/Suma
SD
Min.
Max.
SK
KU
M/Suma
SD
Min.
Max.
SK
KU
Pretest
Sex
50b
58.1c
–
–
–
–
53b
60.2c
–
–
–
–
Age
9.87
1.16
7.84
12.65
0.35
−0.62
10.07
1.16
7.85
12.80
0.33
−0.69
Anx. child
0.97
0.40
0.24
2.24
0.73
0.80
0.99
0.41
0.26
2.24
0.88
0.80
Anx. m.
0.51
0.26
0.03
1.20
0.54
0.05
0.50
0.19
0.16
1.05
0.66
0.13
Dep. mother
1.17
2.04
0.00
8.37
2.37
5.32
1.04
1.79
0.00
8.37
2.28
4.95
Anx. mother
0.54
1.12
0.00
5.47
2.69
7.68
0.57
0.99
0.00
5.47
2.37
7.31
Stress mother
3.13
3.09
0.00
12.51
0.91
0.29
3.10
3.00
0.00
12.51
1.08
1.08
Social SE
2.34
0.68
0.86
4.00
−0.11
−0.31
2.28
0.60
0.53
3.57
−0.12
0.15
Emotional SE
1.84
0.76
0.00
3.29
−0.18
−0.50
1.84
0.73
0.29
3.86
−0.22
0.01
Academic SE
2.43
0.74
0.43
4.00
−0.52
−0.07
2.28
0.63
1.00
3.71
−0.10
−0.40
Int. problems
4.58
3.27
0.00
13.00
0.70
−0.01
4.62
3.54
0.00
14.81
1.18
1.32
Ext. problems
5.42
3.47
0.00
15.00
0.35
−0.24
5.87
4.01
0.00
16.89
0.74
0.33
Posttest
Social SE
2.53
0.67
0.86
4.00
0.27
−0.22
2.49
0.66
0.57
4.00
−0.46
0.34
Emotional SE
2.10
0.73
0.43
4.00
0.58
0.42
2.18
0.76
0.00
4.00
−0.32
0.77
Academic SE
2.57
0.75
0.86
4.00
−0.15
−0.31
2.56
0.67
1.14
4.00
0.07
−0.40
Int. problems
3.48
3.18
0.00
12.00
1.04
0.13
3.70
3.16
0.00
13.14
0.87
0.11
Ext. problems
5.09
3.11
0.00
14.00
0.28
−0.33
4.79
3.33
0.00
15.08
1.12
1.72
3-months follow-up
Social SE
2.67
0.71
0.51
4.00
−0.57
0.54
2.58
0.66
0.51
4.00
−0.49
0.60
Emotional SE
2.25
0.86
0.57
4.00
0.12
−0.59
2.32
0.73
0.57
4.00
0.06
−0.41
Academic SE
2.72
0.78
0.43
4.00
−0.63
0.48
2.60
0.71
0.71
4.00
−0.31
−0.39
Int. problems
3.65
3.43
0.00
13.00
1.12
0.23
3.33
3.01
0.00
13.00
1.11
1.14
Ext. problems
5.09
3.20
0.00
14.00
0.47
0.14
5.07
3.75
0.00
15.54
0.95
0.61
6-months follow-up
Social SE
2.73
0.67
1.29
4.00
−0.22
−0.33
2.63
0.58
1.43
4.00
0.18
−0.48
Emotional SE
2.37
0.86
0.29
4.00
−0.21
−0.24
2.33
0.79
0.14
4.00
−0.29
0.10
Academic SE
2.67
0.75
0.71
4.00
−0.15
−0.40
2.62
0.68
1.00
4.00
−0.10
−0.55
Int. problems
3.18
3.14
0.00
11.93
1.31
1.16
3.27
2.62
0.00
10.00
0.80
−0.27
Ext. problems
4.75
3.33
0.00
15.00
0.50
0.09
4.88
3.53
0.00
15.24
0.80
0.75
Min. minimum, Max. maximum, SK skewness, KU kurtosis, Anx. anxiety, m. mother, Dep. depression, SE self-efficacy, Int. internalizing, Ext. externalizing
aInternalizing and externalizing problems are sum scores
bNot M, but n girls
cNot SD, but %
Bivariate correlations between secondary outcomes across time points are available in Table 2. The self-efficacy subscales were positively correlated at the same time and over time. Similarly, internalizing and externalizing symptoms were positively correlated within assessments and over time. In addition, Table 3 shows bivariate correlations of maternal mental health problems and anxiety symptoms. Child- and mother-reported anxiety were positively correlated at all time-points, except for child-reported anxiety at pre-test and mother-reported anxiety at 6-months follow-up. In addition, mother-reported anxiety was positively correlated with baseline maternal mental health problems at all time-points. Last, maternal depression, anxiety and stress were positively correlated at pre-test.
Table 2
Bivariate correlations of secondary outcomes across time points
Measure
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Pretest
1. SE soc.
–
2. SE emo.
0.51
–
3. SE aca.
0.31
0.39
–
4. Int.
−0.18
−0.14
−0.06
–
5. Ext.
−0.10
−0.09
−0.36
0.48
–
Posttest
6. SE soc.
0.55
0.38
0.40
−0.09
−0.03
–
7. SE emo.
0.41
0.52
0.41
−0.09
−0.06
0.60
–
8. SE aca.
0.33
0.27
0.69
−0.07
−0.22
0.58
0.56
–
9. Int.
−0.21
−0.18
−0.15
0.72
0.41
−0.18
−0.20
−0.13
–
10. Ext.
−0.06
−0.12
−0.27
0.46
0.81
0.02
−0.06
−0.22
0.47
–
3-months FU
11. SE soc.
0.57
0.36
0.39
−0.14
0.03
0.67
0.47
0.37
−0.21
0.00
–
12. SE emo.
0.32
0.54
0.35
−0.08
0.16
0.55
0.69
0.47
−0.14
0.04
0.65
–
13. SE aca.
0.28
0.24
0.64
−0.05
−0.15
0.48
0.51
0.72
−0.10
−0.18
0.60
0.61
–
14. Int.
−0.18
−0.20
−0.11
−0.75
0.40
−0.11
−0.10
−0.08
0.76
0.47
−0.21
−0.18
−0.06
–
15. Ext.
−0.07
−0.11
−0.29
0.41
0.81
0.00
−0.01
−0.18
0.42
0.84
0.01
0.07
−0.15
0.46
–
6-months FU
16. SE soc.
0.46
0.23
0.20
−0.11
0.05
0.62
0.39
0.27
−0.13
0.02
0.72
0.53
0.36
−0.18
−0.07
–
17. SE emo.
0.27
0.41
0.17
−0.03
0.05
0.48
0.59
0.40
−0.08
0.04
0.41
0.70
0.35
−0.09
0.02
0.58
–
18. SE aca.
0.21
0.15
0.63
0.01
−0.18
0.44
0.40
0.72
−0.04
−0.17
0.44
0.50
0.73
−0.05
−0.18
0.44
0.52
–
19. Int.
−0.17
−0.22
−0.04
0.57
0.32
−0.07
−0.07
0.04
0.67
0.40
−0.20
−0.16
0.01
0.76
0.40
−0.21
0.07
0.00
–
20. Ext.
−0.01
−0.10
−0.28
0.29
0.76
0.00
0.03
−0.21
0.33
0.77
0.02
0.10
−0.18
0.40
0.85
0.08
0.08
−0.22
0.36
Correlations in bold are significant with at least p < 0.05
SE soc. social self-efficacy; SE emo. emotional self-efficacy, SE aca. academic self-efficacy, Int. internalizing problems, Ext. externalizing problems
Table 3
Bivariate correlations of (other) predictors and anxiety symptoms across time points
Measure
1
2
3
4
5
6
7
8
9
10
Pretest
1. Anxiety (child report)
–
2. Anxiety (mother report)
0.20
–
3. Depression mother
−0.01
0.26
–
4. Anxiety mother
0.05
0.22
0.46
–
5. Stress mother
0.01
0.31
0.67
0.46
–
Posttest
6. Anxiety (child report)
0.61
0.25
0.08
0.10
−0.01
–
7. Anxiety (mother report)
0.18
0.69
0.18
0.24
0.23
0.29
–
3-months FU
8. Anxiety (child report)
0.50
0.20
0.05
0.04
−0.03
0.72
0.25
–
9. Anxiety (mother report)
0.25
0.69
0.19
0.24
0.27
0.37
0.79
0.39
–
6-months FU
10. Anxiety (child report)
0.44
0.20
−0.04
0.08
−0.04
0.67
0.19
0.83
0.33
–
11. Anxiety (mother report)
0.14
0.50
0.14
0.28
0.22
0.24
0.66
0.30
0.76
0.25
Correlations in bold are significant with at least p < 0.05
Improvement in Mental Health Outcomes over Time
Table 4 presents the change in secondary outcomes and 95% CIs for both intervention programs over the course of the study. It shows that non-inferiority of MindLight to CBT could be demonstrated at post-test, 3-months follow-up and 6-months follow-up for social self-efficacy, emotional self-efficacy, academic self-efficacy and internalizing problems. For externalizing problems, non-inferiority could only be shown at 3-months follow-up. At post-test and 6-months follow-up, the CI lay entirely to the right of zero, indicating significant differences in favor of CBT. The results are visualized in Fig. 2 (self-efficacy), Fig. 3 (internalizing problems), and Fig. 4 (externalizing problems). Results from these analyses in the completers only sample were similar and are available in Supplementary Material A.
Table 4
Change in secondary outcomes over the study (intention-to-treat sample)
Assessment
MindLight
CBT
Mean differencea
SD
95% CI
Social self-efficacy
Post-test—pre-testb
0.19
0.20
0.01
0.62
[−0.09, 0.10]d
3-months FU—pre-testb
0.33
0.29
−0.04
0.63
[−0.14, 0.06]d
6-months FU—pre-testb
0.38
0.34
−0.04
0.65
[−0.13, 0.06]d
n
82
83
Emotional self-efficacy
Post-test—pre-testb
0.26
0.34
0.08
0.73
[−0.03, 0.19]d
3-months FU—pre-testb
0.40
0.49
0.08
0.68
[−0.02, 0.19]d
6-months FU—pre-testb
0.54
0.52
−0.02
0.73
[−0.13, 0.10]d
n
82
83
Academic self-efficacy
Post-test—pre-testb
0.18
0.22
0.04
0.56
[−0.05, 0.12]d
3-months FU—pre-testb
0.31
0.33
0.02
0.61
[−0.08, 0.11]d
6-months FU—pre-testb
0.24
0.33
0.09
0.60
[−0.01, 0.18]d
n
79
78
Internalizing problems
Post-test—pre-testc
−1.28
−1.03
−0.25
2.47
[−0.63, 0.13]d
3-months FU—pre-testc
−1.32
−1.46
0.15
2.37
[−0.22, 0.51]d
6-months FU—pre-testc
−1.47
−1.39
−0.08
2.81
[−0.51, 0.35]d
N
67
66
Externalizing problems
Post-test—pre-testc
−0.49
−1.06
0.57
2.17
[0.24, 0.90]e
3-months FU—pre-testc
−0.62
−0.79
0.17
2.22
[−0.17, 0.51]d
6-months FU—pre-testc
−0.62
−1.02
0.40
2.43
[0.03, 0.77]e
n
67
66
CI confidence interval
aA negative difference is a difference in favor of MindLight
bA positive score means an increase in self-efficacy
cA negative score means a decrease in problems
dThe 95% CI of the difference in change in secondary outcome lies entirely between the equivalence margins, indicating equivalence of MindLight and CBT
eThe 95% CI of the difference in change in secondary outcome lies entirely to the right of zero, indicating significant differences in favor of CBT
Fig. 2
Differences between programs in self-efficacy (SE), in relation to non-inferiority
Fig. 3
Differences between programs in internalizing symptoms, in relation to non-inferiority
Fig. 4
Differences between programs in externalizing symptoms, in relation to non-inferiority
×
×
×
To define the growth function that best reflected children’s change in social, emotional and academic self-efficacy, internalizing problems, and externalizing problems, we first fitted a linear growth model with intercept (i) and linear slope (s) as latent variables for all secondary outcomes separately in the intention-to-treat (ITT) sample. Most model fit indices showed acceptable fit (Supplementary Material B), especially given our small sample size (Chen et al. 2008). The intercept and linear slope component were significant for all secondary outcomes (Table 5). This indicated that (1) children differed in their initial levels of social, emotional and academic self-efficacy, and internalizing and externalizing problems, (2) that their levels of social, emotional and academic self-efficacy increased (positive slope B), and that their levels of internalizing and externalizing problems decreased (negative slope B) significantly over time. The absence of significant variances of the slope reflects the idea that most children in our sample did change in a similar manner. Second, we added condition in the linear growth function. As expected, we did not find any differences in initial levels and rates of change of the secondary outcomes between conditions (Supplementary Material C). Figure 5a shows the increase in emotional self-efficacy and Fig. 5b shows the decrease in internalizing and externalizing problems separate by condition. The patterns in the other self-efficacy models were similar to the one presented in panel a. Results from the LGCM of the secondary outcomes in the completers only sample were similar and are available in Supplementary Material D.
Table 5
Linear growth model growth curve parameters for secondary outcomes, from pre-test to 6-months follow-up (intention-to-treat sample)
Means
Variances
Intercept
Slope
Intercept
Slope
B
SE
t
B
SE
t
B
SE
t
B
SE
t
Social self-efficacy
2.37
0.05
47.37***
0.50
0.08
6.33***
0.26
0.05
5.28***
0.31
0.18
1.69
Emotional self-efficacy
1.92
0.06
33.65***
0.74
0.10
7.54***
0.31
0.06
4.87***
0.42
0.29
1.47
Academic self-efficacy
2.42
0.05
45.68***
0.40
0.08
5.39***
0.33
0.05
6.30***
0.12
0.17
0.72
Internalizing problems
4.28
0.27
15.73***
−1.84
0.36
−5.11***
8.98
1.32
6.81***
6.28
3.63
1.73
Externalizing problems
5.44
0.28
19.41***
−0.94
0.31
−3.08**
10.35
1.66
6.23***
2.13
2.26
0.95
SE standard error
**p < 0.010; ***p < 0.001, two-tailed tests
Fig. 5
Emotional self-efficacy (in panel a), internalizing problems (int. prob.; in panel b) and externalizing problems (ext. prob.; in panel b) across time by program: ML (MindLight) and CBT. Error bars are standard errors
×
For Whom are MindLight and CBT effective?
As described above, we started with the quadratic growth model of anxiety symptoms found in Schoneveld et al. (2018). This model showed that both child- and mother-reported anxiety symptoms decreased quadratically over time (i.e., the rate of the decrease slowed over time) in both the MindLight and CBT group. In the current study, we added social, emotional and academic self-efficacy, levels of maternal depression, anxiety and stress, and baseline anxiety separately as outcome predictors to the quadratic model. None of these predicted the linear nor the quadratic slope component for both child- and mother-reported anxiety symptoms (Table 6). However, social, emotional and academic self-efficacy, and mother-reported baseline anxiety were associated with the intercept of the quadratic growth model of child-reported anxiety. Specifically, children with higher levels of social, emotional and/or academic self-efficacy, and/or lower levels of mother-reported baseline anxiety showed lower initial levels of child-reported anxiety. In addition, levels of maternal depression, anxiety and stress predicted the intercept of the quadratic growth model of mother-reported anxiety: children from mothers with lower levels of depression, anxiety and stress, showed lower initial levels of mother-reported anxiety. Results from the LGCM of the outcome prediction analyses in the completers only sample are similar and available in Supplementary Material E.
Table 6
Linear regression predicting growth parameters of quadratic growth model of anxiety by mental health predictors at pre-test separate for child-reported anxiety and mother-reported anxiety (intention-to-treat sample)
This study reported secondary outcomes and outcome predictors of effectiveness of a non-inferiority RCT comparing the applied game MindLight with a CBT-program (Coping Cat; van Starrenburg et al. 2017) for childhood anxiety prevention. Findings were reported for post-intervention, 3- and 6-months follow-up using child- and mother-reports.
Improvement in Mental Health Outcomes
The first set of hypotheses were supported in that children in both intervention groups showed improvements in internalizing and externalizing problems, and self-efficacy. The CI approach showed affirmatively that MindLight was non-inferior to CBT over the course of the study for social self-efficacy, emotional self-efficacy, academic self-efficacy and internalizing problems. CBT, however, showed a larger decrease in externalizing problems at post-test and 6-months follow-up. Importantly, improvements in secondary outcomes were sustained up to 6 months after intervention completion. Thus, a relatively short intervention, delivered as a game or face-to-face CBT for 8 weeks, seems to have a significant and promising impact on more than the targeted anxiety.
The effects on internalizing problems are consistent with prior findings that interventions targeting anxiety also effectively reduce internalizing problems (Donovan and March 2014) and emotional symptoms (Morgan et al. 2016). The decrease in externalizing problems corresponds with research showing that anxiety and externalizing problems are strongly associated (Priddis et al. 2014). Last, self-efficacy increased in children participating in our study. This finding is in line with improvements in self-efficacy found in school-refusing children (King et al. 1998) and youth with anxiety disorders (Suveg et al. 2009) following a CBT-program, and now extends these results to applied games. The non-inferiority of MindLight to CBT on internalizing symptoms and self-efficacy suggests that MindLight is as effective as CBT in improving these mental health outcomes. However, CBT was more effective in decreasing externalizing symptoms than MindLight. An explanation could be the (minimal) involvement of parents in CBT, which is recommended in treatment guidelines for externalizing problem behavior (Buitelaar et al. 2013). In sum, these results may suggest that MindLight, an applied game, is as effective as a conventional expert-led group-based CBT prevention program in enhancing self-efficacy and internalizing symptoms beyond reductions in anxiety.
For Whom are MindLight and CBT effective?
The second set of hypotheses were not supported: none of the mental health variables predicted interventions’ effectiveness in preventing anxiety symptoms over time. First, we found no differences in the rate of change in anxiety symptoms for children with relatively lower or higher levels of baseline anxiety. This is in contrast to prior research that has demonstrated that anxiety symptoms at baseline could impact the response to CBT positively (van Starrenburg et al. 2017) or adversely (Cunningham et al. 2016). Our results seem to suggest that the responsiveness to MindLight and CBT has little to do with the severity of presenting problems children began with. In addition, levels of children’s self-efficacy did not predict interventions’ effectiveness. A possible explanation is the rather low variance in self-efficacy scores in our sample. Most children rated themselves at the middle “not good, not bad” end of the scale, perhaps because we selected children from the general population. Other prevention studies (e.g., Tak et al. 2014) have also found rather low variance on self-efficacy. This restricted range precludes detecting effects.
Next to child factors, we also examined maternal mental health (i.e., maternal depressive, anxiety and stress symptoms) as possible outcome predictors. In line with our hypothesis, maternal mental health problems did not affect changes in children’s anxiety symptoms over time. It is important to note, however, that the prevalence of maternal mental health problems was relatively low in our sample. Considering the important role of maternal mental well-being in children’s development (Goodman et al. 2011; Powdthavee and Vignoles 2008) more research may be needed. For example, it may be interesting to select children for a prevention program based on whether their mother is highly stressed versus not and examine differential effects.
Although anxiety symptoms at baseline, self-efficacy and maternal mental health did not predict the change in anxiety symptoms, there were some interesting associations between these mental health variables and initial levels of anxiety symptoms that warrant further discussion. First, as expected, children that rated themselves as more anxious at baseline also reported lower levels of social, emotional and academic self-efficacy. Second, initial levels of child-reported anxiety were predicted by mother-reported children’s anxiety at baseline. This result basically represents cross-informant agreement of baseline anxiety levels, which is generally significant but low (De Los Reyes et al. 2015). Last, higher initial levels of mother-reported children’s anxiety were predicted by higher maternal depression, anxiety and stress symptoms. This finding suggests that mothers with mental health problems might have a biased perception of their child’s emotional well-being (Briggs-Gowan et al. 1996). That is that mothers’ rating of their children’s anxiety is not so much a reflection of children’s actual anxiety level but reflects their own struggles with depression, anxiety and/or stress. In sum, these results suggest that MindLight and CBT can enhance mental health despite difficulties children or their parents may have.
Strengths, Limitations and Future Research
The current study used a non-inferiority randomized controlled design to assess the effectiveness of an applied game against the standard of anxiety prevention (i.e., CBT). More specifically, we extended previous work about the effectiveness of MindLight on anxiety symptoms by assessing effects on secondary outcomes and potential outcome predictors, factors that are often neglected in previous research. In addition to the broader range of outcomes, we extended previous studies on anxiety prevention programs by including children’s as well as maternal reports. The result that internalizing and externalizing problems decreased according to both children and mothers seems to imply that skills transferred from the program context to everyday life, where mothers were able to observe their children’s behavior. Last, we addressed a limitation of applied games research—a focus on short-term outcomes—by assessing children and mothers directly, 3 and 6 months after the program. These follow-ups give insight into the immediate, short-term but also longer-term effects of anxiety prevention programs, which informs implementation research. Programs with only short-term benefits might be less favorable than programs that show sustained beneficial effects.
Apart from these strengths, this study has several limitations that need to be addressed in the future. First, the internal consistency statistics for the social self-efficacy subscale were in a questionable range, which may have affected the results related to social self-efficacy. However, when the pattern of these results is compared with the results of the remaining types of self-efficacy, no differences appear. Second, the sample consisted of relatively well-functioning children. Whether the current results hold in a more distressed sample is a question for future research. Third, given the absence of a waitlist control group, passage of time could not be eliminated as an alternative explanation for the change in mental health outcomes.
In addition, future studies might want to investigate program effects on other important (secondary) outcomes. For example, as anxiety problems are related to worse academic performance (Nail et al. 2015), academic functioning indicators such as grades and problem solving skills could be relevant. Furthermore, assessing the effect of anxiety prevention programs on children’s social skills might be another important avenue for future research. Social skills start to develop in childhood, are affected by anxiety issues, and at the same time are an important contributing factor in the maintenance and further aggravation of mental health problems (Clarke et al. 2015). Prior research has shown that increasing children’s social skills helps to reduce their behavioral and emotional difficulties (Humphrey et al. 2010).
Next to investigating outcomes, future studies may want to extend the range of possible predictors of effectiveness. An important general predictor of program effectiveness is children’s motivation to change. Children entering the program more motivated might show a larger decrease in anxiety symptoms than children who are less motivated (Dean et al. 2016). In addition, choice could be a moderator of program effectiveness. Given the positive effect of choice and autonomy on intrinsic motivation (Ryan and Deci 2000), children who choose themselves which program to attend might respond more positively than children who are randomly assigned.
Furthermore, the current study leaves open the question about mechanisms of change. Possible psychological mechanisms worth of investigation in future studies are increased active coping skills (Thorne et al. 2013) and decreased maladaptive cognitions (Hogendoorn et al. 2014). Ideally, these mediators should be assessed repeatedly over the course of the intervention, to investigate whether changes in these mediators contribute to decreases in anxiety (Selig and Preacher 2009). In addition, studying mediators of change in applied games versus conventional prevention programs (i.e., moderated mediation) could be an interesting future direction. MindLight focuses more on relaxation whereas CBT centers on changing cognitions. Thus, contrasting the relative contribution of changes in relaxation and cognitions in anxiety reduction will give insight into program specific mechanisms of change but will also contribute to a broader understanding of what is actually driving changes in anxiety.
Notwithstanding the remaining questions for future research, the findings of the current study and our previous work (Schoneveld et al. 2016, 2018) suggest that MindLight could be implemented as an indicated prevention program in schools to reduce anxiety symptoms, internalizing and externalizing problems, and to improve self-efficacy in children with elevated levels of anxiety, regardless of the potential mental health problems or struggles that children or their parents may have. An applied game as MindLight might facilitate cost-effectiveness because no clinicians or teachers need to be involved and there is no specific training required to guide children through the game. Adding MindLight to the list of other prevention programs that already take place in the school context may provide children an alternatively delivered prevention program at school that is potentially less stigmatizing, more accessible, and thereby increasing adherence and their motivation to participate in such programs. Future research on the implementation and cost-effectiveness of MindLight in the school context is recommended.
This research was supported by the Netherlands Organisation for Scientific Research (NWO; grant number 406-12-017 for E.A.S., and grant number 406-16-524 for A.W.).
Compliance with Ethical Standards
Conflict of Interest
MindLight was produced by the PlayNice Institute. Prof. Dr. I.G. is co-founder of this institute. E.A.S., A.W., A.L.A., and R.O. declare that they have no conflict of interest.
Ethical Approval
This study was approved by the ethics committee of the Faculty of Social Sciences of the Radboud University (EC2013-0410-139a1) and registered at the Dutch Trial Register (www.trialregister.nl; Trial ID: NTR4993). The procedures used in this study adhere to the tenets of the Declaration of Helsinki.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Met BSL Psychologie Totaal blijf je als professional steeds op de hoogte van de nieuwste ontwikkelingen binnen jouw vak. Met het online abonnement heb je toegang tot een groot aantal boeken, protocollen, vaktijdschriften en e-learnings op het gebied van psychologie en psychiatrie. Zo kun je op je gemak en wanneer het jou het beste uitkomt verdiepen in jouw vakgebied.
