Figures
Abstract
Adolescent anxiety is debilitating, the most frequently diagnosed adolescent mental health problem, and leads to substantial long-term problems. A randomized controlled trial (n = 138) was conducted to test the effectiveness of a biofeedback video game (Dojo) for adolescents with elevated levels of anxiety. Adolescents (11–15 years old) were randomly assigned to play Dojo or a control game (Rayman 2: The Great Escape). Initial screening for anxiety was done on 1,347 adolescents in five high schools; only adolescents who scored above the “at-risk” cut-off on the Spence Children Anxiety Survey were eligible. Adolescents’ anxiety levels were assessed at pre-test, post-test, and at three month follow-up to examine the extent to which playing Dojo decreased adolescents’ anxiety. The present study revealed equal improvements in anxiety symptoms in both conditions at follow-up and no differences between Dojo and the closely matched control game condition. Latent growth curve models did reveal a steeper decrease of personalized anxiety symptoms (not of total anxiety symptoms) in the Dojo condition compared to the control condition. Moderation analyses did not show any differences in outcomes between boys and girls nor did age differentiate outcomes. The present results are of importance for prevention science, as this was the first full-scale randomized controlled trial testing indicated prevention effects of a video game aimed at reducing anxiety. Future research should carefully consider the choice of control condition and outcome measurements, address the potentially high impact of participants’ expectations, and take critical design issues into consideration, such as individual- versus group-based intervention and contamination issues.
Citation: Scholten H, Malmberg M, Lobel A, Engels RCME, Granic I (2016) A Randomized Controlled Trial to Test the Effectiveness of an Immersive 3D Video Game for Anxiety Prevention among Adolescents. PLoS ONE 11(1): e0147763. https://doi.org/10.1371/journal.pone.0147763
Editor: Susana Jiménez-Murcia, University Hospital of Bellvitge-IDIBELL; CIBER Fisiopatología Obesidad y Nutrición (CIBERObn), Instituto Salud Carlos III; Department of Clinical Sciences, School of Medicine, University of Barcelona, Spain, SPAIN
Received: August 31, 2015; Accepted: January 5, 2016; Published: January 27, 2016
Copyright: © 2016 Scholten et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: Isabela Granic is CEO of the PlayNice Institute, which has developed a biofeedback video game called MindLight, designed to target anxiety among children.
Introduction
Anxiety disorders are one of the most common adolescent psychiatric disorders, affecting up to 17% of adolescents in Western countries [1,2]. Adolescent anxiety is associated with school dropout, lower school grades, suicidality, early substance use, teen pregnancies, and behavioral problems, such as attention deficit hyperactivity disorder and conduct disorder [3–7]. Moreover, adolescent anxiety tends to persist into adulthood and is related to other health problems, such as substance misuse and depression [7,8]. Despite the number of individuals suffering from anxiety and the accompanying personal, economic, and social costs, the focus in mental health has been primarily on treatment rather than prevention [9]. There are several reasons why the prevention of anxiety is pivotal [10]. First, parents and teachers are often unaware of adolescents’ anxiety problems [11] and even if awareness of the anxiety problems does exist, parents and teachers tend to downplay its importance [9]. Second, when adolescents are referred for treatment, the anxiety disorder is often well established, and the majority of adverse effects on peer relationships and school performance have already occurred and are hard to reverse [9]. Identifying effective prevention strategies for adolescent anxiety is therefore crucial.
A new and upcoming prevention strategy for adolescent mental health is the use of video games. Video games are immersive and motivate a wide range of behaviors in adolescents, with 97% of youth (both boys and girls) playing them in their leisure time [12]. Although the vast majority of psychological research on video games has focused on the negative impact, (e.g., addiction [13]; aggression [14]), there is a growing body of evidence for the potential benefits (i.e., cognitive, social, motivational, and emotional) these games may confer upon users, especially adolescents [15–21]. A recent review on the small but significant empirical work on the benefits of video games showed the enormous potential for video games in mental health contexts that can teach youth new forms of emotional strategies, behavior, and thinking [22].
Importantly, video games could address barriers that are frequently encountered in conventional prevention strategies. Specifically, Cognitive Behavioral Therapy (CBT)—the most widely used effective prevention strategy for anxiety problems—suffers from several limitations. Most important are: (1) Engagement: CBT largely relies on imparting psychoeducational information. Adolescents, especially those who do not recognize that they have a mental health problem, often find this boring. Motivating adolescents to engage is one of the most challenging tasks clinicians face [23]. An engaging video game could address the boredom experienced in CBT. (2) Practice: Even though CBT does a good job at imparting new, useful knowledge, there is a large gap between what an adolescent knows and what an adolescent actually does. This problem has long been recognized and therefore many CBT programs include role-playing exercises and homework assignments [24]. However, these exercises are mostly de-contextualized and rarely involve skill training in authentic emotional experiences. In a video game, on the other hand, adolescents are not just exposed to knowledge, but they practice skills over and over again as they are increasingly emotionally challenged; as a result, effective cognitive and emotional skills can become automatized. (3) Need: Many adolescents have difficulties accessing prevention programs because they live in hard-to-reach rural locations or are physically or psychologically unable to commute [25,26]. Lengthy waiting lists are also a problem [27]. Video games can be played wherever and whenever it is preferred and no waiting lists are necessary [28]. (4) Cost: Cost-effectiveness is often a barrier in delivering prevention programs. Many adolescents and families cannot afford participation in prevention programs and schools worldwide are facing cuts in funding [25,27]. Video games are simply far cheaper to deliver than CBT prevention programs [29]. Overall, the use of video games holds great promise for a radically new approach to delivering prevention programs for adolescents with anxiety and overcoming barriers inherent in CBT approaches.
Dojo: Targets of intervention
The present randomized controlled trial (RCT) is one of the first to test this novel prevention approach by evaluating Dojo (developed by GameDesk; http://gamedesk.org/project/dojo/), a video game specifically designed to reduce anxiety in adolescents. Dojo is an emotion management video game and incorporates two evidence-based strategies for reducing anxiety symptoms; emotion regulation training and heart rate variability (HRV) biofeedback. Emotion regulation training refers to a diverse set of processes that influence the occurrence, duration, intensity, and expression of emotion [30]. Examples include progressive muscle relaxation, guided imagery, deep breathing, and positive self-talk. These techniques are well validated in adolescent anxiety treatment and are therefore commonly used in anxiety prevention programs [31–34].
Heart rate variability (HRV) is an electrocardiographic index of autonomic control of the heart and reflects changes in consecutive heart beat intervals [35]. Adolescents with anxiety disorders generally have short intervals between consecutive heart beats (i.e., low HRV). Low HRV is a hallmark of unhealthy stress recovery and is a marker of poor health [36,37]. HRV biofeedback is a technique that teaches adolescents how to alter physiological activity in order to improve health, performance, and learning [38]. It focuses on HRV enhancement by training adolescents to gain awareness of their bodily arousal, to reduce their physiological arousal, and to become more flexible in their physiological response [39,40]. The aim of training HRV through biofeedback in a video game context is to provide a model for self-regulation and an opportunity to practice these skills so that they can be automatically applied outside the game context [20,40].
Dojo trains emotion regulation strategies (i.e., deep-breathing techniques, progressive muscle relaxation, positive thinking, and guided imagery) with instructional videos and by then engaging players in immersive and emotion evoking puzzles that challenge players to use these newly acquired strategies. To encourage players to effectively regulate their emotions, each challenge becomes increasingly difficult if the player’s heart rate increases. Thus, based on clinical research that shows that the combination of HRV biofeedback with emotion regulation training is effective in reducing anxiety—more so than either strategy alone [41–43]–these techniques are the foundations of the game design in Dojo.
Methodological considerations
Despite this promise of using video games as mental health interventions, caution is also warranted. The most important reason is that systematic, randomized controlled trials are sparse on the effectiveness of video games for the prevention of mental health concerns in general and for anxiety in particular [22,44]. Even in the medical and educational fields, where “serious” or “applied” games have been more readily accepted as training tools, very few of those games have ever been scientifically evaluated [45–47]. As a result, it is still unclear how much more effective these games are in changing health-related, educational, and behavioral outcomes than conventional approaches. To enable these fields to advance, outcome evaluations of applied video games need to employ state-of-the-art designs (e.g., RCTs) with appropriate rigor that are in line with CONSORT guidelines and with adequately powered samples [22,44–48]. Moreover, reliable and valid measures of outcomes and moderators and follow-up durations that are sufficiently long to warrant confidence in the claims about video games and the scientific evidence for their effectiveness are essential [44,47].
Another crucial factor to adequately assess potential benefits of video games is choosing an appropriate control group [45,46]. Active control groups are more scientifically rigorous than wait lists or no-attention control groups [49–51]. They insure that attention, or additional placebo mechanisms such as beliefs, behavioral activation, and expectations, do not account for improvements in the experimental condition [49–51]. Yet, active controls are only superior to “no-contact” controls when participants in the active control condition have the same expectations of improvement as participants in the experimental condition. Only then can differential improvements be attributed to the strength of the intervention. In sum, the most convincing evidence for the effectiveness of a video game as an intervention tool will come from studies using active control participants who receive a similar intervention, but that does not specifically target anxiety.
The current study aimed to address all limitations inherent to the vast majority of past evaluations of the effectiveness of video games. In accordance with the intention-to-treat principle, we conducted a sufficiently powered RCT. We included reliable and valid measures of outcomes and carried out a three month follow-up assessment. We selected Rayman 2: The Great Escape (hereafter shortened to Rayman) as the active control game. In comparison to the experimental game Dojo, Rayman was not explicitly designed to incorporate training components that target anxiety. Importantly, before the start of the intervention and before adolescents knew to what condition they were assigned, expectations were equalized, measured and controlled for. If Rayman is an adequate placebo control for the experimental game Dojo, participants should expect the same levels of improvement for both games before the start of the intervention.
Design and Hypotheses
The present two-armed RCT evaluated the effectiveness of Dojo in reducing adolescent anxiety. Adolescents with subclinical levels of anxiety played either Dojo or the control game Rayman. We expected that adolescents who played Dojo, relative to Rayman, would show reduced anxiety levels at three month follow-up.
Given the importance of understanding moderators of outcomes and the identification of subgroups that are more likely to benefit from an anxiety prevention program [52–54], differential effects for age and sex were also explored. Even though the game is designed for adolescents, there may be differential effects for younger versus older players. Several studies suggest that younger children benefit more from anxiety prevention programs than older children [55,56]. In this line, we expected younger adolescents to report less anxiety symptoms at three month follow-up compared to older adolescents. Furthermore, we tested whether there were any differential effects for sex.
Materials and Methods
Design and procedure
Of the 44 eligible secondary schools that were invited to take part in the RCT (i.e., Dutch schools in the province Gelderland within a 1.5 hour travel radius), five schools agreed to participate. We visited the participating schools and during these visits we provided further information about the research project. The recruitment process of participants went through two phases, the screening and the inclusion phase. In the screening phase, we distributed letters about the goals of the study to all 7th, 8th, and 9th grade students (i.e., students between 11 and 15 years of age) and their parents, in collaboration with the school principals. The letter explained to the parents that they could refuse to have their child participate in the screening. Thus, a passive consent procedure was handled in the screening phase. Initial screening on anxiety levels was conducted among 1,347 adolescents. Screening was performed during school hours in school classes. Adolescents independently completed the Spence Children’s Anxiety Scale (SCAS [57]) in the presence of a teacher and a research assistant. Prior to the screening, adolescents were told that their data would be processed confidentially. There were 349 adolescents who met the inclusion criteria for the RCT (i.e., one standard deviation above the mean on at least two SCAS subscales or on the total SCAS score, with exclusion of the subscale Obsessive Compulsive Disorder). Screening data were collected between November 2013 and May 2014, before the intervention was carried out.
In the inclusion phase, all 349 adolescents and their parents were contacted by phone and informed about the results of the screening and the study goals. Detailed information regarding the results of the screening were only provided to parents with the adolescent’s verbal consent. Adolescents who already received mental health care were excluded from further participation. When an adolescent and his or her parents expressed willingness to participate, they received two information letters (i.e., one for the adolescent and one for the parents) in which the study was explained in full. These letters also explained that adolescents could quit the study at any time and that all gathered information would be treated confidentially. Parents could use the enclosed consent form and return envelopes when both parents and the adolescents agreed to participate. Hence, an active consent procedure was followed during the actual inclusion phase. A total of 138 adolescents (40%) and their parents agreed to participate in the present RCT.
These 138 adolescents were randomly assigned (within schools with an equal gender distribution) to the experimental or the control condition. The experimental group was asked to play Dojo and the control group Rayman. Adolescents in both conditions played their video game at school and after school hours. They played the game six times over three weeks, with two one-hour sessions per week. These gaming sessions were held in a computer room or in a classroom with individual laptops and headphones, with all adolescents playing in the same room (they could not hear each other or the other games, given the headphones). During all gaming sessions, a research assistant was present. At the first session, the research assistant provided instructions about starting up the game and the rationale of the game for both groups separately. Critically, it was at this first session that adolescents were informed about the condition they were randomly allocated to. At all subsequent sessions adolescents could come in, start the game, and play. The research assistant was present for questions, to monitor the use of headphones, and to maintain silence. All adolescents filled out a digital questionnaire at home at three consecutive time points: prior to the intervention (i.e., pre-test), immediately following the intervention (i.e., post-test), and at a three month follow-up. Pre-test and post-test data were collected between January and July 2014. Follow-up data was collected between May and September 2014. The adolescents received 40 Euros as compensation for their participation in the study. Approval for the design and data collection procedures was obtained from the ethics committee of the Faculty of Social Sciences of the Radboud University Nijmegen (ECSW2013-0410-140). This procedure was also registered before the start of the study in the Dutch Trial Register for RCTs (http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4379; Trial ID: NTR4379).
Sample size
Research on indicated anxiety prevention programs generally show small to moderate effect sizes [53,58]. We estimated a small effect of ηp2 = .014 using G*Power 3 [59] with an alpha of .05 and a power of .80. A pre-test/post-test/follow-up (within) by 2 group (between) ANOVA would require 57 adolescents in each study condition leading to a total of 114 adolescents.
Randomization
Randomization was performed by an independent researcher from the research institute and applied within schools; with an equal number of participants in the control and the experimental condition within one school. Randomization was carried out stratified by sex.
Participants
A total of 138 adolescents participated in the RCT. At baseline, adolescents ranged from 11 to 15 years old (M = 13.27, SD = .88) and 35% of the adolescents were male. Nearly all adolescents were born in the Netherlands (97.8%) and the remaining adolescents were born in Turkey, Indonesia, or South Africa (2.2%). Educational levels varied, with 26.1% of the adolescents attending lower general education or lower vocational education, 5.1% attending a combination of lower vocational, lower general education, and higher general education, 9.4% attending higher general education, 10.1% attending a combination of higher general education and pre-university education, and the remaining 49.3% attending pre-university education. Thus, the majority of the sample came from high-streamed education tracks.
Loss to Follow-up
At pre-test, 138 adolescents took part in the study. The response rate for the post-test was 93.5% (n = 129). Six adolescents dropped out during the intervention, and three participants attended the intervention but did not fill out the post-test questionnaire (Fig 1). Of these nine adolescents, five were randomized to the experimental condition and four to the control condition. The response rate for the three month follow-up was 91.3% (n = 126). Of the 12 adolescents that did not complete questionnaires, five were randomized to the experimental condition and seven to the control condition. One of the adolescents who did not fill out the post-test questionnaire did fill out the three month follow-up questionnaire. Attrition analyses were conducted to examine if adolescents who stayed in the study and completed the follow-up assessment differed on sex, age, education, ethnicity, study condition, and baseline levels of anxiety symptoms from adolescents who were lost to follow-up. Logistic regression analyses with loss to follow-up as the dependent variable showed no differences on sex (p = .765), age (p = .733), education (p = .311), ethnicity (p = .472), study condition (p = .828), baseline level of total anxiety symptoms (p = .392) and baseline level of personalized anxiety symptoms (p = .698).
Intervention
Control condition (Rayman 2: The Great Escape).
Rayman is a commercially available platform video game developed by Ubisoft Entertainment S.A. that is played on a PC or laptop. It is played from a third-person perspective, and the player has control over the camera.
Rayman was chosen to match the type of attention (adolescents actively playing a video game), but also because it seemed to incorporate game mechanisms that could teach adolescents general skills that may be related to mental health outcomes (e.g., perseverance in the face of failure [60]).
Rayman takes place in a world called the Glade of Dreams. Admiral Razorbeard and his army of Robot Pirates have invaded this world thereby greatly weakening Rayman's powers. Rayman learns that in order to stand a chance against the Pirates, he must navigate through the world, defeat numerous threatening enemies, and solve puzzles. Eventually, Rayman succeeds and the Robot Pirates are destroyed.
Experimental condition (Dojo).
Dojo is a commercially available emotion management biofeedback video game developed by Gamedesk (http://gamedesk.org/project/dojo/; Fig 2) that is played on a PC or laptop. It is played from a first-person perspective.
Dojo takes place in a secret temple below an urban subway system with the player experiencing Dojo in the role of a young person that is going through a hard time in life. The hidden temple houses three rooms that are thematically designed based on different emotions (i.e., anger, frustration, and fear). In each room, the player encounters a dojo master that instructs the player on emotion regulation strategies, and then presents the player with a challenge that tests the player’s newly acquired emotion regulation skills. In the anger room, the dojo master trains the player in positive self-talk and guided imagery skills; here, the player is challenged to a hand slapping contest where negative sentences (e.g., are you lonely?) pop up on-screen to provoke and distract the player. In the frustration room, the dojo master trains the player in muscle relaxation skills; here, the player is challenged to maneuver a glowing ball through an increasingly complex maze, while not being allowed to touch the maze’s walls. Finally, in the fear room, the dojo master trains the player in deep breathing skills; here, the player is challenged to collect bones in a scary labyrinth and at the same time evade a powerful, frightening, and angered ghost.
During these challenges the player’s heart rate is monitored by a biofeedback system manufactured by Wild Divine that measures heart rate via sensors attached to the player’s fingers (http://www.wilddivine.com/accessories/iom-active-feedback-hardware-by-wild-divine/). Heart rate measures from this system are directly read into Dojo so that players’ arousal levels are continuously displayed in the right corner of the video game. To encourage player’s regulation of emotions, each of the game’s challenges become increasingly difficult if the player’s heart rate increases. For example, in one room, the size of the ball which players maneuver through a maze without touching the walls is influenced by the player’s heart rate. The biofeedback system feeds the arousal status of the player back into the game so that if the player gets anxious and his or her heart rate accelerates, the ball becomes bigger making it harder for the player to avoid hitting the maze’s walls. Only by remaining calm and using the emotion regulation strategies taught by Dojo’s dojo masters can the player reduce his or her heart rate, diminish the size of the ball, and beat the challenge.
Measures
Game experience.
Game experience was assessed at pre-test with the question: “How many hours per week do you play video games?” Adolescents could respond by typing in a number (up to 2 decimals) representing the amount of hours they generally play video games.
Game expectations.
Expectations about how much playing Dojo and Rayman could improve anxiety levels were equalized, assessed, and controlled for at pre-test. Before the start of the intervention, all participants read a short overview of both games. Thereby, we intentionally equalized expectations for both games. Participants then indicated on four items whether they thought their real-life behavior could be improved by playing Dojo [49]: “Do you think that playing [Dojo/Rayman] can help you experience less anxious feelings?”; “Do you think that playing [Dojo/Rayman] can help you to be less impulsive?”; “Do you think that playing [Dojo/Rayman] can help you to better cope with frustration?”; and “Do you think that playing [Dojo/Rayman] can help you to relax more?” Response options were “yes” (= 1) and “no” (= 0). Thereby we primed both groups with the same potential expectations. Both overviews can be found in S1 Appendix.
Anxiety.
Anxiety was assessed at all time-points with the Spence Children Anxiety Scale (SCAS [57]. The SCAS measures the adolescents’ perception of the frequency in which they experience anxiety-related symptoms (i.e., generalized anxiety, panic/agoraphobia, social phobia, separation anxiety, obsessive compulsive disorder, and fear of physical injury). The SCAS consists of 44 items, 38 of them reflect specific symptoms of anxiety and 6 are positive filler items to reduce negative response bias. Items are answered on a 4-point scale ranging from “never” (= 0) to “always” (= 3). An example item is: “I worry about things”. The SCAS showed good psychometric properties in a Dutch sample [61]. Internal consistency of the 38 specific anxiety symptoms items in the present sample of n = 138 was high (α = .86). Two outcome variables were calculated based on the SCAS: the mean scale score of the total SCAS (i.e., total anxiety symptoms), and a personalized anxiety symptoms score. This personalized anxiety symptoms score reflects an individual mean of the subscale on which each participant scored highest on at screening. When a participant scored equally high on two or more subscales at screening, a weighted mean of those scales was calculated for each time point.
Strategy of analysis
We performed Chi-square tests and Analysis of Variance (ANOVA) to examine whether randomization resulted in an equal baseline distribution of relevant participant characteristics across the two conditions. Means and standard deviations were computed for anxiety symptoms at all measurement points separately for both conditions, as were correlations, and independent t-tests for sex. Data were analyzed using Mplus 7 [62].
For the main effect analyses, we compared the experimental condition with the control condition. In accordance with the intent-to-treat principle (ITT), all participants that were randomized to a condition were included in these analyses. Missing data were handled by imputing 50 datasets for each condition separately with multiple imputation (MI), using the Markov Chain Monte Carlo (MCMC) method [63]. Linear regression was used with anxiety symptoms (i.e., total anxiety symptoms and personalized anxiety symptoms) as the dependent variable and study condition as independent variable, while controlling for baseline anxiety levels and clustering of data. We used the TYPE = COMPLEX procedure in Mplus to correct for this potential non-independence (complexity) of the data. Between-group and within-group effect sizes (expressed as Cohens’s d) were calculated. Potential differential effects of Dojo depending on sex were explored by calculating an interaction term between condition (Rayman = 0, Dojo = 1) and sex (boy = 0, girl = 1) and including this term in the model as a predictor variable. To examine possible differential effects of Dojo depending on age, we calculated an interaction term between condition (Rayman = 0, Dojo = 1) and age. To avoid the problem of multicollinearity, age was centered before computing the interaction term [64]. The interaction term then was included in the model as a predictor variable.
Next, we conducted Latent Growth Curve Modeling (LGCM), including screening, pre-test, post-test and follow-up, to examine the effects of Dojo on individual levels of anxiety symptoms at baseline (i.e., the intercept) and changes in anxiety symptoms over time (i.e., the slope [65]). LGCM allows individuals to differ on their starting level of anxiety symptoms and the rate of change in anxiety symptoms over time [65]. To deal with missing data all available pairwise information in the data was used [66]. The Chi-square and p-value (p-value >.05), the Comparative Fit Index (CFI >.95), and the Root Mean Square Error of Approximation (RMSEA < .05) were applied to assess the goodness of model fit [67].
In the first step of the analysis the growth function (i.e., linear, quadratic, or cubic) that best reflected changes in anxiety symptoms over time was examined for both conditions separately. Anticipating the results by inspecting the means of anxiety symptoms across the four time points, we expected a quadratic growth curve function to fit the data best. Latent growth parameters estimated within a quadratic growth function are: the intercept (i), the linear component of the growth curve (s), and the quadratic component of the growth curve (q). The most appropriate growth function was selected by comparing all functions and interpreting the model fit indices [67]. In a second step, we added condition to the model to test if Dojo could predict the initial level, the rate of change, or the curvature of anxiety symptoms. To examine the differences in latent growth parameters between both groups, we estimated and compared a baseline model, an intercept model, a linear model and a quadratic model. If adding new components to the model resulted in a significant increase in chi-square, this would indicate a difference between the control and experimental condition on this component. Finally, we explored if sex or age moderated the relationship between the intervention condition and anxiety symptoms.
Results
Descriptive statistics
No differences were observed between the two conditions at baseline for sex, age, ethnicity, education, game expectations, game experience, total anxiety symptoms, and personalized anxiety symptoms (Table 1). Therefore, we did not control for these background variables in our subsequent analyses. Table 2 shows the means and standard deviations for anxiety symptoms at all measurement points, for both conditions separately. The correlations between outcome and background variables are presented in Table 3.
Table 3 shows that adolescents who believed Dojo could help them feel less anxious had similar expectations for Rayman. Additionally, participants who believed that Dojo could help also reported less game experience compared to adolescent who did not believe Dojo could help. Finally, adolescents who thought that either Dojo or Rayman could help them to experience less feelings of anxiety, reported higher levels of total anxiety symptoms at baseline. Independent t-tests for sex and all other variables revealed that boys reported playing video games more often than girls (t = 6.53, p < .001) and that girls reported higher levels of total anxiety symptoms than boys (t = -2.70, p = .008).
Main Effects of Dojo on Anxiety Symptoms
For the main effect analyses, the experimental condition (Dojo) was compared with the control condition (Rayman). Regression analyses showed that anxiety symptoms significantly decreased at follow-up in both conditions: total anxiety symptoms (β = 0.70, SE = 0.04, p < .001) and personalized anxiety symptoms (β = 0.63, SE = 0.05, p < .001). Yet, no prevention effect of study condition at follow-up was found for either total anxiety symptoms (β = 0.03, SE = 0.08, p = 0.724) or personalized anxiety symptoms (β = 0.04, SE = 0.05, p = 0.457). The within-group effect sizes for change in total anxiety symptoms from pre-test to post-test was d = .25 in the control condition and d = .29 in the experimental condition, and from pre-test to follow-up, d = .46 in the control condition, and d = .39 in the experimental condition. The within-group effect size for change in personalized anxiety symptoms from pre-test to post-test was d = .31 in the control condition and d = .32 in the experimental condition, and from pre-test to follow-up, d = .56 in the control condition, and d = .43 in the experimental condition. Between-group effect sizes for change from pre-test to post-test were d = -.02 and from pre-test to follow-up d = .08 for total anxiety symptoms. For personalized anxiety symptoms, between-group effect sizes were d = -.02 for change from pre-test to post-test and d = .14 from pre-test to follow-up. Given the small number of drop outs, the completers-only analyses were almost identical to the intention-to-treat results. More detailed information on the completers only analyses can be obtained from the corresponding author. As can be seen from Table 4, no differential effects were observed for sex or age in the relationship between condition and anxiety symptoms.
Latent Growth Curve Modeling
To examine the growth function that best reflected change in anxiety symptoms over time, we first tested linear models with intercept (i) and slope (s) as latent variables for both conditions separately (Table 5). Since the fit indices were unsatisfactory (p-value was < .05, CFI-value was < .95 and RMSEA was > .05), we introduced a quadratic element (q) to the model. As can be seen from Table 5, the models including the quadratic component showed an excellent fit for both conditions. For total anxiety symptoms, the parameter estimates for the control condition were i = .941 (p < .001), s = -.098 (p < .001), and q = .010 (p = .014), resulting in the following quadratic function: ANX = .941-.098*T+.010*T2 where ANX stands for total anxiety symptoms and T for the months after screening (0, 1, 2, 5). The estimated values of the decrease in total anxiety symptoms for the control condition can be calculated for each time point: .09 (screening—pre-test), .07 (pre-test—post-test), and .08 (post-test—follow-up) (Fig 3). The parameter estimates for total anxiety symptoms in the experimental condition were i = .989 (p < .001), s = -.174 (p < .001), and q = .024 (p < .001) leading to ANX = .989-.174*T+.024*T2. The estimated values of the decrease in total anxiety symptoms for the experimental group were: .15 (screening—pre-test), .10 (pre-test—post-test), and .02 (post-test—follow-up).
The parameter estimates for personalized anxiety symptoms were i = 1.585 (p < .001), s = -.236 (p < .001), and q = .027 (p < .001) for the control condition. The quadratic function was ANX = 1.585-.236*T+.027*T2 with ANX being personalized anxiety symptoms and T the months after screening. The estimated values of the decrease in personalized anxiety symptoms were: .21 (screening—pre-test), .16 (pre-test—post-test), and .14 (post-test—follow-up) (Fig 4). The parameter estimates for the experimental condition were i = 1.661 (p < .001), s = -.382 (p < .001), and q = .053 (p < .001), resulting in ANX = 1.661-.382*T+.053*T2. The estimated values of the decrease in personalized anxiety symptoms were: .33 (screening—pre-test), .22 (pre-test—post-test), and .03 (post-test—FU).
Finally, we tested the relative effect of Dojo on the development of anxiety symptoms over time, by including condition as a predictor in the model (Table 6). To determine the nature of differences between conditions, we compared a baseline model, an intercept model, a linear model, and a quadratic model. The baseline models of both anxiety outcomes showed an excellent fit. No significant differences between the conditions were found for the intercept and the quadratic component in the total anxiety symptoms models (Table 7). The linear component approached statistical significance (Δχ²(1) = 3.19, p = .074), indicating that there was a marginally steeper linear decrease in total anxiety symptoms observed in the experimental condition compared to the control condition (Fig 3). We did not find differences between conditions in the personalized anxiety symptoms models for the intercept and the quadratic component (Table 7). The linear decrease in personalized anxiety symptoms was significantly higher (Δχ²(1) = 5.28, p = .022) in the experimental condition (s = -.382) than in the control condition (s = -.236), indicating that the decrease in personalized anxiety symptoms was significantly steeper for the experimental condition compared to the control condition (Fig 4).
Furthermore, we found no significant interactions for sex or age in the relation between condition and either total anxiety symptoms or personalized anxiety symptoms anxiety (Table 6).
Discussion
The present study examined the effectiveness of the video game Dojo on adolescent anxiety symptoms in a two-armed RCT among an indicated population of 11 to 15 year old adolescents with sub-clinical levels of anxiety. Results showed that adolescent anxiety symptoms significantly decreased in both conditions. Contrary to what we expected, we did not find differences between Dojo and the closely matched control condition. We did observe a steeper decrease of personalized anxiety symptoms (not of total anxiety symptoms) in the experimental condition compared to the control condition in our latent growth curve models. Moderation analyses did not show any differences in outcomes between boys and girls nor did age differentiate outcomes.
Surprisingly, we found that participants in both conditions showed equal improvements in anxiety symptoms at follow-up. It could be that neither Dojo nor the control condition had an advantageous effect on anxiety symptoms or that both conditions were equally effective. The lack of a no-contact control, such as a wait list, means that we cannot be sure that the decrease in anxiety symptoms was any greater than without an intervention at all. In a meta-analysis, school-based CBT anxiety prevention interventions were compared with wait-list control groups and a mean effect size of g = .50 (pre- to post-test) for the school-based CBT prevention interventions and g = .19 for the wait-list control group were reported [68]. Comparing our within-group effect sizes with their mean effect size of the wait-list control group, it seems that more improvement was observed in the present RCT than would be expected in an RCT without an intervention as all within-group effect sizes are almost twice the wait-list effect size. Yet, deductions from these indirect comparisons should only be used to stimulate hypotheses for future research and cannot be taken as conclusions.
Even if it is granted that both conditions showed a reduction in anxiety symptoms, it is still unclear why no differences in anxiety symptoms were found between both conditions. There are several explanations for this findings. First, and what we consider most likely, Rayman, our control game, may have trained some of the same skills that Dojo targets. Even though Rayman was not specifically designed for anxiety reduction, it may incorporate some of the more general action mechanisms that have benefits for adolescent’s emotional development more broadly, and anxiety more specifically [22,69]. Video game designers as commercially successful as those who developed Rayman are wizards of engagement, they are able to design environments that urge adolescents to work hard toward meaningful goals and celebrate the exceptional moments of success after completing challenging tasks [22]. In Rayman, players control a strong and fearless, but also small “underdog” avatar that adolescents can identify with. Players also overcome seemingly insurmountable challenges without instruction or guidance from adults around them. Additionally, this game is built for trial and error learning; most in-game challenges have to be practiced over and over again, and emotions such as anxiety, anger and frustration need to be overcome before players prevail. Through repeated experiences of perseverance in the face of failure [60], perhaps core emotional resilience skills that also relate to anxiety regulation were being trained among our participants. Thus, Rayman might have been more than a mere attention control condition by helping to reduce participants’ frustration and anxiety symptoms. We intentionally chose to avoid cluttering the treatment and prevention research literature with a design that used a no-contact control; but by carefully matching our control condition, we are left less clear about the meaning of both groups improving on their anxiety levels [11,49,70]. Future research would benefit from comparing Dojo to a control group that is less emotionally stimulating, more focused on “cold” cognitive challenges, and therefore, less related to training skills for regulating anxiety or to an existing more traditional intervention such as regular CBT.
A second explanation for the absence of differences in anxiety symptoms across conditions might be that the duration of the intervention downplayed possible beneficial effects of Dojo. In the process evaluation of Dojo, adolescents reported that the duration of the intervention (six training sessions) was too long. Specifically, they reported difficulties in maintaining motivation and boredom after approximately four sessions. Dojo only includes three rooms with training elements and challenges and most adolescents completed these challenges within three to four sessions. In the remaining sessions these participants had to play the game over and over again, with some reporting frustration with this repetition. Considering that programs with a shorter duration tend to have larger effects and the reported frustration with replaying already completed content suggests that shorter duration play time with Dojo might be more effective [71,72].
A last potential explanation might originate in several design-related issues. The intervention was carried out in a group-based context to reach a large number of adolescents. There are studies that report equally strong effects of group-based interventions and individual-based interventions on anxiety [73]. Yet, there are also indications that adolescents with high social anxiety levels respond better to individual interventions [73,74]. The reassurance and social approval provided in an one-on-one context may enhance individual intervention gains. Furthermore, uncontrolled exposure to a group situation may be somewhat overwhelming for socially anxious adolescents, which makes it difficult to fully engage and learn new coping skills [73,74]. As the majority of our sample consisted of adolescents with elevated social anxiety symptoms (70.3%) and multiple adolescents reported issues related to the group context (e.g., not feeling free to fully engage in their video game because of the presence of peers) in their process evaluation, this could have influenced the results.
Another issue concerns the composition of the gaming session groups. Both conditions were randomized within each school and both video games were played in one room simultaneously. Thereby, the control condition could have been contaminated by the experimental condition; adolescents allocated to the control condition may have accidentally received some facets of the intervention since they were in proximity of the experimental group [75,76]. This seems unlikely given that all participants used headphones the entire time they were playing their games; thus, players could not hear what was going on in other games, nor could they talk with one another. Nevertheless, it is recommended for future research to consider whether an individual- or group-based intervention is more applicable and whether two rooms instead of one could be used to deliver the game interventions.
Inspecting the findings on total anxiety symptoms and personalized anxiety symptoms in more detail (Figs 3 and 4), one can see that the largest decrease in anxiety symptoms occurred after screening. This drop in symptoms prior to any intervention is commonly found in studies that report screening to pre-test results [77,78] and is explained by the possible role of non-specific factors in generating intervention effects [79]. Non-specific factors that could have played a role are attention by researchers, the willingness to change, self-monitoring of anxiety symptoms, or emergence of hope [79–81]. We anticipated this phenomenon in both groups because we explicitly and by design attempted to equalize expectations for both groups by framing both Dojo and Rayman as interventions that could have beneficial effects on mental health. Recall that these expectations were triggered before adolescents knew to what condition they were assigned. By doing so, all non-specific factors related to expectations were activated not only in the experimental, but also in the control group.
While non-specific factors could possibly explain the large decrease in anxiety symptoms after screening in both conditions, Figs 3 and 4 also show an observable difference between conditions after screening. Our growth curve models demonstrated that participants in the experimental condition showed a steeper decrease in anxiety symptoms than participants in the control condition between screening and pre-test. This cannot be explained by differences in baseline anxiety symptoms since there were none. Moreover, adolescents’ game expectations prior to randomized assignment were the same across conditions. Although we controlled for expectations, our results still drove a steeper drop in anxiety in the Dojo group. Replication and further exploration of possible experimenter effects should be examined in future studies. Moreover, even though participants of both groups had equal expectations before the start of the intervention, it is possible that their expectations changed while playing. For now, however, we argue for the importance of actually attempting to equalize and then adequately measure these expectations at multiple time points and control for them. Otherwise, in our study, we might have interpreted those expectations to be drivers of our differential effects when they clearly were not.
Exploratory analyses did not show moderation effects of sex or age on the relation between condition and anxiety symptoms in our analyses. This finding was consistent over study condition and different sets of analyses; it seems that the games’ effectiveness did not differ between girls and boys, nor for younger versus older adolescents. Although these findings need replication, it is promising to find that video game interventions may appeal equally to boys and girls in different age groups. However, it is also important to note that both games were adventure, puzzle games and the lack of sex and age differences may not hold for different genres of games or diverse mental health concerns [82–84].
Limitations and implications
A first limitation concerns our use of self-reports. Self-report is necessary to measure subjective anxiety experiences of adolescents in daily life in an ecological valid way, but in addition it is important to use methods that provide insight into the underlying mechanisms of those experiences. Greater physiological reactions to stress (i.e., higher heart rate) and a decreased ability to recover from stressful situations are hallmarks of anxiety [85]. HRV biofeedback video games, such as Dojo, could be perfectly suitable to obtain continuously recorded heart rate data of participants during all training sessions and examine heart rate recovery over time. Unfortunately, at this moment the heart rate data recorded by the Wild Divine IOM is hard to interpret because of occasional temporary signal loss [86]. Our recommendation for future research is to incorporate an objective measure of regulated emotional responding and to improve HRV biofeedback materials such as the IOM. HRV does not have to be continuously recorded during interventions; assessments related to real-world anxiety regulation could be equally useful. For example, an assessment of autonomic markers of arousal and regulation before, during, and after a stressful speech task—measured before and after playing Dojo for several sessions—could be used for testing whether improvements in psychophysiological responses and recovery mediate intervention success [87].
A second limitation concerns the limited outcome variables that were assessed, given the content of Dojo and its training elements. A specific mediating process may be effectively targeted by an intervention, yet if success is primarily evaluated via a broader measure of a disorder (e.g., anxiety), this could result in underestimation of its effectiveness [88]. In this RCT, Dojo might target individuals’ anxiety indirectly via coping skills [89]. In future investigations that examine the effects of Dojo, it seems important to measure specific coping strategies that are taught in the game, namely adolescents’ tendency to use physiological regulation, relaxation techniques, and self-talk in their lives outside of the game. It seems likely that these strategies may have differentiated Dojo from Rayman, suggesting separate mechanisms by which the games worked to decrease anxiety. One’s ability to cope with anxious feelings could play a mediating role in the prevention of anxiety. Since we did not include a proper coping outcome measure, we could not test this hypothesis.
Conclusion
Although we did not find differences between our groups, the effect sizes corresponding to improvements in anxiety symptoms suggest there is promise to a gaming approach. A useful next step may be to develop an extended version of Dojo so that it is a longer, more absorbing game that can compete with the high-quality, commercial-grade games on the current market. It also seems important to examine in-game data for both Dojo and Rayman in order to understand whether certain types of game play mediate outcomes. Although “serious” or “applied” games have been designed for intervention purposes in the medical and education fields, the domain of mental health has been virtually unexplored. We are eager to address this gap by not only designing and improving these games, but also by conducting the most rigorous research on them so that the contribution to prevention science becomes clear and potentially compelling.
Supporting Information
S1 Appendix. Overviews Dojo and Rayman 2: The Great Escape.
https://doi.org/10.1371/journal.pone.0147763.s001
(PDF)
S1 Dataset. Relevant dataset excluding personal information.
https://doi.org/10.1371/journal.pone.0147763.s003
(SAV)
Acknowledgments
We gratefully acknowledge GameDesk, the developers of Dojo, for providing us with access to their game at no cost and for generously engaging in productive discussions around maximizing the impact of games that build emotional resilience. We also want to thank the adolescents who enthusiastically participated in the study, the schools that facilitated this participation and the support of the administrators, without whom this study could never have been completed.
Author Contributions
Conceived and designed the experiments: HS AL RCME IG. Performed the experiments: HS. Analyzed the data: HS MM. Wrote the paper: HS. Critically edited manuscript: HS MM AL RCME IG.
References
- 1. Beesdo K, Knappe S, Pine DS. Anxiety and anxiety disorders in children and adolescents: developmental issues and implications for DSM-V. Psychiat Clin N Am. 2009; 32: 483–524. 2009
- 2.
Costello EJ, Egger HL, Copeland W, Erkanli A, Angold A. The developmental epidemiology of anxiety disorders: phenomenology, prevalence, and comorbidity. In Silverman WK, Field AP, editors. Anxiety disorders in children and adolescents. New York: Cambridge University; 2011. pp. 56–75.
- 3. Duchesne S, Vitaro F, Larose S, Tremblay RE. Trajectories of anxiety during elementary-school years and the prediction of high school noncompletion. J Youth Adolescence. 2008; 37: 1134–1146.
- 4. Kendall PC, Safford S, Flannery-Schroeder E, Webb A. Child anxiety treatment: outcomes in adolescence and impact on substance use and depression at 7.4-year follow-up. J Consult Clin Psychol. 2004; 72: 276–287. pmid:15065961
- 5. Marmorstein NR. Relationships between anxiety and externalizing disorders in youth: the influences of age and gender. J Anxiety Disord. 2007; 21: 420–432. pmid:16875798
- 6. Mazzone L, Ducci F, Scoto MC, Passaniti E, D'Arrigo VG, Vitiello B. The role of anxiety symptoms in school performance in a community sample of children and adolescents. BMC Public Health. 2007; 7: 347–352. pmid:18053257
- 7. Woodward LJ, Fergusson DM. Life course outcomes of young people with anxiety disorders in adolescence. J Am Acad Child Adolesc Psychiatr. 2001; 40: 1086–1093.
- 8. Cartwright-Hatton S, Roberts C, Chitsabesan P, Fothergill C, Harrington R. Systematic review of the efficacy of cognitive behavioural therapies for childhood and adolescent disorders. Brit J Clin Psychol. 2004; 43: 421–436.
- 9. Donovan CL, Spence SH. Prevention of childhood anxiety disorders. Clin Psychol Rev. 2000; 20: 509–531. pmid:10832552
- 10. Rapee RM. The development and modification of temperamental risk for anxiety disorders: prevention of a lifetime of anxiety? Biol Psychiat. 2002; 52: 947–957. pmid:12437936
- 11. Neil AL, Christensen H. Efficacy and effectiveness of school-based prevention and early intervention programs for anxiety. Clin Psychol Rev. 2009; 29: 208–215. pmid:19232805
- 12.
Lenhart A, Kahne J, Middaugh E, Macgill AR, Evans C, Vitak J. Teens, video games, and civics. Pew Research Center: Internet, Science and Tech. 16 Sep 2008. Available: http://www.pewinternet.org/Reports/2008/Teens-Video-Games-and-Civics.aspx. Accessed 28 July 2014.
- 13. van Rooij AJ, Schoenmakers TM, Vermulst AA, van den Eijnden RJ, van de Mheen D. Online video game addiction: Identification of addicted adolescent gamers. Addiction. 2011; 106: 205–212. pmid:20840209
- 14. Anderson CA, Shibuya A, Ihori N, Swing EL, Bushman BJ, Sakamoto A, et al. Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: a meta-analytic review. Psychol Bull. 2010; 136: 151–173. pmid:20192553
- 15. Olson CK. Children’s motivation for video game play in the context of normal development. Rev Gen Psychol. 2010; 14: 180–187.
- 16. Anguera JA, Boccanfuso J, Rintoul JL, Al-Hashimi O, Faraji F, Janowich J, et al. Video game training enhances cognitive control in older adults. Nature. 2013; 501: 97–101. pmid:24005416
- 17. Beale IL, Kato PM, Marin-Bowling VM, Guthrie N, Cole SW. Improvement in cancer-related knowledge following use of a psychoeducational video game for adolescents and young adults with cancer. J Adolesc Health. 2007; 41: 263–70. pmid:17707296
- 18. Cole SW, Yoo DJ, Knutson B. Interactivity and reward-related neural activation during a serious videogame. PLoS One. 2012; 7: e33909. pmid:22442733
- 19. Fagundo AB, Santamaría JJ, Forcano L, Giner-Bartolomé C, Jiménez-Murcia S, Sánchez I, et al. Video game therapy for emotional regulation and impulsivity control in a series of treated cases with bulimia nervosa. Eur Eat Disord Rev. 2013 Nov;21(6):493–9. Epub 2013 Sep 20. pmid:24092598
- 20. Fernández-Aranda F, Jiménez-Murcia S, Santamaría JJ, Gunnard K, Soto A, Kalapanidas E, et al. Video games as a complementary therapy tool in mental disorders: PlayMancer, a European multicentre study. J Ment Health. 2012; 21: 364–74. pmid:22548300
- 21. Kühn S, Gleich T, Lorenz RC, Lindenberger U, Gallinat J. Playing Super Mario induces structural brain plasticity: Gray matter changes resulting from training with a commercial video game. Mol Psychiatry. 2014; 19: 265–71. pmid:24166407
- 22. Granic I, Lobel A, Engels RCME. The benefits of playing video games. Am Psychol. 2014; 69: 66–78. pmid:24295515
- 23.
Crenshaw DA. Therapeutic engagement of children and adolescents: Play, symbol, drawing, and storytelling strategies. 1st ed. Lenham: Rowman & Littlefield; 2008.
- 24.
Kendall PC. Child and adolescent therapy: Cognitive-behavioral procedures. 4th ed. New York: Guilford; 2011.
- 25. Booth ML, Bernard D, Quine S, Kang MS, Usherwood T, Alperstein G, et al. Access to health care among Australian adolescents: young people’s perspectives and their sociodemographic distribution. J Adolescent Health. 2004; 34: 97–103.
- 26. Przeworski A, Newman MG. Efficacy and utility of computer-assisted cognitive behavioural therapy for anxiety disorders. Clinical Psychologist. 2006; 10: 43–53.
- 27. Gega L, Marks I, Mataix-Cols D. Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol. 2004; 60: 147–157. pmid:14724922
- 28. Spek V, Cuijpers P, Nyklicek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: A meta-analysis. Psychol Med. 2007; 37: 319–328. pmid:17112400
- 29. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, et al. Computerassisted cognitive therapy for depression: Maintaining efficacy while reducing therapist time. Am J Psychiatry. 2005; 162: 1158–1164. pmid:15930065
- 30.
Gross JJ, Thompson RA. Emotion regulation: Conceptual foundations. In: Gross JJ, editor. Handbook of emotion regulation. New York: Guilford; 2007. pp. 3–24.
- 31. Albano AM, Kendall PC. Cognitive behavioural therapy for children and adolescents with anxiety disorders: clinical research advances. Int Rev Psychiatr. 2002; 14: 129–134.
- 32. Barrett PM. Evaluation of cognitive-behavioral group treatments for childhood anxiety disorders. J Clin Child Psychol. 1998; 27: 459–468. pmid:9866083
- 33.
Rapee RM, Wignall A, Hudson JL, Schniering CA. Treating anxious children and adolescents: An evidence-based approach. 1st ed. Oakland: Harbinger; 2000.
- 34. Spence SH, Donovan C, Brechman-Toussaint M. The treatment of childhood social phobia: the effectiveness of a social skills training-based, cognitive-behavioural intervention, with and without parental involvement. J Child Psychol Psych. 2000; 41: 713–726.
- 35. Henriques G, Keffer S, Abrahamson C, Horst SJ. Exploring the effectiveness of a computer-based heart rate variability biofeedback program in reducing anxiety in college students. Appl Psychophysiol Biofeedback. 2011; 36: 101–112. pmid:21533678
- 36. Dekker JM, Crow RS, Folsom AR, Hannan PJ, Liao D, Swenne CA, et al. Low heart rate variability in a 2-minute rhythm strip predicts risk of coronary heart disease and mortality from several causes The ARIC Study. Circulation. 2000; 102: 1239–1244. pmid:10982537
- 37. Friedman BH. An autonomic flexibility-neurovisceral integration model of anxiety and cardiac vagal tone. Biol Psychol. 2007; 74: 185–199. pmid:17069959
- 38.
Shaffer F, Moss D. Biofeedback. In: Yuan CS, Bieber EJ, Bauer BA, editors. Textbook of complementary and alternative medicine. Abingdon: Informa Healthcare; 2006. pp. 291–312
- 39. Knox M, Lentini J, Cummings TS, McGrady A, Whearty K, Sancrant L. Game-based biofeedback for paediatric anxiety and depression. Ment Health Fam Med. 2011; 8: 195–203. pmid:22942901
- 40.
Yucha C, Montgomery D. Evidence-based practice in biofeedback and neurofeedback. 1st ed. Colorado: Association for Applied Psychophysiology and Biofeedback; 2008.
- 41. Bradley RT, McCraty R, Atkinson M, Tomasino D, Daugherty A, Arguelles L. Emotion self-regulation, psychophysiological coherence, and test anxiety: results from an experiment using electrophysiological measures. Appl Psychophysiol Biofeedback. 2010; 35: 261–283. pmid:20559707
- 42. Moss D. Heart rate variability and biofeedback. Psychophysiology Today: The Magazine for Mind-Body Medicine. 2004; 1: 4–11.
- 43. Thurber MR, Bodenhamer-Davis E, Johnson M, Chesky K, Chandler CK. Effects of heart rate variability coherence biofeedback training and emotional management techniques to decrease music performance anxiety. Biofeedback. 2010; 38: 28–40.
- 44. Baranowski T, Buday R, Thompson DI, Baranowski J. Playing for real: video games and stories for health-related behavior change. Am J Prev Med. 2008; 34: 74–82. pmid:18083454
- 45. Connolly TM, Boyle EA, MacArthur E, Hainey T, Boyle JM. A systematic literature review of empirical evidence on computer games and serious games. Comput Educ. 2012; 59: 661–686.
- 46. Girard C, Ecalle J, Magnan A. Serious games as new educational tools: how effective are they? A meta-analysis of recent studies. J Comput Assist Lear. 2013; 29: 207–219.
- 47. Primack BA, Carroll MV, McNamara M, Klem ML, King B, Rich M, et al. Role of video games in improving health-related outcomes. Am J Prev Med. 2012; 42: 630–638. pmid:22608382
- 48. Kane RL, Wang J, Garrard J. Reporting in randomized clinical trials improved after adoption of the CONSORT statement. J Clin Epidemiol. 2007; 60: 241–249. pmid:17292017
- 49. Boot WR, Simons DJ, Stothart C, Stutts C. The pervasive problem with placebos in psychology: why active control groups are not sufficient to rule out placebo effects. Perspect Psychol Sci. 2013; 8: 445–454. pmid:26173122
- 50. Christensen H, Griffiths KM, Jorm AF. Delivering interventions for depression by using the internet: randomised controlled trial. BMJ. 2004; 328: 265–276. pmid:14742346
- 51. White KS, Payne LA, Gorman JM, Shear MK, Woods SW, Saksa JR, et al. Does maintenance CBT contribute to long-term treatment response of panic disorder with or without agoraphobia? A randomized controlled clinical trial. J Consult Clin Psychol. 2012; 81: 47–57. pmid:23127290
- 52. Feldner MT, Zvolensky MJ, Schmidt NB. Prevention of anxiety psychopathology: A critical review of the empirical literature. Clin Psychol-Sci Pr. 2004; 11: 405–424.
- 53. Fisak BJ, Richard B, Mann A. The prevention of child and adolescent anxiety: A meta-analytic review. Prev Sci. 2011; 12, 255–268. pmid:21437675
- 54. Gillham JE, Shatte AJ, Reivich K. Needed for prevention research: Long-term follow-up and the evaluation of mediators, moderators, and lay providers. Prevention and Treatment. 2001; 4.
- 55. Barrett PM, Dadds MR, Rapee RM. Family treatment of childhood anxiety: a controlled trial. J Consult Clin Psychol. 1996; 64: 333–342. pmid:8871418
- 56. Lock S, Barrett PM. A longitudinal study of developmental differences in universal preventative intervention for child anxiety. Behav Change. 2003; 20: 183–199.
- 57.
Scholing A, Nauta MH, Spence SH. Spence Children’s Anxiety Scale (Dutch translation of child version). Amsterdam: University of Amsterdam; 1999.
- 58. Tuebert D, Pinquart M. A meta-analytic review on the prevention of symptoms of anxiety in children and adolescents. J Anxiety Disord. 2011; 25: 1046–1059.
- 59. Faul F, Erdfelder E, Lang AG, Buchner A. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007; 39: 175–191. pmid:17695343
- 60. Ventura M, Shute V, Zhao W. The relationship between video game use and a performance-based measure of persistence. Comput Educ. 2013; 60: 52–58.
- 61. Muris P, Merckelbach H, Ollendick T, King N, Bogie N. Three traditional and three new childhood anxiety questionnaires: Their reliability and validity in a normal adolescent sample. Behav Res Ther. 2002; 40: 753–772. pmid:12074371
- 62.
Muthén LK, Muthén BO. Mplus. User’s guide. 7th ed. Los Angeles: Muthén & Muthén; 1998–2012.
- 63. Graham JW. Missing data analysis: Making it work in the real world. Annu Rev Psychol. 2009; 60: 549–576. pmid:18652544
- 64. Holmbeck GN. Post-hoc probing of significant moderational and mediational effects in studies of pediatric populations. J Pediatr Psychol. 2002; 27: 87–96. pmid:11726683
- 65.
Duncan TE, Duncan SC, Strycker LA. An introduction to latent variable growth curve modeling: Concepts, issues, and applications. 2nd ed. Mahwah: Lawrence Erlbaum Associates; 2006.
- 66.
Asparouhov T, Muthén B. Weighted least squares estimation with missing data. Mplus Technical Appendix. 14 Aug 2010. Available: http://www.statmodel.com/download/GstrucMissingRevision.pdf. Accessed 28 July 2014.
- 67. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999; 6: 1–55.
- 68. Mychailyszyn MP, Brodman DM, Read KL, Kendall PC. Cognitive-Behavioral School-Based Interventions for Anxious and Depressed Youth: A Meta-Analysis of Outcomes. Clin Psychol-Sci Pr. 2012; 19: 129–153.
- 69.
McGonigal J. Reality is broken: Why games make us better and how they can change the world. 1st ed. New York: Penguin; 2011.
- 70. Furukawa TA, Noma H, Caldwell DM, Honyashiki M, Shinohara K, Imai H, et al. Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis. Acta Psychiat Scandinavica. 2014; 130: 181–192.
- 71. Stice E, Shaw H. Eating disorder prevention programs: a meta-analytic review. Psychol Bull. 2004; 130: 206–227. pmid:14979770
- 72. Stice E, Shaw H, Bohon C, Marti CN, Rohde P. A meta-analytic review of depression prevention programs for children and adolescents: factors that predict magnitude of intervention effects. J Consult Clin Psychol. 2009; 77, 486–503. pmid:19485590
- 73. Manassis K, Mendlowitz SL, Scapillato D, Avery D, Fiksenbaum L, Freire M, et al. Group and individual cognitive-behavioral therapy for childhood anxiety disorders: A randomized trial. J Am Acad Child Adolesc Psychiatry. 2002; 41: 1423–1430. pmid:12447028
- 74. Stangier U, Heidenreich T, Peitz M, Lauterbach W, Clark DM. Cognitive therapy for social phobia: individual versus group treatment. Behav Res Ther. 2003; 41: 991–1007. pmid:12914803
- 75. Hahn S, Puffer S, Torgerson DJ, Watson J. Methodological bias in cluster randomised trials. BMC Med Res Methodol. 2005; 5: 10–17.pmid:15743523
- 76. Puffer S, Torgerson DJ, Watson J. Cluster randomized controlled trials. J Eval Clin Pract. 2005; 11: 479–483. pmid:16164589
- 77. Allart-van Dam E, Hosman CM, Hoogduin CA, Schaap CP. The Coping with Depression Course: Short-term outcomes and mediating effects of a randomized controlled trial in the treatment of subclinical depression. Behav Ther. 2003; 34: 381–396.
- 78. Wijnhoven LA, Creemers DH, Vermulst AA, Scholte RH, Engels RCME. Randomized controlled trial testing the effectiveness of a depression prevention program (‘Op Volle Kracht’) among adolescent girls with elevated depressive symptoms. J Abnorm Child Psych. 2014; 42, 217–228.
- 79. Stice E, Rohde P, Seeley JR, Gau JM. Testing mediators of intervention effects in randomized controlled trials: An evaluation of three depression prevention programs. J Consult Clin Psychol. 2010; 78: 273–280. pmid:20350038
- 80. Arrindell WA. Changes in waiting-list patients over time: data on some commonly-used measures. Beware!. Behav Res Ther. 2001; 39: 1227–1247. pmid:11579991
- 81.
Henderson S, Byrne DG, Duncan-Jones P. Neurosis and the social environment. 1st ed. New York: Academic; 1981.
- 82. Terlecki M, Brown J, Harner-Steciw L, Irvin-Hannum J, Marchetto-Ryan N, Ruhl L, et al. Sex differences and similarities in video game experience, preferences, and self-efficacy: Implications for the gaming industry. Curr Psychol. 2011; 30: 22–33.
- 83. Greenberg BS, Sherry J, Lachlan K, Lucas K, Holmstrom A. Orientations to video games among gender and age groups. Simulat Gaming. 2010; 41: 238–259.
- 84. Homer BD, Hayward EO, Frye J, Plass JL. Gender and player characteristics in video game play of preadolescents. Comput Hum Behav. 2012; 28: 1782–1789.
- 85. Monk C, Kovelenko P, Ellman LM, Sloan RP, Bagiella E, Gorman JM, et al. Enhanced stress reactivity in paediatric anxiety disorders: implications for future cardiovascular health. Int J Neuropsychoph. 2001; 4: 199–206.
- 86.
Schuller B, Friedmann F, Eyben F. Automatic recognition of physiological parameters in the human voice: Heart rate and skin conductance. Proceedings of the IEEE International Conference; 2013 May 26–31. 10.1109/ICASSP.2013.6639064
- 87. Klimes-Dougan B, Hastings PD, Granger DA, Usher BA, Zahn-Waxler C. Adrenocortical activity in at-risk and normally developing adolescents: Individual differences in salivary cortisol basal levels, diurnal variation, and responses to social challenges. Dev Psychopathol. 2001; 13: 695–719. pmid:11523855
- 88. Reardon S. NIH rethinks psychiatry trials. Nature. 2014; 507: 288. pmid:24646977
- 89. Muris P, Mayer B, den Adel M, Roos T, van Wamelen J. Predictors of change following cognitive-behavioural treatment of children with anxiety problems: a preliminary investigation on negative automatic thoughts and anxiety control. Child Psychiat Hum D. 2009; 40: 139–151.