Effectiveness, Acceptability, and Mechanisms of Change of the Internet-Based Intervention StudiCare Mindfulness for College Students: a Randomized Controlled Trial

College students were recruited via circular emails sent out by colleges across Germany, Switzerland, and Austria. Additionally, recruitment was conducted via flyers and posters, social media, student unions, and student counseling. Students were informed about StudiCare and given a link to the StudiCare homepage (www.​studicare.​com). The homepage provided detailed information on StudiCare Mindfulness and study participation, as well as the possibility to sign up. Participants had to (a) be at least 18 years, (b) be currently enrolled in college, (c) have a moderate to low level of mindfulness according to a cut-off of < 37 on the Freiburg Mindfulness Inventory (FMI; cutoff was chosen as it represents the medium FMI value in the general population (Walach et al., 2006)), (d) are not currently undergoing psychotherapy, (e) are not currently enrolled in another mindfulness training, (f) have sufficient knowledge of the German language (measured by the capability to proceed through enrollment and screening process), (g) have Internet access, and (h) provide written informed consent.

Using G*Power 3.1.9.2 (Faul et al., 2007), the sample size was estimated a priori for the expected effectiveness of StudiCare Mindfulness compared to WL on the primary outcome mindfulness at post-intervention (t1). In a meta-analysis of Internet-based mindfulness interventions, an effect size of g = 0.3 was found (Spijkerman et al., 2016). To detect an effect of this size, the sample size was calculated to be n = 60 subjects per group, with power set at 1-ß = 0.9 and α = 0.05. In total, 25% of the sample size was added to account for expected dropout (Spijkerman et al., 2016), resulting in a total of N = 150.

The study flow is illustrated in Fig. 1. Between May and September 2017, 415 persons registered for the study via the StudiCare homepage (see study flow, Fig. 1). Subsequently, 276 of them participated in the eligibility screening, and 150 ultimately enrolled in the study. There were various reasons for exclusion (FMI > 37, n = 25; no informed consent, n = 89; no baseline data, n = 2; withdrawal of consent; n = 1). Twelve participants in the IG (attrition rate: 16%) and 3 participants in the WL (attrition rate: 4%) did not participate in the post-assessment (t1).

Baseline characteristics were comparably distributed in both study arms, as can be obtained from Table 1 and Table 2. Participants had a mean age of 28 years (SD = 8.3), and the majority of them were female (81%), single (58%), and of German citizenship (83%). They mainly were full-time students (71%), had absolved an average of 10 semesters at college (SD = 5.3), and almost one-third of them studied psychology (28%). Concerning previous treatment, nearly one-quarter of participants had psychotherapy experience (23%). Participants’ general attitude towards Internet- and mobile-based interventions (IMI) and their confidence in the effectiveness of IMI were moderate to high, whereas their perception of risks, technologization threats, and anonymity benefits were moderate.

In this two-armed randomized controlled trial of parallel-group design, we compared the effectiveness of the newly developed guided Internet-based intervention StudiCare Mindfulness (intervention group, IG) to a waitlist control group (WL) (see Fig. 1 for flowchart). Both study groups had unrestricted access to usual treatment options (TAU). The StudiCare Project (https://​www.​studicare.​com/​) was funded by the health insurance company BARMER and has set itself the task of examining and promoting college students’ mental health (Ebert et al., 2019a; Kählke et al., 2019). It is embedded in the “World Mental Health International College Student Initiative” (WMH-ICS) (https://​www.​hcp.​med.​harvard.​edu/​wmh/​college_​student_​survey.​php) and provides a wide selection of Internet- and mobile-based interventions (IMI) (e.g., procrastination, test anxiety, physical activity, depression, substance abuse) to students from German, Swiss, and Austrian colleges and universities while simultaneously evaluating them (Harrer et al., 2018; Kählke et al., 2019). This RCT was conducted and reported according to the CONSORT 2010 statement (Schulz et al., 2010). It was registered a priori at the WHO International Clinical Trials Registry Platform via the German Clinical Studies Trial Register (TRN: DRKS00012559; registration date: 06/29/2017). Additionally, details of study design were described in a study protocol published on OSF (Open Science Framework; https://​osf.​io/​5msn6/​).

Participants were screened for eligibility via the online survey platform “Unipark” (www.​unipark.​com). After providing written informed consent, they completed baseline assessment (t0). They were then randomly allocated to one of the two study groups (IG, WL) by an independent researcher not otherwise involved in the study using an online-based, automated randomization program (www.​sealedenvelope.​com). For this procedure, permuted block randomization with an allocation ratio of 1:1 and variable block sizes of 2 and 4 (randomly arranged) was employed. Because the study was open-label, blinding of group allocation to participants and e-coaches was not possible.

Following randomization, participants were informed about their group allocation, and the IG was given immediate access to the intervention with e-coach guidance. At this point, WL participants only received an information leaflet (the same as the IG) about alternative support options such as psychotherapy, student counseling services, or helplines, and were encouraged to consult the leaflet and seek support if they noticed a deterioration of wellbeing. Six weeks (t1) after randomization, post-assessment took place and WL participants received access to the same intervention as the IG, but without e-coach guidance. Instead, WL participants only received support from the study team in case of technical difficulties. Both IG and the WL participants were informed that the intervention involves some kind of support but did not know about differences in guidance to prevent potential bias.

The intervention StudiCare Mindfulness integrated elements of acceptance and commitment therapy (ACT) (S. C. Hayes et al., 1999), mindfulness-based stress reduction (MBSR) (Kabat-Zinn, 1982), and general stress management techniques (Kaluza, 2015). The intervention was developed by the Department for Clinical Psychology and Psychotherapy of Ulm University in collaboration with psychology students (“by students, for students”). All five modules had a different focus and were developed to be completed in about 60 min (see Table 3 for a detailed description). Contents were chosen based on easy learnability, applicability, and usability for everyday college life. Even though they were a mixture of the three different approaches mentioned above, care was taken to address all six core processes of ACT (Being present, Acceptance, Cognitive Defusion, Self as Context, Values, Committed Action). Participants were recommended to work on one module each week. At the beginning of each module, participants were encouraged to review their most and least mindful moments. Contents of the intervention were mostly provided via text, images, case examples, and interactive elements (e.g., questionnaires), and also emphasized the regular practice of formal (e.g., body scan) and informal (e.g., mindful nature walk) mindfulness exercises by providing a new audio file (á 5–10 min) each module. Homework assignments between modules promoted daily formal meditation practice. Intervention design and content were tailored to the college student population (e.g., student case examples, identification of study-related personal stressors). Participants could access the intervention on the Minddistrict platform (www.​minddistrict.​com), a company specializing in the provision of IMIs. The intervention was available via personal username and password on a 24/7 basis. All transferred data was secured based on ISO27001 and NEN7510 guidelines. Participants were informed that StudiCare Mindfulness is not a replacement for psychotherapy, recommended to seek counseling in case of distinctive mental health problems, and provided with further support options and contact details. To increase intervention adherence (Mohr et al., 2011), e-coaches (psychologists DV and KP, trained and supervised by HB and AK) gave weekly semi-standardized feedbacks after completion of each module specific to the participants’ input via the online-massaging system of the Minddistrict platform and responded to messages within two workdays. Additionally, they could be contacted by participants on demand. The main aim of the feedback messages was to provide positive reinforcement and encourage participants to continue working on the intervention. For this purpose, e-coaches also sent reminders if participants did not meet their self-chosen deadline for the next module. If participants did not respond to three reminders, the module was temporarily disabled until participants got back in touch with the e-coach.

Measurement took place before (t0; baseline) and 6 weeks after randomization (t1; post-intervention) via the online survey platform “Unipark.” An email reminder strategy was employed to minimize study dropout.

The primary outcome mindfulness was assessed with the short scale of the Freiburg Mindfulness Inventory (FMI) (Walach et al., 2006). The 14 items are rated on a 4-point scale (1 = “rarely” to 4 = “almost always”; total score: 14–56). It was chosen due to its brevity and multi-aspect operationalization of the mindfulness construct, covering the aspects “Mindful presence,” “Non-judgmental Acceptance,” “Openness to experience,” and “Insight.” However, as factor analysis revealed high inter-correlations between these aspects, the FMI measures mindfulness as a holistic, single-factor construct. Because it is semantically independent of a Buddhist or meditation context, the FMI is also suitable for participants without meditation experience. It has previously demonstrated high internal consistency (α = 0.84) (Heidenreich et al., 2006), as well as sensitivity to change (Walach et al., 2006). In this study, Cronbach’s α was 0.73 and McDonald’s ω was 0.72.

The widely used 9-item depression screening of the Patient Health Questionnaire (PHQ-9) was used to measure depressive symptoms (Kroenke & Spitzer, 2002). The nine items range from 0 = “not at all” to 3 = “nearly every day” (total score: 0–27). The PHQ-9 has proven to be a valid instrument with excellent internal consistency of Cronbach’s α = 0.89 (Kroenke et al., 2001). A Cronbach α of 0.72 and a McDonald ω of 0.71 were calculated in this study.

The 7-item Generalized Anxiety Disorder Questionnaire (GAD-7) (Spitzer et al., 2006) was used for assessing anxiety. It is rated on a 4-point scale from 0 = “not at all” 3 = to “nearly every day” (total score: 0–21) and is a valid and reliable screening instrument for anxiety with a high internal consistency of Cronbach’s α = 0.89 (Löwe et al., 2008). In this study, Cronbach’s α of 0.85 and McDonald’s ω of 0.86 were found.

Subjectively experienced stress was measured using the Perceived Stress Questionnaire (PSQ-20) (Fliege et al., 2005). The PSQ-20 consists of 20 items rated on a 4-point Likert scale from 1 = “almost never” to 4 = “usually” (total score: 0–100). Its four subscales (worries, tension, joy, demands) with five items each were shown to have satisfactory internal consistencies with Cronbach’s α = 0.80–0.86 (Fliege et al., 2005). Both Cronbach’s α and McDonald’s ω were 0.91 in this study.

The Short-Form Survey 12 (SF-12) is a widely used instrument to measure health-related quality of life (Ware et al., 1996). It consists of 12 items and two subscales, the physical component summary score (PCS) and the mental component summary score (MCS) (total scores: 0–100). The SF-12 has been demonstrated to be psychometrically valid and reliable, with Cronbach’s α of 0.89 for the PCS and 0.79 for the MCS (Wirtz et al., 2018). In this study, for the PCS Cronbach’s α was 0.65 and McDonald’s ω was 0.69, and for the MCS Cronbach’s α was 0.74 and McDonald’s ω was 0.75.

Acceptability was operationalized by assessing intervention satisfaction and adherence. Based on the Client Satisfaction Questionnaire (CSQ-8) (Larsen et al., 1979), four items were created to briefly assess intervention satisfaction (“To what extent did our online-training meet your needs?”). The items were rated on a 4-point scale of specific response alternatives (e.g., 1 = “met almost all of my needs,” 4 = “didn’t meet any of my needs”). Additionally, participants rated each module regarding how much they liked it on a 10-point scale (1 = “not at all,” 10 = “very much”). Adherence was assessed by recording the number of modules completed, defining “per protocol” (PP) adherence as the percentage of participants that completed at least 4 out of 5 modules (80% of the intervention) 6 weeks after randomization (t1). Finally, quantitative and qualitative data were collected on participants’ satisfaction with various aspects of the intervention using self-constructed items (e.g., “What could be optimized?”).

Several sociodemographic and other variables were assessed to investigate potential effect-modifying influences: age, gender, marital status, psychotherapy experience and baseline mindfulness, depressive symptoms, anxiety symptoms, and stress. Additionally, baseline personality scores were assessed via the Big Five Inventory (BFI-10) (Rammstedt & John, 2007), which measures the five dimensions extraversion, agreeableness, conscientiousness, neuroticism, and openness. Two items for each dimension are rated on a 5-point Likert scale from 1 = “disagree strongly” to 5 = “agree strongly” (total scores: 1–5). The BFI-10 has been demonstrated to have acceptable validity and reliability (test–retest reliability: 0.65–0.87) (Rammstedt & John, 2007). In this study, test–retest reliability of the five dimensions was 0.64 to 0.85. Finally, the Attitudes towards Psychological Online Interventions Questionnaire (APOI) (Schröder et al., 2015) was employed to measure attitudes towards IMI. It consists of 16 items that form a total score (16–80) as well as four sub-dimensions (“Skepticism and Perception of Risks,” “Confidence in Effectiveness,” “Technologization Threat,” “Anonymity Benefits”; total scores: 4–20) and is rated on a 5-point Likert scale from 1 = “totally agree” to 5 = “totally disagree.” The APOI has previously demonstrated acceptable to good internal consistency with Cronbach’s α of 0.77 (Schröder et al., 2015). Both Cronbach’s α and McDonald’s ω in this study were α = 0.70. For the moderator analyses, only the baseline total score was tested.

All statistical analyses were performed with IBM SPSS (version 26) with a significance level of α = 0.05 (two-sided). Reliability of outcome measures was assessed by calculating both Cronbach’s alpha (α) and McDonald’s omega (ω) at t0. An exception was the BFI-10, for which test–retest reliability was calculated due to only two items in each dimension.

Data analyses were conducted according to intention-to-treat (ITT) principle, comprising all randomized participants. Missing data was imputed employing a Markov chain Monte Carlo multivariate imputation algorithm (van Buuren, 2007), setting all viable variables from t0 and t1 as predictors. N = 20 datasets were imputed. We assumed data to be missing at random (Enders, 2010). Predictive mean matching was applied as the imputation method (Little, 1988). Additional per-protocol analyses (PP) were conducted for intervention completers to examine potential influence of intervention adherence on effectiveness. Following the PP definition above, these subanalyses (n = 122) included all IG participants who completed at least 80% of the intervention (4 of 5 modules) at t1. Additionally, to investigate potential baseline differences between intervention completers and dropouts, independent samples t-tests (continuous variables) or chi-squared tests (discrete variables) were calculated.

To examine group differences post-intervention (t1), general linear modeling (GLM) was utilized. While checking assumptions for the initially planned ANCOVAs, unequal regression slopes were detected for some of the secondary outcome variables, violating the assumption of homogeneity of regression slopes. Consequently, a switch to GLM was decided to account for interactions between group variable and baseline scores. For these respective secondary outcome variables (depressive symptoms, anxiety symptoms, stress), interactions were added to the regression models and reported accordingly. To further examine significant interactions between participants’ baseline characteristics and outcome variables, simple slopes analyses were conducted via the “Process” macro for SPSS, version 3.5 (A. Hayes, 2017). Dichotomous variables were dummy-coded; continuous variables were z-standardized. Means (M), standard deviations (SD), standardized regression coefficients (β), and their 95% confidence intervals (CI) were reported. Additionally, Cohen’s d (within-group, between-groups) along with 95% CI was calculated. Following Cohen’s rule of thumb (Cohen, n.d.), d = 0.2 was interpreted as a small, d = 0.5 a medium, and d = 0.8 a large effect.

For the primary outcome mindfulness (FMI), Reliable Change Index (RCI) was calculated in order to assess the number of participants achieving a reliable, positive outcome vs. no change vs. reliable deterioration (Jacobson & Truax, 1991; Morley & Dowzer, 2014) using the following formula: 1.96 * SD * sqrt(2) * sqrt(1—rel). We used the SD of the whole sample at t0 (SD = 4.43) and the internal reliability of the FMI (α = 0.84) according to the test authors (Walach et al., 2006). If participants’ FMI score increased or decreased more than 4.91 points from t0 to t1, they were coded as reliably improved reliably deteriorated, respectively.

Six weeks post-intervention (t1), mindfulness was significantly improved by 1.18 standard deviations in intervention group (IG) compared to waitlist control group (WL) (β = 1.18, 95% CI: 0.96 to 1.40, p > 0.001), corresponding to a between-group effect size of d = 1.37 (95% CI: 1.01; 1.73) (see Table 2).

More participants in IG compared to WL showed reliable improvement in mindfulness (IG: n = 74, 77.0%, WL: n = 75, 10.7%; χ²(1) = 66.70, p < 0.001). Additionally, fewer patients in the IG compared to the WL showed reliable deterioration (IG: n = 74, 1.4%; WL: n = 75, 10.7%; χ²(1) = 5.70, p < 0.034).

For four of the five secondary outcomes, significant group differences were found in IG compared to WL at t1 (see Table 2 for overview and Table 5 in the supplementary material for detailed regression results). For depressive symptoms (interaction: F_{(3, 145)} =  − 2.06, p = 0.041), anxiety symptoms (interaction: F_{(3, 145)} =  − 2.72, p = 0.007), and stress (interaction: F_{(3, 145)} =  − 3.11, p = 0.002), interactions of group and respective baseline values were significant and therefore added to the model. For these three outcomes, size and, in one case (anxiety symptoms), even significance of the intervention effect depended on participants’ baseline level of the respective outcome. Depressive symptoms were significantly reduced by 0.75 standard deviations in IG compared to WL at average baseline depressive symptom level (β =  − 0.75, 95% CI: − 0.99 to − 0.51, p < 0.001). Participants with low baseline depressive symptoms (− 1 SD) profited less from the intervention (β =  − 0.50, 95% CI: − 0.84 to − 0.16), whereas participants high on baseline depressive symptoms (+ 1 SD) benefited the most (β =  − 1.00, 95% CI: − 1.34 to − 0.66). Similar results were found for stress, which was also significantly reduced in IG compared to WL at average baseline stress level (β =  − 0.94, 95% CI: − 1.17 to − 0.72, p < 0.001). Participants with low baseline stress (− 1 SD) profited less from the intervention (β =  − 0.59, 95% CI: − 0.90 to − 0.27), whereas participants with high baseline stress level (+ 1 SD) benefited the most (β =  − 1.30, 95% CI: − 1.61 to − 0.98). Finally, anxiety symptoms were also significantly decreased in IG compared to WL at average baseline anxiety level (β =  − 0.49, 95% CI: − 0.77 to − 0.22, p < 0.001). Participants with low baseline anxiety symptoms (− 1 SD) profited less from the intervention, with the CI crossing 0 and the difference between IG and WL no longer being significant at this level of baseline anxiety (β =  − 0.11, 95% CI: − 0.50 to 0.28). Again, participants with high baseline anxiety levels (+ 1 SD) benefited the most from the intervention (β =  − 0.87, 95% CI: − 1.27 to − 0.48).

The mental quality of life component was significantly improved by 0.70 standard deviations (β = 0.70, 95% CI: 0.32 to 0.60, p < 0.001). No significant differences were found for the physical quality of life component (β = 0.02, 95% CI: − 0.26 to 0.30).

For the primary outcome mindfulness, per-protocol analysis (PP; all IG participants that had completed at least 80% of the intervention at t1) showed similar results to the ITT analysis (β = 1.25, 95% CI: 0.99 to 1.50, p < 0.001). The same applies for the secondary outcomes physical (β =  − 0.04, 95% CI: − 0.39 to 0.30, p = 0.801) and mental quality of life (β = 0.75, 95% CI: 0.42 to 1.07, p < 0.001). PP effects for depressive symptoms (β =  − 0.79, 95% CI: − 1.06 to − 0.51, p < 0.001), stress (β =  − 0.96, 95% CI: − 1.22 to − 0.69, p < 0.001) and anxiety symptoms (β =  − 0.50, 95% CI: − 0.83 to − 0.17, p = 0.003) were also similar to ITT results. However, intervention effects no longer varied with baseline levels of the respective outcomes.

At t1, participants of the IG had completed an average of 3.57 modules (SD = 1.55), which equals around 70% of the intervention. Seventy participants had completed module 1 (95%), 64 completed module 2 (86%), 55 module 3 (74%), 48 module 4 (65%), and 27 had completed all modules (36%). Consequently, two-thirds of IG participants were adherent according to the “per protocol” definition (4 of 5 modules completed). No significant differences between adherent and non-adherent participants were found on any of the baseline variables. Participants were able to keep using the intervention after t1, increasing the average number of modules in the IG to 4.00 (SD = 1.58; 80% of the intervention) by the time of data analysis in June 2020. In IG participants that took part in t1 assessment (n = 62), satisfaction with the intervention was high. The intervention was rated as excellent or good by 95%, 90% would definitely or probably recommend it to a friend, 90% definitely or generally received the intervention they had hoped for, and 87% stated that their needs were almost all or mostly met. The same applies to satisfaction with every single module, with a medium of M = 8.08 (SD = 0.41) on a 10-point scale. The highest-rated module was module 5 (“Being mindful towards yourself”) with M = 8.60 (SD = 1.54), whereas the lowest-rated module was module 4 (“What makes your life valuable?”) with M = 7.39 (SD = 2.42).

Of the variables potentially moderating the effect of the intervention on the primary outcome mindfulness at t1, only baseline level of BFI-10 personality dimension “Openness to experience” was found to significantly moderate intervention effect (interaction: β =  − 0.24, 95% CI: − 0.44 to − 0.04; p = 0.018). Mindfulness at t1 was increased most effectively in participants low on baseline openness (− 1 SD; β = 1.42, 95% CI: 1.17 to 1.68) in IG compared to WL, whereas participants with high baseline openness levels (+ 1 SD; β = 0.94, 95% CI: 0.62 to 1.27) benefited the least. None of the other examined moderator variables yielded significant results (see Table 6 in the supplementary material).

Table 4 reports the results of the mediation analyses for the three outcome variables, which examined whether mindfulness at t1 mediated the association between group and depressive symptoms/anxiety symptoms/stress at t1 for the IG versus the WL, controlling for the baseline values of mindfulness and the respective outcome variable (for path diagrams, see Fig. 2 in supplementary material). Mindfulness at t1 was found to mediate the association between being in IG vs. WL and depressive symptoms, anxiety symptoms, and stress 6 weeks post-intervention (t1), which is indicated by bootstrapped 95% confidence intervals of the indirect effects (a*b) not crossing 0.

First, this study lacks long-term follow-up assessment, which made evaluation of long-term effects impossible.

A second limitation is our choice of a waitlist control group. It has been discussed that using passive control groups can be associated with bias in estimating intervention effects (i.e., overestimation) (Furukawa et al., 2014). Future studies could evaluate the effectiveness of StudiCare Mindfulness in comparison to attention placebo or alternative interventions such as face-to-face mindfulness training.

Third, our study design did not allow us to evaluate effectiveness of the Internet-based intervention independent of e-coaching. To answer this question and ensure intervention scalability, future trials should dismantle the effectiveness of an unguided versus guided version of StudiCare Mindfulness.

A fourth limitation concerns the exploratory mediation and moderation analyses. Because we only had two assessment time points, we could not establish chronology of change in the mediation analyses (Kazdin, 2007). Therefore, we cannot definitively exclude the possibility that mediation flows in a different direction. Additionally, other meaningful mediation variables might be at work that we did not explore, such as constructs specific to ACT (e.g., psychological flexibility, meaningfulness). Because of our limited sample size, we might have failed to find existing moderators due to insufficient power. Furthermore, our eligibility criteria and recruitment measures resulted in restricted variance for some variables (e.g., low mindfulness, young college students, positive attitudes towards IMI). This might have limited the generalizability of the results of both the main and exploratory analyses.

A fifth limitation concerns the fact that the exclusive use of self-report instruments can lead to overestimating intervention effects (e.g., social desirability bias) (Nederhof, 1985). One way to deal with this problem is to combine self-report data with psychobiological markers. StudiCare Mindfulness is currently being evaluated in another subsequent study that includes the examination of effects on psychobiological markers such as hair stress markers (Schultchen et al., 2020).

As no universally accepted definition of mindfulness exists, another important concern relates to the operationalization of mindfulness chosen for a given trial (Vam Dam et al., 2018). Following Van Dam et al.’s recommendations, we carefully outlined our primary outcome measure (mindfulness as holistic, single factor construct assessed with the Freiburg Mindfulness Inventory), mindfulness/meditation practices, and intervention protocol. When interpreting results, one must consider the boundaries associated with this specific operationalization of mindfulness. For example, we were not able to differentiate between different aspects of mindfulness or between state and trait mindfulness.

A final limitation concerns the potential negative effects of mindfulness, which we did not explore in this RCT. Although evidence points to the potential of mindfulness interventions for mental health promotion, researchers call for a better understanding of differential effects, adverse effects, and optimal dosage of such interventions (Britton, 2019). Future study designs should allow for a systematic investigation of these questions.