Entrée or Sampler? A Randomized Controlled Trial of Two Approaches to Single Session Internet-Based Interventions

Cognitive-behavioral therapy (CBT) is an effective treatment for people with depression (Strunk et al., 2017), with similar efficacy for face-to-face and online modalities (Carlbring et al., 2018). Cognitive-behavioral interventions focus on helping clients develop specific skills. Skill improvements in cognitive-behavioral interventions are associated with greater reductions in depressive symptoms (Barber & DeRubeis, 2001; Hundt et al., 2013; Schmidt et al., 2019). Quality of skill use has also been found to predict lower depressive symptoms at follow-up (Neimeyer & Feixas, 1990) and reduced risk of relapse following a successful course of treatment (Strunk et al., 2007). However, cognitive-behavioral interventions do not provide symptom relief for a substantial portion of clients (Cuijpers et al., 2014). One approach to improving the impact of cognitive-behavioral interventions is to infuse evidence-based decision-making into intervention design. Although research has tested the use of different clinical interventions, many questions that might inform cognitive-behavioral interventions have yet to be tested empirically. In a recent experiment addressing one such question, Bruijniks et al. (2020a) tested once- and twice-weekly variations of both CBT and interpersonal psychotherapy. Despite being offered the same number of sessions overall, those assigned to twice-weekly sessions experienced more depressive symptom change.

Examining these experimental questions may also be useful for designing brief internet-based cognitive-behavioral interventions. Such brief interventions have attracted considerable interest for their potential to reach a greater number of people with little or no ongoing provider time (Leykin et al., 2014). In the treatment of depression or elevated depressive symptoms, unguided internet-based cognitive-behavioral interventions have been shown to reliably outperform control conditions (Karyotaki et al., 2021; Leykin et al., 2014). Although the rates of dropout in unguided interventions are somewhat higher than alternative treatments, unguided interventions may still be worth examining as they are inexpensive to implement and can easily reach a wide audience. In the context of the recent pandemic, there was also reason for attention to the clinical problem of loneliness, which is related to depression and a host of other negative outcomes (Cacioppo et al., 2015). The prevalence of loneliness increased considerably with the COVID-19 pandemic (Killgore et al., 2020; McGinty et al., 2020). CBT interventions focusing on identifying and evaluating maladaptive social cognitions have been found to decrease loneliness (Cacioppo et al., 2015; Käll et al., 2020). Studies of online cognitive-behavioral interventions appear to be a valuable avenue for expanding the reach of mental health treatments and for examining how the design of interventions impacts outcome.

In this study, we examined two approaches to designing unguided single-session internet-based cognitive-behavioral interventions: an entrée and a sampler approach. In the entrée approach, a single skill is taught in-depth. In the sampler approach, limited coverage of multiple skills is provided (with comparable overall intervention length). We examined whether these approaches would yield differences in symptom reduction, loneliness, and cognitive behavioral coping skills. To place our effort in context, we first review various interventions teaching cognitive, behavioral, and interpersonal skills, highlighting key aspects of those interventions.

Cognitive restructuring interventions teach clients to identify and evaluate the evidence for their negative automatic thoughts. These procedures may lead to cognitive change, which refers to modifying one’s thoughts or beliefs to include alternative, balanced perspectives (Beck et al., 1979). The use of cognitive strategies and cognitive change has been related to improvements in depressive symptoms (Schmidt et al., 2019; Tang & DeRubeis, 1999). Research suggests focusing on identifying thoughts versus gathering information on thoughts can lead to improvement within the context of treatment (Teasdale & Fennell, 1982). A standalone course on CBT skills yielded more improvement in these skills than control conditions but did not outperform the control conditions in reducing depressive symptoms or dysfunctional beliefs (Bruijniks et al., 2018, 2020b). In the context of an online cognitive-behavioral intervention, change in cognitive skills was found to predict subsequent depressive symptom change (Forand et al., 2018).

Behavioral interventions in CBT teach clients to examine their day-to-day behaviors and help them consider alternatives (Beck et al., 1979). These procedures may lead to behavioral activation, which refers to a process by which people engage in activities likely to increase positive reinforcement in their lives (Dimidjian et al., 2011). Research suggests brief interventions teaching behavioral activation strategies can lead to reductions in depressive symptoms (Gawrysiak et al., 2009). Internet-delivered behavioral activation appears to exert small to medium-sized effects on symptoms of depression (Han & Kim, 2022).

A major focus of interpersonal interventions in CBT is teaching clients assertiveness skills. Clients may be asked to craft a response to another person in which they identify an issue, express their feelings, make a request, or negotiate a difficult topic. Previous reports suggest clients’ most common goal in CBT is to improve relationships, with one study suggesting 83% of clients set such goals (Uebelacker et al., 2008). Other treatments include interpersonal elements as an important aspect of treatment, including dialectical behavior therapy in its use of assertiveness training (Linehan, 2014). Some research has suggested considerable benefit from social skills training in terms of reducing depressive symptoms (Bellack et al., 1981). Although not all forms of CBT emphasize assertiveness skills, an emphasis on assertiveness can be part of many forms of CBT and some cognitive behavioral interventions have placed a special focus on assertiveness skills (Cheavens et al., 2012; Speed et al., 2018). The results highlighted from cognitive, behavioral, and interpersonal interventions suggest they improve therapeutic outcomes, but leave unanswered questions about how to best draw on these different approaches when designing an intervention.

Cognitive-behavioral interventions have often drawn on one or more of these intervention approaches. The Unified Protocol (Barlow et al., 2011a, 2011b) is an influential face-to-face cognitive-behavioral treatment that draws on multiple interventions (i.e., using a sampler approach). The Unified Protocol offers a number of modules, including psychoeducation, emotion awareness, addressing avoidance, flexible thinking, and emotional exposure (Barlow et al., 2011a). An early study showed the Unified Protocol outperformed waitlist control (Farchione et al., 2012). In a non-inferiority trial, the Unified Protocol was comparable to single-disorder treatments in symptom reduction (Barlow et al., 2017). Interestingly, dropout was lower with the Unified Protocol than in the single-disorder treatments [but see Osma et al. (2022)]. Although it is unclear what might have accounted for the lower rate of dropout, the authors suggested the impact of the Unified Protocol may be its transdiagnostic focus. It is unclear what role a variety of intervention components might play in the effects of the Unified Protocol or other cognitive-behavioral treatments. Internet cognitive-behavioral interventions including multiple components, such as Beating the Blues and Thrive, have been shown to reduce symptoms of depression, compared to control conditions (Forand et al., 2018; Stuart et al., 2022).

A variety of intervention strategies appear to alleviate depressive symptoms. Although interventions vary in their depth/breadth of skill education (i.e., use of an entrée versus sampler approach), we are not aware of any direct comparison of these two approaches. In this study, we provided an experimental comparison in the context of unguided single-session internet-based interventions intended to teach cognitive-behavioral skills. To assess the impact of these interventions, we also included a no-intervention control condition. Although some interventions rely on a wide variety of strategies, a large portion of interventions focus on developing a narrower, or even individual skill, as the primary effort to achieving positive change, such as behavioral activation (e.g., Martell et al., 2022). Given the success of these treatments, we hypothesized the entrée condition would outperform the sampler condition in reducing depressive symptoms and loneliness and increasing the development of cognitive behavioral coping skills. We also hypothesized both the entrée and sampler conditions would outperform the control condition on these outcomes.

Descriptive statistics for participant demographics are provided in Table 1. The sample was predominantly female and tended to be middle-aged. Just over a third described their race as other than White. For descriptive statistics for outcome variables by time and condition, see the online supplement. Figure 1 shows the flow of participants through the study.  

We conducted a focal test of two approaches to designing unguided single-session internet-based cognitive-behavioral interventions: an entrée approach providing in-depth training on one skill and a sampler approach briefly introducing participants to three CBT skills. To our knowledge, this is the first experimental test examining these alternate ways of structuring interventions. We failed to find differences between conditions on change in depressive symptoms, loneliness, coping skill usage, or coping skill quality. Unguided online cognitive-behavioral interventions such as those used in this study generally have modest effects (Karyotaki et al., 2021). Although our intervention was similar in content to other online interventions, it was quite brief, lasting approximately 30 min. However, participants were asked to complete multiple coping skill worksheets during the intervention and to complete an additional coping skill worksheet between Time 1 and 2. Although the dropout rate we observed was higher than we had hoped, a recent meta-analysis examining dropout in unguided interventions found dropout rates ranged from 40 to 83% (Karyotaki et al., 2015). In one recent study of a multi-part unguided online intervention, only 43% of participants returned following the first training session (Ji et al., 2021). Among those who provided data at Time 2 in our study, there was no condition difference on the primary outcomes.

One of the most interesting differences that emerged from the comparison of the entrée and sampler conditions was the higher retention rate for the sampler condition. The higher dropout in the intervention conditions compared to the control condition may be accounted for by the lower requirements of the control condition. However, there is no such explanation for the difference in dropout between the entrée and sampler conditions. These conditions were matched for time and all study requirements. Thus, participants likely found it more engaging to learn about a variety of coping skills than to learn about one skill in-depth. Our finding echoes that reported for dropout in Barlow et al.’s (2017) trial of the Unified Protocol, albeit that finding involved face-to-face treatment. In that trial, clients participating in the Unified Protocol were less likely to drop out than those participating in a single-disorder treatment. Although our interpretation emphasizes the focus on a narrow or broader range of skills, other differences between the treatments may have played a role in either study. For example, the developers of the Unified Protocol have emphasized its transdiagnostic focus. It is unclear to what extent the interventions in the present study impact a narrow or broader set of emotional disorder symptoms. In any case, both in the present study and in Barlow et al.’s study, the condition that included a greater variety of cognitive-behavioral intervention strategies reduced participants’ risk of dropout. Although caution is warranted in making this connection, it raises the possibility that using a larger variety of interventions reduces dropout in a robust way, across brief and face-to-face interventions.

We also conducted analyses evaluating two moderators of condition differences. In analyses of initial depressive symptom severity, we found that among those with higher initial symptom severity those in the sampler condition outperformed by showing greater coping skill usage those in the control and entrée conditions. These findings broadly align with previous work suggesting a brief intervention was related to increased CBT skills, but not improvements in depressive symptoms (Bruijniks et al., 2020b). In general, the existing literature suggests treatment differences among those with low severity are negligible, whereas more meaningful treatment differences occur among those with more severe depressive symptoms (Driessen et al., 2010; Fournier et al., 2010). The moderating effect between the sampler and control conditions was in fact numerically stronger than the effect involving sampler and entrée. Thus, we take our moderation findings to suggest a benefit of the sampler approach. This approach performed particularly well among those with greater depressive symptoms, which is the context that treatments that advantages tend to show most reliably.

Although the relationship between condition and skill usage was moderated by initial depressive symptom severity, the relationship between condition and skill quality was not. The reasons for this are not clear. But, one possibility is that brief interventions such as those we used are more effective in encouraging skill usage, but may not be sufficient to substantially increase the quality of participants’ CBT skills.

Contrary to what one might expect, we found the entrée and sampler conditions were more effective when they were not aligned with the preferred focus of participants. The entrée condition outperformed the sampler condition among those with a greater preference for using multiple skills. The sampler condition outperformed the entrée condition among those with a greater preference for using one skill. Some previous findings suggest treatments offering a contrast with clients’ typical coping style offer greater benefits. Among patients with depression, avoidant personality characteristics predicted stronger response to cognitive therapy, whereas obsessive personality characteristics predicted a more positive response to interpersonal therapy (Barber & Muenz, 1996; but see also Kikkert et al., 2016). These findings have been interpreted as consistent with the hypothesis of complementarity (i.e., a treatment complementing a patient’s style will be more advantageous). This evidence suggests offering strategies distinct from one’s preference, at least in specific ways, may lead to added benefit. Our finding suggests an advantage to encouraging those who rely on a broader range of skills to focus in on one skill and encouraging those who rely on one skill to broaden their repertoire.

Several limitations merit comment. First, the population sampled from was not limited to those seeking treatment. Participants recruited via Mturk likely participate for the financial incentive, though these incentives were modest and may have been inadequate to retain them. This may be a partial explanation for the high dropout rate. Regardless, it is unclear to what extent results may differ among those who are seeking treatment versus those not seeking treatment. Second, our study involved brief, unguided online interventions. The inclusion of guidance (i.e., coaching, help with technical issues) has been found to improve the therapeutic impact of online interventions (Karyotaki et al., 2021). It is unclear whether guidance might impact the relative efficacy of the entrée and sampler interventions. Third, our measure of preference for using one vs. multiple coping skills (the BDQ) was created for the purposes of this study. There is limited information on the psychometric properties of the measure and the findings should be interpreted with caution. Fourth, the assessment of participants completing a coping worksheet is limited as a measure of engagement. More detailed assessments of participants engagement with the skills introduced in the interventions are needed in future research. Finally, the moderation analyses were exploratory and no correction for the number of tests was applied. As such, replication is especially important.

In conclusion, an entrée and a sampler approach to single-session internet-based cognitive-behavioral interventions yielded similar effects across primary outcomes. Both intervention conditions did not yield significantly greater effects than a control condition. Nonetheless, interesting differences emerged from additional comparison of these conditions. The sampler condition produced a higher retention rate than the entrée condition. For those with more severe depressive symptoms, the sampler condition yielded greater improvements in frequency of coping skill usage than the entrée condition. We encourage future research that continues to consider how interventions are structured so as to optimize the reach and impact of the interventions available. People may differ in the intensity and structure of interventions they find most appealing and effective. Studies of experimental comparisons between intervention approaches that include longer interventions and studies that utilize longer-term follow-up assessments are needed to test the generalizability and durability of these effects.