Effectiveness of Psychotherapy for Internalising Symptoms in Children and Adolescents When Delivered in Routine Settings: A Systematic Review and Meta-analysis

Approximately 13% of young people worldwide are estimated to experience a mental health problem, with anxiety (6.5%) and depressive disorders (2.6%) being two of the most common (Polanczyk et al., 2015). Globally, mental health problems in youth cause the largest burden of disease (Gore et al., 2011) and their impact is projected to continue to increase (Baranne & Falissard, 2018). Along with neurodevelopmental and behavioural disorders, internalising disorders (anxiety and depression) are leading contributors to disease burden caused by mental health problems (Gore et al., 2011). A recent United Nations Children’s Fund report (2021) indicates that the annual loss in human capital due to mental health conditions in 0–19-year-olds is US$387.2 billion, with USD$340.2 billion reflecting disorders that include anxiety and depression. Effective and scalable treatments are a critical public health concern.

A large body of evidence has established that psychological therapies show efficacy in treating anxiety (Baker et al., 2021; Higa-McMillan et al., 2016; James et al., 2020; Reynolds et al., 2012; Weisz et al., 2017) and depression (Eckshtain et al., 2020; Weersing et al., 2017; Weisz et al., 2017) in children and adolescents. Cognitive behavioural therapy (CBT) produces increased anxiety remission compared to waitlist and attention control conditions, but the evidence is weaker when compared to treatment as usual (TAU; James et al., 2020; Weisz et al., 2017). Similarly, CBT is efficacious for the treatment of adolescent depression (Weersing et al., 2017), yet effects appear to be larger for interpersonal psychotherapy (Eckshtain et al., 2020). Therefore, CBT and interpersonal psychotherapy (for depression) are likely to be associated with improved symptoms for mood and anxiety disorders, and are ideal candidates for use in routine settings.

While research on the efficacy of treatments is critical, these studies have typically been conducted in research settings where expert clinicians deliver the treatments in ideal conditions to often restricted (i.e. highly selected) samples (Weisz et al., 2013). Following a systematic search of clinical trials on youth psychopathology, Weisz and colleagues (2005) found that fewer than 2% of all studies had involved youth referred for treatment (vs. recruited via advertisements) by practitioners (vs. research clinicians) in clinical practice settings (vs. research settings). Research in routine settings typically involves more heterogeneous samples and the expertise, training and supervision of therapists is more closely aligned with regular service delivery (Bauer et al., 2015) and these organisational, service delivery, and clientele-based differences have been shown to impact on effectiveness (Wuthrich et al., 2021). Understanding whether and which psychotherapies can effectively treat youth anxiety and depression in routine settings is a critical public health concern if treatment of these common and debilitating mental health conditions is to be implemented in mental health services.

Two recent systematic review and meta-analyses have started to address this gap. Bear et al. (2020) compared the effectiveness of treatment for anxiety and depression compared to TAU delivered in mental health services. The study found some evidence of effectiveness of treatment on the primary outcome at pre-post with a medium to large effect size (Hedges’ g =  − 0.74), and a large effect size (Hedges’ g =  − 0.87) at follow-up. However, effects were pooled across psychotherapy, pharmacotherapy, or combined treatments compared to TAU with differences in effect size found for informant type (i.e. pre-post effects differed for clinician assessors [− 1.3], youth [− 0.70] and parents [− 0.59]), and presenting problem type (i.e. at pre-post effects were largest for depression [− 0.89], then anxiety [− 0.66], and mixed [− 0.52]). However, it is unclear what the effectiveness is for psychological therapy when compared to a range of control conditions (e.g. TAU, waitlist, placebo, active), whether differences exist between setting types (such as schools, outpatient clinics, hospitals), disorder types and informant types in terms of outcomes on pre-post, follow-up, and remission status. It is also important to determine the effectiveness of psychological interventions when applied in routine settings by routine staff, not by research staff who are likely to differ from routine staff with respect to level of training, therapeutic preferences, and supervised practice (Wuthrich et al., 2021). Wergeland et al. (2021) examined the effectiveness of CBT specifically when applied by clinicians in routine settings in both controlled and uncontrolled trials and found a large effect size pre–post-treatment (Hedges’ g =  − 1.50); however, this was based on outcome measures pooled across clinician rating scales, self-report, and parent report, and pooled across controlled and uncontrolled trials. It is also not entirely clear if clinicians were employed by the services themselves or were externally employed research staff delivering the intervention within the setting; therefore, the methodological approaches limit an understanding of whether psychotherapy is more effective than other therapies and TAU in these settings.

Therefore, the aim of this systematic review and meta-analysis was to extend the literature to examine the effectiveness of psychotherapies delivered in routine care settings by setting staff to treat internalising disorders (anxiety and depressive disorders) in school-aged children (4–18 years) based on child, parent, and independent evaluator report on outcomes at pre-post, follow-up, and remission status. The second objective was to compare the effectiveness of traditional CBT (current best practice) delivered in routine care settings to the effectiveness of other psychological treatments delivered in routine care settings. The final objective was to examine potential moderators of these effects by type of control group and treatment setting.

Figure 1 depicts the method for identifying, screening, and including studies. The database search identified 12,921 studies, and 2204 duplicates were found and removed. Of the 388 studies screened at full text, 343 did not meet the criteria. Two further studies appeared to meet all the inclusion criteria but were excluded because there were insufficient data to calculate effect sizes (Dorsey et al., 2020; Konanur et al., 2015). A total of 45 studies met criteria for inclusion in the review.

Populations who are recruited within typical randomised controlled trials for the treatment of paediatric anxiety and depression are usually conducted in research settings, and their samples are commonly more educated, wealthier, and racially homogenous than the general population and especially relative to populations seeking treatment from community settings (Ehrenreich-May et al., 2011; James et al., 2020). Professionals that deliver interventions in routine settings are also likely to differ in qualifications, therapeutic preferences, and frequency of peer supervision received (Wuthrich et al., 2021). Therefore, it cannot be assumed that outcomes from typical research trials will generalise to delivery within broader community settings. The current study aimed to evaluate this question of effectiveness rather than efficacy using strict criteria to ensure the clinicians worked in the community settings and that interventions were not delivered by external research staff. We examined effects against non-active controls (which as defined by the Cochrane Handbook includes placebo, no treatment, waitlist and treatment as usual) and active controls (that included variants of therapies and pharmacotherapy).

In this study, the majority of included studies used child-reported outcomes from pre-to-post-treatment to evaluate outcomes in community settings, with inconsistent inclusion of parent and independent evaluator reporting of outcomes. Fewer studies examined changes from pre to follow-up periods (with follow-up periods varying from 1 to 12 months) limiting conclusions that can be drawn as to maintenance of effects. Fewer studies examined changes in remission status over time. This reflects differences in the routine assessment approaches typically used in community settings.

Overall, the results indicated that current psychotherapies for anxiety and depression in youth are partially effective when delivered in routine settings compared to non-active controls, with some differences in effect depending on problem type (depression vs. anxiety), the informant, and whether post or follow-up treatment period was examined (as previously found by Bear et al., 2020; De Los Reyes & Kazdin, 2005; Eckshtain et al., 2020; Weisz et al., 2017). Psychotherapy was associated with significant small to medium effects for reduction in anxiety symptoms according to child report (ES =  − 0.26), parent report (ES =  − 0.25) and independent evaluator report (ES =  − 0.50) from pre-to-post-treatment over non-active controls. However, effects were nonsignificant for child and parent report from pre to follow-up, and were unable to be tested for independent evaluator reports. Remission status for anxiety symptoms could only be examined according to independent evaluator report and findings indicated significant small effects for improved remission of anxiety disorders from pre- to post-treatment for psychotherapy over non-active controls, but nonsignificant effects from pre to follow-up matching the pattern of findings for anxiety symptoms.

Similarly, when symptoms of depression were the target, psychotherapy was associated with small significant effects over non-active interventions from pre- to post-treatment according to both child (ES =  − 0.19) and parent (ES =  − 0.34) reports, with nonsignificant effects for independent evaluator reports. The effects were also small and significant from pre to follow-up for child report (ES =  − 0.36), with insufficient studies available to examine effects for the other reporter types. There were nonsignificant effect differences for remission of depression symptoms according to child and independent evaluator reports from pre to post compared to non-active controls. The effects according to other reporter types could not be evaluated due to small study numbers.

The findings in relation to pre-to-post-treatment effects for reductions in anxiety and depressive symptoms are promising and indicate translation of benefits into routine settings over non-active controls. On initial examination, it appears that the effects from effectiveness trials are smaller than shown in most meta-analyses of efficacy trials particularly for anxiety. For example, reviews of the efficacy of treatments for youth anxiety have indicated average controlled effect sizes of around 0.7 according to reports from either child or parent (James et al., 2020; Reynolds et al., 2012), while reports from youth in trials on the treatment of depression indicate effects of around 0.35 (Eckshtain et al., 2020; Weisz et al., 2017). However, direct comparison is difficult to make since it is confounded with methodology. Of their nature, most studies in routine settings are unable to delay or deny treatment to participants and hence compare their experimental treatment against TAU or another active condition. In fact, over 80% of the trials included in the current meta-analysis compared against TAU, placebo, or an active control. In contrast, the majority of efficacy research delivered through university clinics still utilises waitlist or no treatment controls. Meta-analyses have shown that the effects of treatments when compared against TAU or active controls are considerably smaller than comparison against waitlist (Weisz et al., 2017). For example, in the treatment of youth anxiety, Reynolds et al. (2012) report pre-post effects of 0.76 in comparison to passive controls and 0.35 in comparison to active controls including TAU based on combined effect sizes across reporter. Similarly, in the treatment of youth depression, Eckshtain et al. (2020) report pre-post effects of 0.49 in comparison to no treatment and 0.29 in comparison to TAU based on combined effect sizes across reporter type. In comparison to these figures, the results from the current meta-analysis are not dissimilar. Therefore, it may be that the apparently modest effects from effectiveness trials for youth anxiety and depression is a result of the methods used in these studies (comparison against TAU) rather than any loss of efficacy when moving from university trials to real world application. In line with this, it should be noted that in the studies included in this review, TAU comprised a wide range of interventions including supportive counselling, CBT skills, pharmacology and self-help resources which would be expected to lead to improved symptoms in their own right; hence smaller effect size differences are to be expected.

Direct comparisons with the results of the two previous systematic reviews in routine settings is difficult due to methodological differences. Bear et al.’s (2020) effectiveness review pooled psychotherapy and pharmacotherapy effects, as well as informant type, and included studies in which treatments were delivered by research clinicians making direct comparisons difficult. Wergeland et al. (2021) only included studies of CBT and pooled effects from controlled and uncontrolled studies for within-subject comparisons only. By separating effects for psychotherapy type, informant type and only including controlled studies with evidence for psychotherapy being delivered by routine setting staff, our review provides some unique insights into the effectiveness of psychotherapy in routine settings.

Evidence for longer-term maintenance of outcomes from effectiveness trials over and above non-active controls was limited, particularly in regard to depression outcomes. Overall, there was little indication that the small post-treatment benefits over and above non-active controls lasted into the various follow-up periods. Pre to follow-up effects across child and parent informants were not significantly different from non-active controls for child anxiety but untested for independent evaluator reports. In reporting of depression, children reported significant small (ES =  − 0.36) effect size benefits of psychotherapy over non-active controls at follow-up, with analyses for other informants not conducted due to small sample size. These results indicate ongoing benefits for depression for psychotherapy over non-active controls, but that the benefits of psychotherapy over non-active controls disappears over follow-up periods for anxiety. However, comparisons could not be made for all reporter types due to small study numbers, and independent evaluator reports, in particular, were missing. Hence limited information exists to understand the longer-term effects of interventions over and above non-active controls.

Findings related to remission was exceedingly limited making conclusions difficult. There was evidence according to independent evaluator report of greater remission of anxiety disorders with psychotherapy over non-active controls at post (51.3% vs 36.2%), but not at follow-up (50.9% compared to 38.9% for non-active control), with insufficient data available to examine remission status using child or parent report. This pattern of results matched the results for anxiety symptom severity in which psychotherapy effects over non-active control were no longer significantly greater at follow-up. The lack of difference between therapy condition at follow-up may be due to increased relapse in the psychotherapy condition, increased remission in the non-active control, or a combination of both effects. Regardless of the nonsignificant benefits of psychotherapy over non-active control at follow-up the remission rates remained at relatively high rates. For depression, remission status could only be examined using child (54.1% vs 48.2%) and independent evaluator report (48.6% vs 44.3%) from pre to post, and these comparisons were associated with nonsignificant effects over non-active controls. This finding is perhaps surprising given the significant effects showing benefits of psychotherapy over non-active controls on depressive symptoms from pre to post. However, as not all studies examined remission status, the pattern of findings may relate to the small sample size or other methodological differences between the small number of studies in this analysis. Further the rates of remission in the non-active control were relatively high. Comparisons of remission status at follow-up or for other reporters could not be made. Therefore, whether there are additional benefits of psychotherapy on remission of anxiety and depression over non-active controls is unclear especially at follow-up. The lack of remission data reported in studies is likely due to the challenges of conducting diagnostic interviews in routine settings, especially following discharge from services. It is important to note that while the remission rates for psychotherapy were seldom significantly better than remission rates of the non-active control, the remission rates in both conditions were reasonable with 40–60% of cases remitted. As such, this lends support to the findings noted above that the reason for the smaller effect size benefits of psychotherapy over non-active controls seen in this review likely relate to the relative effectiveness of the non-active control which in the majority of cases was TAU that incorporated known evidence-based strategies. These findings suggest that future reviews need to evaluate TAU components separately.

Most studies were conducted in community settings or schools. Due to small sample sizes, only a few comparisons between setting type could be conducted. Apart from the finding that non-school settings were associated with larger effects than school settings for the treatment of depression according to child follow-up report, all other comparisons indicated no significant differences between setting type and this finding aligns with previous studies that have not found any significant differences in effects across settings (Eckshtain et al., 2020). This favourable outcome suggests that the benefits of psychotherapy for anxiety and depression are apparent across a range of implementation settings and when delivered by a wide range of allied health professionals, medical practitioners, teachers, and trained lay people. As such the benefits of psychotherapy were translatable not only to participants in routine settings, but also when delivered by the variety of professionals who work in these settings.

Across all analyses there was limited evidence that treatment described as CBT was significantly better than other forms of active psychotherapy. Although it is important to note the small sample sizes available, as seven of the studies compared two different types of CBT against each other (e.g. low intensity vs. regular CBT), and therefore could not be included in this analysis. There was one exception where CBT was associated with significant small effects (ES =  − 0.36) for depression with according to child report, from pre to follow-up, but nonsignificant effects from pre to post compared to other therapies. There were insufficient studies to examine differences based on parent or independent evaluator reports for depression. Similarly, anxiety studies did not have sufficient parent report data to examine differences between treatment conditions. The findings related to child and independent evaluator reports showed nonsignificant benefits of CBT over other psychotherapy at pre-post, and at follow-up. On one hand this is perhaps concerning as efficacy studies typically show some benefits of CBT over other psychotherapies (Eckshtain et al., 2020; Reynolds et al., 2012; Weisz et al., 2017), and suggest that CBT may lose its superiority in these settings. However, it is unclear if this effect is due to differences in the population, adherence to CBT protocols, or differences in the differences in the types of active therapies tested. Components of active treatment components was often vague, and delivered in a non-manualised and unmonitored format.

Naturally, the results of the meta-analysis are limited by the quality of the studies within it. Effectiveness trials within routine services are especially difficult to run within the parameters of careful scientific control. As a result, the studies in this review varied widely in methods, measures, and attention to CONSORT guidelines, likely adding to heterogeneity in the results. Therefore, the identified effects need to be interpreted with caution since it is likely that methodological differences and the impact of moderators may heavily influence outcomes. Unfortunately, the relatively small number of studies meant that most obvious moderators such as treatment duration or youth age, were difficult to evaluate. We included studies for the treatment of primary anxiety and mood disorders including PTSD and OCD. The sensitivity analysis indicated that the inclusion of studies for primary PTSD might have influenced the significant outcomes reported for child-reported anxiety and child-reported depression, but not for the other comparisons. It is not known if the inclusion of PTSD studies effected the results due to the increase in sample size, and therefore statistical power for detecting effects, or whether the effectiveness of interventions for PTSD performed differently to those for OCD, anxiety or mood disorders. It should be noted that this finding was only evidence for child-reported symptoms, and so it is unlikely that there were true differences in effectiveness between disorders. It is also important to note that outcomes for primary and comorbid symptoms were examined in combination such that outcomes for secondary disorders not necessarily targeted in treatment were included. This provides a broad understanding of the impacts of interventions, but also might have led to weaker overall effect sizes.

Overall, the results are promising. There were clear benefits of psychotherapy over non-active treatments for anxiety from pre to post across all informants when delivered by setting staff. Similarly, for depression, there were clear benefits of psychotherapy over non-active controls from pre-to-post-outcomes on child and parent report, although the effect for independent evaluator was not significant. Given that the majority of non-active controls were TAU that included evidence-based components this effect might be particularly notable. There was also some suggestion that effects on depression were sustained from pre to follow-up according to child report, with outcomes on the other reporters not tested. Despite the promising findings, more research is needed to understand the true effectiveness of psychotherapy for internalising disorders. Only a small number of studies examined longer-term follow-up or remission. Very few studies compared CBT to another active intervention, and so our understanding of the benefits of different types of psychotherapies in these settings are limited. For anxiety, the benefits of CBT were not significantly different from active control, although the number of studies in the analysis was small and so caution is needed in interpreting the results. For depression, very few studies were available such that comparisons could only be made for child report, which was associated with significant effect (− 0.33) favouring CBT. In conclusion, there is some evidence for the effectiveness of psychotherapy for internalising disorders when delivered in routine settings over non-active controls that predominantly included TAU comparisons, particularly for pre-to post-outcomes, but insufficient evidence for longer-term outcomes, or remission status.