Cognitive-behavioral therapy versus other therapies: Redux

https://doi.org/10.1016/j.cpr.2013.01.004Get rights and content

Abstract

Despite the evidence suggesting that all treatments intended to be therapeutic are equally efficacious, the conjecture that one form of treatment, namely cognitive-behavioral therapy (CBT), is superior to all other treatment persists. The purpose of the current study was to (a) reanalyze the clinical trials from an earlier meta-analysis that compared CBT to ‘other therapies’ for depression and anxiety (viz., Tolin, 2010) and (b) conduct a methodologically rigorous and comprehensive meta-analysis to determine the relative efficacy of CBT and bona fide non-CBT treatments for adult anxiety disorders. Although the reanalysis was consistent with the earlier meta-analysis' findings of small to medium effect sizes for disorder-specific symptom measures, the reanalysis revealed no evidence for the superiority of CBT for depression and anxiety for outcomes that were not disorder-specific. Following the reanalysis, a comprehensive anxiety meta-analysis that utilized a survey of 91 CBT experts from the Association of Behavioral and Cognitive Therapists (ABCT) to consensually identify CBT treatments was conducted. Thirteen clinical trials met the inclusion criteria. There were no differences between CBT treatments and bona fide non-CBT treatments across disorder-specific and non-disorder specific symptom measures. These analyses, in combination with previous meta-analytic findings, fail to provide corroborative evidence for the conjecture that CBT is superior to bona fide non-CBT treatments.

Highlights

► Reanalyzed studies from an earlier meta-analysis (viz., Tolin, 2010) of CBT treatments. ► Meta-analyzed the relative efficacy of CBT and non-CBT treatments for adult anxiety. ► No differences between CBT and non-CBT for non-targeted measures in the re-analysis. ► CBT experts were surveyed to consensually identify treatments as CBT or non-CBT. ► No differences between CBT and bona fide non-CBT treatments for anxiety disorders.

Introduction

Since the origin of psychotherapy, there have been disagreements about whether some psychological treatments are more efficacious than others. Advocates of particular treatments have claimed, often without much evidence, that “their” treatment is undeniably superior to others. As Norcross and Newman (1992) noted two decades ago:

Rivalry among theoretical orientations has a long and undistinguished history in psychotherapy, dating back to Freud. In the infancy of the field, therapy systems, like battling siblings, competed for attention and affection in a “dogma eat dogma” environment… Mutual antipathy and exchange of puerile insults between the adherents of rival orientations were much the order of the day. (p. 3)

In the past several decades, research evidence from randomized controlled trials (RCTs) has been used in an effort to identify treatments that are superior to others. Typically, the results from RCTs are aggregated using meta-analysis to determine whether there is sufficient evidence to reject the null hypothesis that there are no differences between two or more treatments and declare one treatment superior to others. Unfortunately, treatment differences suggested by the research evidence remain inconclusive.

While the advocates of particular treatments seek to find evidence supporting their preferred approach, there is a competing conjecture that the therapeutic factors of psychotherapy are derived from procedures common to all treatments. Such common factors are hypothesized to include a cogent explanation that is acceptable to the patient as well as a set of treatment procedures consistent with that explanation, which together lead to healthier actions by the patient (Frank and Frank, 1991, Wampold, 2001, Wampold, 2007, Wampold et al., 2006). According to this hypothesis, known as the dodo bird conjecture, treatments that are intended to be therapeutic are equally efficacious (Luborsky et al., 1975, Rosenzweig, 1936, Wampold, Mondin, Moody, Stich, et al., 1997).

Early meta-analyses tended to show that behavioral and cognitive-behavioral treatments (CBT) were superior to other treatments (e.g., Shapiro and Shapiro, 1982, Smith and Glass, 1977, Smith et al., 1980). However, there were two perspicuous threats to the validity of this conclusion. First, the effects for different treatments were derived from comparisons to inactive control conditions rather than direct comparisons of treatments, which introduced the possibility that the differences were due to variations in study characteristics such as the reactivity of study measures (Shadish & Sweeney, 1991). Indeed, Smith and Glass (1977) and Smith et al. (1980) found that the advantage for behavioral and cognitive-behavioral treatments disappeared when the reactivity of measurement was statistically controlled. Second, the comparison among classes of treatments, such as CBT and dynamic therapies, relied on assignment of treatments to classes based on essential treatment characteristics. However, classification of treatments has turned out to be ambiguous — the same treatment often gets classified differently depending on the criteria used to define classes and on idiosyncrasies of particular classification (see Ehlers et al., 2010, Wampold, 2001, Wampold, Mondin, Moody and Ahn, 1997).

To address these and several other issues, Wampold, Mondin, Moody, Stich, et al. (1997) examined only direct comparisons of treatments that were intended to be therapeutic regardless of the diagnosis and without the need to assign treatments to classes. They found that the distribution of effects was consistent with the hypothesis that there were no differences among treatments; that is, there was insufficient evidence to reject the omnibus null hypothesis that the true differences among treatment outcomes is zero. Unsurprisingly, this conclusion did not go unchallenged.

One perspicuous criticism of Wampold, Mondin, Moody, Stich, et al. (1997) was that it makes little sense to examine treatment differences across disorders (Crits-Christoph, 1997, DeRubeis et al., 2005). DeRubeis et al. (2005, p. 175) stated that examining relative efficacy blind to the type of disorder “is akin to asking whether insulin or an antibiotic is better, without knowing the condition for which these treatments are to be given…. Alternatively, researchers should begin with a problem and ask how treatments compare in their effectiveness for that problem.” Consequently, several replications of Wampold, Mondin, Moody, Stich, et al. were conducted for particular disorders, including post-traumatic stress disorder (Benish, Imel, & Wampold, 2008), substance use disorders (Imel, Wampold, Miller, & Fleming, 2008), and childhood disorders (Miller, Wampold, & Varhely, 2008), with essentially the same results.

A second criticism of Wampold, Mondin, Moody, Stich, et al. (1997) was that most of the comparisons were between two variations of CBT, nullifying any conclusions about the superiority of CBT and other forms of treatment (Crits-Christoph, 1997). However, the potency of this criticism was attenuated by the fact that it was based on a rather expansive definition of CBT. For example, emotion-focused therapy (EFT; Goldman & Greenberg, 1992), which assumed that “psychological symptoms are seen as emanating from the deprivation of unmet adult needs,” and involved, in part, “identification with previously unacknowledged aspects of experience by enactment of redefined cycle” was classified as CBT (see Wampold, Mondin, Moody, & Ahn, 1997, p. 964). As we will see, using an ambiguously defined class of treatments can lead to premature conclusions.

Despite the evidence suggesting that all treatments intended to be therapeutic are equally efficacious, there is a conjecture that one form of treatment, namely CBT, is superior to other treatments. To test this conjecture, meta-analysts have examined direct comparisons between treatments classified as CBT versus some other treatments. Many of such meta-analyses have failed to show that CBT is superior to any other treatment (e.g., Cuijpers et al., 2008, Spielmans et al., 2007). However, some other meta-analyses have shown CBT to be more effective than at least one other class of treatments. For example, Gloaguen, Cottraux, Cucherat, and Blackburn (1998) found that CBT for the treatment of depression was superior to a class of treatments called “verbal” therapies, although not to other types of treatments. However, verbal therapies contained treatments that were clearly intended to be control conditions in that they had no cogent theoretical rationale and therapists were aware that they were delivering a sham treatment (see Wampold, Mondin, Moody, Stich, et al., 1997). When these control conditions were removed from the “verbal” therapy class, CBT was not found to be superior to “verbal” therapies for the treatment of depression (Wampold, Minami, Baskin, & Tierney, 2002). However, it appears the issue of the relative efficacy of CBT is not yet settled.

To address this issue, Tolin (2010) recently conducted a meta-analysis to evaluate the relative efficacy of CBT across a variety of disorders. This meta-analysis addressed many of the issues discussed above by including only direct comparisons of CBT to other treatments intended to be therapeutic. Tolin found that across disorders, CBT was not superior to interpersonal therapy or supportive therapies, but was superior to dynamic therapies. Furthermore, CBT was found to be superior to other therapies only for depression, producing a small effect (viz., d = 0.21), and anxiety disorders, producing a moderate effect (d = 0.43). The results of this meta-analysis, which are summarized in part in Table 1, appear to support the conjecture that CBT is superior to other treatments in certain instances.

There are a number of issues in the Tolin meta-analysis that require further examination. The first issue is, in a sense, theoretical. Recall that the Wampold, Mondin, Moody, Stich, et al. (1997) meta-analysis was severely criticized for addressing a hypothesis without regard to disorder, yet this is exactly what Tolin examined: Is CBT more effective than alternate treatments? Tolin did, however, examine sets of disorders, and found that CBT was more effective for two classes of disorders (anxiety and depression) rather than for particular disorders (e.g., PTSD, MDD). This criticism remains because one specific anxiety disorder, such as panic disorder, may have a diathesis related to faulty cognitive processes, as proposed by Clark (1986), while another, obsessive–compulsive disorder, may not (see Anholt et al., 2008). Consequently, CBT may be more effective for the former than the latter. The problem of generalizing to all anxiety disorders is exacerbated by the fact that only four RCTs of anxiety disorders were included in Tolin's meta-analysis, and these four trials are quite dated (to wit, published in 1967, 1972, 1994, and 2001). However, there are more than four trials that have compared CBT to a bona fide non-CBT treatment for anxiety disorders (e.g., Benish et al., 2008, reviewed 10 such studies for PTSD alone1).

A second problem is that the conclusion that CBT is more efficacious than other treatments for depression contradicts several well-conducted meta-analyses, including Cuijpers et al. (2008) and Wampold et al. (2002). Again, in the area of depression, some prominent studies were not included in the Tolin (2010) meta-analysis; for example the Watson, Gordon, Stermac, Kalogerakos, and Steckley (2003) study, which compared CBT to EFT, was not included.

Perhaps the Tolin (2010) meta-analysis excluded the Watson et al. (2003) trial comparing CBT to EFT because EFT was classified as a CBT treatment in the same manner as Crits-Christoph (1997). This raises the issue of which treatments would and would not be included in the CBT rubric. According to the National Association of Cognitive-Behavioral Therapists (National Association of Cognitive-Behavioral Therapists, 2012), “Cognitive-behavioral therapy does not exist as a distinct therapeutic technique. The term cognitive-behavioral therapy (CBT) is a very general term for a classification of therapies with similarities” (“What is Cognitive-Behavioral Therapy?” para. 1). Similarly, the Association for Behavioral and Cognitive Therapies (Association of Behavioral and Cognitive Therapies, 2012), the foremost organization for the training and dissemination of CBT, avoids providing a definitive description of CBT and instead defines CBT as “the term used for a group of psychological treatments that are based on scientific evidence,” (About Psychological Treatment section, para. 1). Others, it seems, make finer distinctions: For example, Ehlers et al. (2010) made a distinction between CBT and stress management treatments (e.g., stress inoculation training). Tolin classified a treatment as CBT if it contained any of the following components: relaxation training (including progressive muscle relaxation, meditation, or breathing retraining), exposure therapy (imaginal or in vivo exposure, including flooding and implosive therapy), behavior rehearsal (behavioral training in social skills, habit reversal, or problem solving), cognitive restructuring (including direct strategies to identify and alter maladaptive thought processes), or operant procedures (systematic manipulation of reinforcers or punishers for behavior, including behavioral activation). Such a classification system condenses an exceedingly broad spectrum of treatments together into a single category (e.g., relaxation therapy and problem-solving therapy would both be classified as CBT). It also appears that Tolin classified eye-movement and reprocessing therapy (EMDR) as CBT since none of the many trials that have compared EMDR and CBT were included in his meta-analysis, a classification that is at odds with common practice in the PTSD literature of differentiating CBT and EMDR (e.g., Ehlers et al., 2010; see also Bisson and Andrew, 2007, Bisson et al., 2007).

Another limitation of Tolin's (2010) meta-analysis was that he considered only disorder-specific (i.e., targeted) symptom measures for depression and anxiety. Many treatments, for example psychodynamic treatments, focus more on global change:

The goals of psychodynamic therapy include, but extend beyond, alleviation of acute symptoms. Psychological health is not merely the absence of symptoms; it is the positive presence of inner capacities and resources that allow people to live life with a greater sense of freedom and possibility. (Shedler, 2010, p. 105)

It may well be that CBT is able to effect change in disorder-specific symptom measures vis-à-vis other treatments, but not show any superiority with regard to non-disorder specific (i.e., non-targeted or global) measures.

Meta-analyses, as with any method, can produce contradictory conclusions, depending on how the meta-analyses were conducted and operations adopted. Meta-analyses are particularly sensitive to the manner in which studies are selected, the coding of characteristics of the studies and the effects, and on which measures effects are based. As with primary studies, there is a need to replicate and extend meta-analytic findings, being careful to document the method. Accordingly, the purpose of the present study was to further examine the claim that CBT was superior to other treatments for various disorders. Specifically, we examined the conclusions made by Tolin (2010) with regard to depression and anxiety by (a) reanalyzing the studies reviewed by Tolin, but including analyses of non-disorder specific symptom measures as well as disorder-specific symptom measures, and (b) conducting a comprehensive meta-analysis of anxiety trials comparing CBT to another bona fide treatment. Vis-à-vis the anxiety meta-analysis, we were able to locate many more RCTs of CBT versus bona fide non-CBT treatments and avoided the classification problem by using a consensus of ABCT members about whether or not a treatment was classified as CBT.

Section snippets

Methods

Due to the aforementioned qualitative and quantitative limitations, a reanalysis of the studies included in Tolin's (2010) meta-analysis was conducted (k = 26; see online Appendix A). Only studies in the original meta-analysis were included (see the online Appendix C for references). Seven graduate students in counseling psychology (six doctoral students, one master's student) coded the study data in order to compute effect sizes. Two graduate students coded each study with disagreements reviewed

Method

The main purpose of this study was to evaluate the relative efficacy of bona fide CBT treatments to bona fide non-CBT treatments. Several methodological steps were performed in order to conduct the final meta-analysis. First, published studies from randomized clinical trials were selected through an a priori set of inclusion criteria. Second, trained raters evaluated the bona fide status of psychotherapies from the initial pool of relevant randomized clinical trials. Third, the treatment titles

General conclusions

As discussed by Lakatos (Lakatos, 1970, Lakatos, 1976, Larvor, 1998), science and mathematics involve two processes. First, taxons or concepts are established, developed, and/or defined. For example, Newtonian physics posits a force called gravity; Mathematics has the class of things defined as polyhedra. Second, formal relationships among the taxons or concepts are conjectured. For gravity, Newton stated that the gravitational force is proportional to the product of the two masses and

References (62)

  • Association of Behavioral and Cognitive Therapies

    What is cognitive behavior therapy?

  • A.T. Beck et al.

    Anxiety disorders and phobias: A cognitive perspective

    (1985)
  • J. Bisson et al.

    Psychological treatment of post-traumatic stress disorder (PTSD)

  • J.I. Bisson et al.

    Psychological treatments for chronic post-traumatic stress disorder. Systematic review and meta-analysis

    The British Journal of Psychiatry

    (2007)
  • T.D. Borkovec

    Control groups and comparison groups in psychotherapy outcome research

    National Institute on Drug Abuse Research Monograph

    (1990)
  • K.M. Carroll et al.

    Efficacy of disulfiram and cognitive behavior therapy in cocaine-dependent outpatients: A randomized placebo-controlled trial

    Archives of General Psychiatry

    (2004)
  • J. Cohen

    Statistical power analysis for the behavioral sciences

    (1988)
  • P. Crits-Christoph

    Limitations of the dodo bird verdict and the role of clinical trials in psychotherapy research: Comment on Wampold et al. (1997)

    Psychological Bulletin

    (1997)
  • P. Cuijpers et al.

    Psychotherapy for depression in adults: A meta-analysis of comparative outcome studies

    Journal of Consulting and Clinical Psychology

    (2008)
  • A.C. Del Re

    RcmdrPlugin.MAd: Meta-Analysis with Mean Differences (MAd) Rcmdr Plug-in.R package version 1.0.9

  • A.C. Del Re et al.

    MAd: Meta-analysis with mean differences. R package version 0.8

  • R.J. DeRubeis et al.

    A conceptual and methodological analysis of the nonspecifics argument

    Clinical Psychology: Science and Practice

    (2005)
  • R. Development Core Team

    R: A language and environment for statistical computing

    (2010)
  • J.D. Frank et al.

    Persuasion and healing: A comparative study of psychotherapy

    (1991)
  • A. Goldman et al.

    Comparison of integrated systemic and emotionally focused approaches to couples therapy

    Journal of Consulting and Clinical Psychology

    (1992)
  • L.V. Hedges et al.

    Statistical methods for meta-analysis

    (1985)
  • W. Hiller et al.

    Defining response and remission in psychotherapy research: A comparison of the RCI and the method of percent improvement

    Psychotherapy Research

    (2012)
  • Z.E. Imel et al.

    Distinctions without a difference: Direct comparisons of psychotherapies for alcohol use disorders

    Psychology of Addictive Behaviors

    (2008)
  • A.E. Kazdin

    Understanding how and why psychotherapy leads to change

    Psychotherapy Research

    (2009)
  • T.S. Kuhn

    Logic of discovery or psychology of research

  • Cited by (95)

    • Efficacy and acceptability of interventions for co-occurring PTSD and SUD: A meta-analysis

      2021, Journal of Anxiety Disorders
      Citation Excerpt :

      Although the largest review to date, as noted above, it is likely that some tests (e.g., moderators, between-group models) were underpowered. This is a particularly salient concern for the between-group models in which small effects are to be expected (Baardseth et al., 2013; Goldberg et al., 2018; Wampold & Imel, 2015). The high degree of heterogeneity in many within- and between-group models further reduced statistical power and highlights the tenuous nature of some effect size estimates.

    View all citing articles on Scopus

    The views expressed are those of the authors and do not necessarily represent those of the Department of Veterans Affairs.

    View full text