Is cognitive–behavioral therapy more effective than other therapies?: A meta-analytic review

https://doi.org/10.1016/j.cpr.2010.05.003Get rights and content

Abstract

Cognitive–behavioral therapy (CBT) is effective for a range of psychiatric disorders. However, it remains unclear whether CBT is superior to other forms of psychotherapy, and previous quantitative reviews on this topic are difficult to interpret. The aim of the present quantitative review was to determine whether CBT yields superior outcomes to alternative forms of psychotherapy, and to examine the relationship between differential outcome and study-specific variables. From a computerized literature search through September 2007 and references from previous reviews, English-language articles were selected that described randomized controlled trials of CBT vs. another form of psychotherapy. Of these, only those in which the CBT and alternative therapy condition were judged to be bona fide treatments, rather than “intent-to-fail” conditions, were retained for analysis (28 articles representing 26 studies, N = 1981). Four raters identified post-treatment and follow-up effect size estimates, as well as study-specific variables including (but not limited to) type of CBT and other psychotherapy, sample diagnosis, type of outcome measure used, and age group. Studies were rated for methodological adequacy including (but not limited to) the use of reliable and valid measures and independent evaluators. Researcher allegiance was determined by contacting the principal investigators of the source articles. CBT was superior to psychodynamic therapy, although not interpersonal or supportive therapies, at post-treatment and at follow-up. Methodological strength of studies was not associated with larger or smaller differences between CBT and other therapies. Researchers' self-reported allegiance was positively correlated with the strength of CBT's superiority; however, when controlling for allegiance ratings, CBT was still associated with a significant advantage. The superiority of CBT over alternative therapies was evident only among patients with anxiety or depressive disorders. These results argue against previous claims of treatment equivalence and suggest that CBT should be considered a first-line psychosocial treatment of choice, at least for patients with anxiety and depressive disorders.

Introduction

The diversity of psychotherapeutic approaches in clinical practice (Garfield and Bergin, 1986, Goisman et al., 1999), as well as outcome data that would appear to support many of these approaches (Butler et al., 2006, de Mello et al., 2005, Dobson, 1989, Driessen et al., 2010, Hofmann and Smits, 2008, Leichsenring and Rabung, 2008), unavoidably leads to controversy regarding which form of psychotherapy is most effective. Some authors, for example, have argued that cognitive–behavioral therapy (CBT) is superior to alternative psychotherapies (Eysenck, 1994, Hunsley and Di Giulio, 2002). This argument is bolstered by the results of two limited-scope meta-analyses: Shapiro and Shapiro (1982) examined 143 comparative outcome studies published over a 5-year period and found an effect size (Cohen's d) of 1.06 for behavioral treatments vs. 0.40 for dynamic/humanistic treatments. Shadish, Matt, Navarro and Phillips (2000) sampled 90 studies conducted under “clinically representative” conditions; regression analyses indicated that behavioral orientation was a significant predictor of effect size.

On the other hand, some meta-analysts have failed to find evidence for the superiority of one form of psychotherapy over another. The collective argument for psychotherapy equivalence has been commonly termed the “Dodo Bird Verdict” (Luborsky, Singer, & Luborsky, 1975), after the character in Alice in Wonderland who proclaimed after a chaotic race that “Everyone has won, and all must have prizes.” The seminal meta-analysis by Smith and Glass (1977) revealed few differences between behavior therapy and other forms of psychotherapy: effect size estimates (Cohen's d) were 0.8 for behavior therapy and 0.6 for all other therapies combined. This meta-analysis has been criticized in several respects, including the categorization of cognitive interventions as “non-behavioral” therapies, despite their common co-utilization in practice (Hunsley and Di Giulio, 2002, Wilson and Rachman, 1983); failure to include many well-controlled studies of behavior therapy (Rachman & Wilson, 1980); and absence of correction for methodologically weak studies (Wilson & Rachman, 1983). Subsequent re-analyses of Smith and Glass's data, correcting for these concerns, have suggested superior outcomes for behavioral therapies (Andrews and Harvey, 1981, Hunsley and Di Giulio, 2002).

In what has perhaps become the de facto last word on the matter, Wampold et al. (1997) used a novel approach to meta-analysis in which the primary outcome measure was heterogeneity of effect sizes across all forms of psychotherapy. Effect sizes were homogeneous around 0, leading to a conclusion of general equivalency across forms of psychotherapy. Criticisms of the Wampold et al. meta-analysis include the use of homogeneity testing, rather than tests of mean effect size differences (Howard, Krause, Saunders & Kopta, 1997); absence of attempt to discriminate among different kinds of outcome measures, different forms of psychotherapy, or different patient populations (Crits-Christoph, 1997, Howard et al., 1997); and the fact that the majority [69% (Crits-Christoph, 1997) to 80% (Hunsley & Di Giulio, 2002)] of selected studies were pre-clinical studies, usually of college students, in which one variant of CBT was compared to another variant of CBT.

The question of differential therapeutic efficacy is, therefore, far from settled. As evidence for the efficacy of CBT continues to accumulate and as cognitive–behavioral interventions become more widespread, there is a need to understand more clearly how CBT compares to other forms of psychotherapy. Ideally, such research would follow the prescription of Gordon Paul (1967), who recommended over 40 years ago that “…the question towards which all outcome research should ultimately be directed is the following: What treatment, by whom, is most effective for this individual with that specific problem, and under which set of circumstances?” (p. 111). Although no single study could ever hope to answer so complicated a question [Beutler (1991) estimated that there would be nearly 1.5 million potential combinations of therapy, therapist, and patient types], the aim of the present study is to capture the spirit of Paul's question using meta-analytic strategies. First, the treatments being used need to be described clearly. The present study examines direct comparisons of CBT vs. other forms of psychotherapy, with specific attention to what kind of CBT is being compared to what kind of alternative psychotherapy. Second, the present study focuses on who is being treated: each study is coded according to the target diagnosis or presenting problem, how the sample was selected (e.g., from a clinic vs. from a pool of student volunteers), and the age range of the patients. Third, the present analyses will attend to how clinical outcome is defined: for example, is outcome defined as a reduction in primary symptoms (such as reduced depression severity in a study of depressed patients), global severity (such as clinician's ratings of overall improvement), or improvement in functional measures (such as quality of life, work productivity, or social adjustment)? Fourth, the present study examines the impact of study quality and methodological adequacy according to criteria proposed by Jadad et al., 1996, Foa and Meadows, 1997. Fifth, the present study includes only comparisons of bona fide psychotherapies, which are distinguished from “intent-to-fail” treatments commonly used to control for nonspecific treatment effects; bona fide treatments were identified using criteria proposed by Wampold et al., 1997, Westen et al., 2004. Finally, the present analysis accounts for who conducted the original source studies. In particular, the impact of researcher allegiance to one school of psychotherapy or another is examined. Researcher allegiance is attracting increased attention as a possible complicating factor in comparative outcome studies; previous meta-analyses have suggested that measures of researcher allegiance can account for more than half of the variance in observed differences in outcomes (Gaffan et al., 1995, Luborsky et al., 1999). The presence of researcher allegiance does not necessarily imply bias, as others have noted (Hollon, 1999, Leykin and DeRubeis, 2009); nevertheless, it is included here as a possible moderator of study outcome. Previous researchers have attempted to quantify researcher allegiance indirectly by examining articles for references to previous published research showing superiority of one treatment, a specific hypothesis or rationale as to why one treatment should be superior, a detailed description of a treatment's procedure and aims, treatments devised or first introduced by one of the authors, or only one treatment being included in the study (Gaffan et al., 1995). These criteria seem inadequate to assess a researcher's personal allegiances, as many of them may simply reflect good science. Instead, the present study includes allegiance ratings collected directly from the principal investigators of each study.

Section snippets

Data sources

Journal articles were identified through searches of the Medline and PsycINFO electronic databases through September 2007 using the search terms [(Behavior Therapy or Cognitive Therapy or Cognitive Behavior Therapy or Rational Emotive Behavior Therapy or CBT or Exposure Therapy or Behavior Modification or Skill Learning or Dialectical or Cognitive Restructuring or Cognitive Behavioral Therapy or Schema Therapy) and (Dynamic or Psychodynamic or Psychoanalysis or Interpersonal Psychotherapy or

Comparisons between CBT and other therapies

Overall, CBT was associated with lower scores on measures of primary symptoms at post-treatment than were alternative treatments (see Table 3). Comparison across different types of alternative treatment did not indicate significant heterogeneity (Qbetween = 3.05, p = 0.38). Examination of Table 3 indicates that CBT proved significantly more effective than psychodynamic therapy, but not interpersonal, supportive, or “other” therapy. None of the comparisons were robust against the file drawer effect.

Discussion

Several limitations of the present analyses should be acknowledged. It is important to recognize that only pairwise comparisons between CBT and other therapies were conducted. The present results cannot, therefore, be used to infer similarities or differences among the alternative therapies (e.g., between interpersonal vs. psychodynamic therapy) or among variants of CBT (e.g., between exposure therapy vs. cognitive restructuring). Although the overall number of studies is reasonable (26 studies

Acknowledgements

The author thanks Drs. Shawn Cahill, Gretchen Diefenbach, and Fiona Kehoe for their helpful comments on a previous draft of this manuscript.

Dr. Tolin receives research support from the National Institutes of Health, Organon/Schering-Plough, and Indevus Pharmaceuticals. He has previously received research support from Eli Lilly and Company, Pfizer, and Solvay. Dr. Tolin had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the

References (69)

  • L.E. Beutler

    Have all won and must all have prizes? Revisiting Luborsky et al'.s verdict

    Journal of Consulting and Clinical Psychology

    (1991)
  • L.E. Beutler et al.

    Predictors of differential response to cognitive, experiential, and self-directed psychotherapeutic procedures

    Journal of Consulting and Clinical Psychology

    (1991)
  • K.L. Bleiberg et al.

    Interpersonal psychotherapy for depression

  • M. Borenstein et al.

    Introduction to meta-analysis

    (2009)
  • K.M. Carroll et al.

    A comparative trial of psychotherapies for ambulatory cocaine abusers: Relapse prevention and interpersonal psychotherapy

    American Journal of Drug and Alcohol Abuse

    (1991)
  • K.M. Carroll et al.

    Efficacy of disulfiram and cognitive behavior therapy in cocaine-dependent outpatients: A randomized placebo-controlled trial

    Archives of General Psychiatry

    (2004)
  • J. Cohen

    Statistical power analysis for the behavioral sciences

    (1988)
  • P. Crits-Christoph

    Limitations of the Dodo Bird verdict and the role of clinical trials in psychotherapy research: Comment on Wampold et al. (1997)

    Psychological Bulletin

    (1997)
  • M.F. de Mello et al.

    A systematic review of research findings on the efficacy of interpersonal therapy for depressive disorders

    European Archives of Psychiatry and Clinical Neuroscience

    (2005)
  • K.S. Dobson

    A meta-analysis of the efficacy of cognitive therapy for depression

    Journal of Consulting and Clinical Psychology

    (1989)
  • R.C. Durham et al.

    Cognitive therapy, analytic psychotherapy and anxiety management training for generalised anxiety disorder

    British Journal of Psychiatry

    (1994)
  • R.C. Durham et al.

    One year follow-up of cognitive therapy, analytic psychotherapy and anxiety management training for generalized anxiety disorder: Symptom change, medication usage and attitudes to treatment

    Behavioural and Cognitive Psychotherapy

    (1999)
  • R.C. Durham et al.

    Tayside-Fife clinical trial of cognitive–behavioural therapy for medication-resistant psychotic symptoms. Results to 3-month follow-up

    British Journal of Psychiatry

    (2003)
  • I. Elkin et al.

    National Institute of Mental Health Treatment of Depression Collaborative Research Program. General effectiveness of treatments

    Archives of General Psychiatry

    (1989)
  • P.M. Emmelkamp et al.

    Comparison of brief dynamic and cognitive–behavioural therapies in avoidant personality disorder

    British Journal of Psychiatry

    (2006)
  • E.B. Foa et al.

    Psychosocial treatments for posttraumatic stress disorder: A critical review

    Annual Review of Psychology

    (1997)
  • E.A. Gaffan et al.

    Researcher allegiance and meta-analysis: The case of cognitive therapy for depression

    Journal of Consulting and Clinical Psychology

    (1995)
  • D. Gallagher-Thompson et al.

    Comparative effects of cognitive–behavioral and brief psychodynamic psychotherapies for depressed family caregivers

    Journal of Consulting and Clinical Psychology

    (1994)
  • M.G. Gelder et al.

    Desensitization and psychotherapy in the treatment of phobic states: A controlled inquiry

    British Journal of Psychiatry

    (1967)
  • J. Giesen-Bloo et al.

    Outpatient psychotherapy for borderline personality disorder: Randomized trial of schema-focused therapy vs transference-focused psychotherapy

    Archives of General Psychiatry

    (2006)
  • G.V. Glass et al.

    Meta-analysis in social research

    (1981)
  • R.M. Goisman et al.

    Psychosocial treatment prescriptions for generalized anxiety disorder, panic disorder, and social phobia, 1991–1996

    American Journal of Psychiatry

    (1999)
  • M.R. Goldfried et al.

    Clinical behavior therapy

    (1994)
  • Cited by (424)

    View all citing articles on Scopus
    View full text