Abstract
This paper proposes an intuitive yet statistical advancement of the benchmarking method (e.g., Weersing and Weisz, 2002, Journal of Consulting and Clinical Psychology 70: 299–310) that could facilitate the assessment of pre-post treatment effectiveness of psychotherapy and other interventions delivered in clinical settings against efficacy observed in clinical trials. Primary development was in the use of the “good-enough principle” (Serlin and Lapsley, 1985 American Psychologist 40: 73–13, 1993, In: G. Keren & C. Lewis (eds.), A handbook for Data Analysis in Behavioral Sciences: Methodological Issues. Hillsdale, NJ: Lawrence Erlbaum Associated, pp. 199–228), which allowed for setting a clinically relevant margin between the benchmarks and the effect sizes observed in clinical settings so as to avoid attaining statistical significance with clinically trivial differences. Examples are given using clinical trials benchmarks of adult depression treatment, followed by instructions and limitations for its use.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Addis M.E. (2002). Methods for disseminating research products and increasing evidence-based practice: Promises, obstacles, and future directions. Clinical Psychology: Science and Practice 9: 367–378
Barlow D.H., Craske M.G. (1994). Mastery of Your Anxiety and Panic II. Albany, NY, Graywind
Barlow D.H., Craske M.G., Cerny J.A., Klosko J.S. (1989). Behavioral treatment of panic disorder. Behavior Therapy 20: 261–282
Beck A.T., Steer R.A. (1987). Beck Depression Inventory Manual. San Antonio, TX, Harcourt Brace Jovanovich
Becker B.J. (1988). Synthesizing standardized mean-change measures. British Journal of Mathematical and Statistical Psychology 41: 257–278
Chorpita B.F., Yim L.M., Donkervoet J.C., Arensdorf A., Amundsen M.J., McGee C. et al. (2002). Toward large-scale implementation of empirically supported treatments for children: A review and observations by the Hawaii empirical basis to task force. Clinical Psychology: Science and Practice 9: 165–190
Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd edn. Hillsdale NJ, Erlbaum
Derigatis L.R. (1983). SCL-90-R: Administration, scoring, and procedural manual-II. Baltimore, MD, Clinical Psychometric Research
Gleser L.J., Olkin I. (1994). Stochastically dependent effect sizes. In: Cooper H., Hedges L.V. (eds), The Handbook of Research Synthesis. New York, Russel Sage, pp. 339–355
Goldfried M., Wolfe B. (1998). Toward a clinically valid approach to therapy research. Journal of Consulting and Clinical Psychology 66: 143–150
Hamilton M.A. (1960). A rating scale for depression. Journal of Neurology, Neurosurgery, and Psychiatry 23: 56–62
Hedges L.V., Olkin I. (1985). Statistical Methods for Meta-Analysis. San Diego, CA, Academic Press
Lambert M.J., Hatch D.R., Kingston M.D., Edwards B.C. (1986). Zung, Beck, and Hamilton Rating Scales as measures of treatment outcome: A meta-analytic comparison. Journal of Consulting and Clinical Psychology 54: 54–59
Merrill K.A., Tolbert V.E., Wade W.A. (2003). Effectiveness of cognitive therapy for depression in a community mental health center: A benchmarking study. Journal of Consulting and Clinical Psychology 71: 404–409
Minami T., Wampold B.E., Serlin R.C., Kircher J.C., & Brown G.S. (in press). Benchmarks for psychotherapy efficacy in adult major depression. Journal of Consulting and Clinical Psychology.
Rounsaville B.J., O’Malley, S., Foley S., Weissman M.M. (1988). Role of manual-guided training in the conduct and efficacy of interpersonal psychotherapy for depression. Journal of Consulting and Clinical Psychology 56: 681–688
Rupert P.A., Baird K.A. (2004). Managed care and the independent practice of psychology. Professional Psychology: Research and Practice 35: 185–193
Seligman M.E.P. (1995). The effectiveness of psychotherapy: The Consumer Reports Study. American Psychologist 50: 965–974
Serlin R.C., Lapsley D.K. (1985). Rationality in psychological research: The good-enough principle. American Psychologist 40: 73–83
Serlin R.C., Lapsley D.K. (1993). Rational appraisal of psychological research and the good-enough principle. In: Keren G., Lewis C., (eds), A Handbook for Data in the Behavioral Sciences: Methodological Issues. Hillsdale, NJ, Lawrence, pp. 199–228
Shadish W.R., Matt G.E., Navarro A.M., Phillips G. (2000). The effects of psychological therapies under clinically representative conditions: A meta-analysis. Psychological Bulletin 126: 512–529
Telch M.J., Lucas J.A., Schmidt N.B., Hanna H.H., Jaimez T.L., Lucas R.A. (1993). Group cognitive-behavioral treatment of panic disorder. Behaviour Research and Therapy 31: 279–287
Wade W.A., Treat T.A., Stuart G.L. (1998). Transporting an empirically supported treatment for panic disorder to a service clinic setting: A benchmarking strategy. Journal of Consulting and Clinical Psychology 66: 231–239
Weersing V.R., Weisz J.R. (2002). Community clinic treatment of depressed youth: Benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology 70: 299–310
Westen D., Novotny C.M., Thompson-Brenner H. (2004). The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controllled clinical trials. Psychological Bulletin 130: 631–663
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Minami, T., Serlin, R.C., Wampold, B.E. et al. Using Clinical Trials to Benchmark Effects Produced in Clinical Practice. Qual Quant 42, 513–525 (2008). https://doi.org/10.1007/s11135-006-9057-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11135-006-9057-z