Skip to main content
SpringerLink
Log in

Building an Evidence Base for Effective Supervision Practices: An Analogue Experiment of Supervision to Increase EBT Fidelity

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Treatments that are efficacious in research trials perform less well under routine conditions; differences in supervision may be one contributing factor. This study compared the effect of supervision using active learning techniques (e.g. role play, corrective feedback) versus “supervision as usual” on therapist cognitive restructuring fidelity, overall CBT competence, and CBT expertise. Forty therapist trainees attended a training workshop and were randomized to supervision condition. Outcomes were assessed using behavioral rehearsals pre- and immediately post-training, and after three supervision meetings. EBT knowledge, attitudes, and fidelity improved for all participants post-training, but only the SUP+ group demonstrated improvement following supervision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Accurso, E. C., Taylor, R. M., & Garland, A. F. (2011). Evidence-based practices addressed in community-based children’s mental health clinical supervision. Training and Education in Professional Psychology, 5(2), 88–96. doi:10.1037/a0023537.

    Article  PubMed  PubMed Central  Google Scholar 

  • American Board of Examiners in Clinical Social Work (2004, October). Clinical supervision: A practice specialty of clinical social work. https://www.abecsw.org/images/ABESUPERV2205ed406.pdf. Accessed Retrieved 9 Apr 2015.

  • Association for Counselor Education and Supervision Taskforce on Best Practices in Clinical Supervision (2011, April). Best practices in clinical supervision. http://www.acesonline.net/wp-content/uploads/2011/10/ACES-Best-Practices-in-clinicalsupervisiondocument-FINAL.pdf. Accessed 8 Apr 2015.

  • Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi:10.18637/jss.v067.i01.

    Article  Google Scholar 

  • Bearman, S.K., Herren, J., & Weisz, J.R. (2012). Therapist adherence to evidence based intervention—observational coding manual. Unpublished manual.

  • Bearman, S. K., Wadkins, M., Bailin, A., & Doctoroff, G. (2015). Pre-practicum training in professional psychology to close the research-practice gap: Changing attitudes toward evidence-based practice. Training and Education in Professional Psychology, 9(1), 13–20. doi:10.1037/tep0000052.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bearman, S. K., Weisz, J. R., Chorpita, B., Hoagwood, K., Ward, A., Ugueto, A., & Bernstein, A. (2013). More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Administration and Policy In Mental Health, 40(6), 518–529. doi:10.1007/s10488-013-0485-5.

    Article  PubMed  PubMed Central  Google Scholar 

  • Beidas, R. S., Cross, W., & Dorsey, S. (2014). Show me, don’t tell me: Behavioral rehearsal as a training and analogue fidelity tool. Cognitive and Behavioral Practice, 21(1), 1–11. doi:10.1016/j.cbpra.2013.04.002.

    Article  PubMed  Google Scholar 

  • Beidas, R. S., Edmunds, J. M., Marcus, S. C., & Kendall, P. C. (2012). Training and consultation to promote implementation of an empirically supported treatment: A randomized trial. Psychiatric Services, 63(7), 660–665.

    Article  PubMed  PubMed Central  Google Scholar 

  • Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1–30. doi:10.1111/j.1468-2850.2009.01187.x.

    Google Scholar 

  • Bennett-Levy, J. (2006). Therapist skills: A cognitive model of their acquisition and refinement. Behavioural and Cognitive Psychotherapy, 34(1), 57–78. doi:10.1017/S1352465805002420.

    Article  Google Scholar 

  • Bennett-Levy, J., McManus, F., Westling, B. E., & Fennell, M. (2009). Acquiring and refining CBT skills and competencies: Which training methods are perceived to be most effective? Behavioural and Cognitive Psychotherapy, 37(5), 571–583. doi:10.1017/S1352465809990270.

    Article  PubMed  Google Scholar 

  • Borntrager, C. F., Chorpita, B. F., Higa-McMillan, C., & Weisz, J. R. (2009). Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals? Psychiatric Services, 60(5), 677–681. doi:10.1176/appi.ps.60.5.677.

    Article  PubMed  Google Scholar 

  • Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18. doi:10.1037/0022-006X.66.1.7.

    Article  CAS  PubMed  Google Scholar 

  • Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77(3), 566–579. doi:10.1037/a0014565.

    Article  PubMed  Google Scholar 

  • Chorpita, B.F., Daleiden, E.L., Ebesutani, C., Young, J., Becker, K.D., Nakamura, B.J.,… Starace, N. (2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18(2), 154–172. doi:10.1111/j.1468-2850.2011.01247.x.

  • Chorpita, B. F., Park, A., Tsai, K., Korathu-Larson, P., Higa-McMillan, C. K., Nakamura, B. J., et al. (2015). Balancing effectiveness with responsiveness: therapist satisfaction across different treatment designs in the child STEPs randomized effectiveness trial. Journal of Consulting and Clinical Psychology,. doi:10.1037/a0039301.

    Google Scholar 

  • Chorpita, B. F., & Weisz, J. R. (2009). MATCH-ADTC: Modular approach to therapy for children with anxiety, depression, trauma, or conduct problems. Satellite Beach, Florida: PracticeWise.

    Google Scholar 

  • Cicchetti, D. V., & Sparrow, S. A. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86(2), 127–137.

    CAS  PubMed  Google Scholar 

  • Cook, J. M., Schnurr, P. P., Biyanova, T., & Coyne, J. C. (2009). Apples don’t fall far from the tree: Influences on psychotherapists’ adoption and sustained use of new therapies. Psychiatric Services, 60(5), 671–676. doi:10.1176/appi.ps.60.5.671.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cross, W. F., Seaburn, D., Gibbs, D., Schmeelk-Cone, K., While, A. M., & Caine, E. D. (2011). Does practice make perfect? A randomized control trial of behavioral rehearsal on suicide prevention gatekeeper skills. Journal of Primary Prevention, 32(3–4), 195–211. doi:10.1007/s10935-011-0250-z.

    Article  PubMed  PubMed Central  Google Scholar 

  • Dimeff, L. A., Koerner, K., Woodcock, E. A., Beadnell, B., Brown, M. Z., Skutch, J. M., et al. (2009). Which training method works best? A randomized controlled trial comparing three methods of training clinicians in dialectical behavior therapy skills. Behaviour Research and Therapy, 47(11), 921–930. doi:10.1016/j.brat.2009.07.011.

    Article  PubMed  Google Scholar 

  • Edmunds, J. M., Kendall, P. C., Ringle, V. A., Read, K. L., Brodman, D. M., Pimental, S. S., & Beidas, R. S. (2013). An examination of behavioral rehearsal during consultation as a predictor of training outcomes. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 456–466. doi:10.1007/s10488-013-0490-8.

    Article  PubMed  Google Scholar 

  • Ellis, A. (2009). Cognitive restructuring of the disputing of irrational beliefs. In W. T. O’Donohue, J. E. Fisher, W. T. O’Donohue, & J. E. Fisher (Eds.), General principles and empirically supported techniques of cognitive behavior therapy (pp. 189–193). Hoboken, NJ, US: Wiley.

    Google Scholar 

  • Falender, C. A., Erickson Cornish, J. A., Goodyear, R., Hatcher, R., Kaslow, N. J., Leventhal, G., et al. (2004). Defining competencies in psychology supervision: A consensus statement. Journal of Clinical Psychology, 60(7), 771–785. doi:10.1002/jclp.20013.

    Article  PubMed  Google Scholar 

  • Fouad, N. A., Grus, C. L., Hatcher, R. L., Kaslow, N. J., Hutchings, P. S., Madson, M. B., et al. (2009). Competency benchmarks: A model for understanding and measuring competence in professional psychology across training levels. Training and Education in Professional Psychology, 3(4,Suppl), S5–S26. 10.1037/a0015832.

    Article  Google Scholar 

  • Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration And Policy In Mental Health And Mental Health Services Research, 40(1), 6–22. doi:10.1007/s10488-012-0450-8.

    Article  PubMed  PubMed Central  Google Scholar 

  • Henggeler, S. W., Schoenwald, S. K., Liao, J. G., Letourneu, E. J., & Edwards, D. L. (2002). Transporting efficacious treatments to field settings: The link between supervisory practices and therapist fidelity in MST programs. Journal of Clinical Child and Adolescent Psychology, 31(2), 155–167. doi:10.1207/153744202753604449.

    Article  PubMed  Google Scholar 

  • Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. doi:10.1016/j.cpr.2010.02.005.

    Article  PubMed  PubMed Central  Google Scholar 

  • Imel, Z. E., Baer, J. S., Martino, S., Ball, S. A., & Carroll, K. M. (2011). Mutual influence in therapist competence and adherence to motivational enhancement therapy. Drug and Alcohol Dependence, 115(3), 229–236. doi:10.1016/j.drugalcdep.2010.11.010.

    Article  PubMed  PubMed Central  Google Scholar 

  • Imel, Z. E., Baldwin, S. A., Baer, J. S., Hartzler, B., Dunn, C., Rosengren, D. B., & Atkins, D. C. (2014). Evaluating therapist adherence in Motivational Interviewing by comparing performance with standardized and real patients. Journal of Consulting and Clinical Psychology, 82(3), 472–481. doi:10.1037/a0036158.

    Article  PubMed  PubMed Central  Google Scholar 

  • James, I. A., Milne, D., & Morse, R. (2008). Microskills of clinical supervision: Scaffolding skills. Journal of Cognitive Psychotherapy, 22(1), 29–36. doi:10.1891/0889.8391.22.1.29.

    Article  Google Scholar 

  • Kaslow, N. J., Borden, K. A., Collins, F. L., Forrest, L., Illfelder-Kaye, J., Nelson, P. D., et al. (2004). Competencies conference: Future directions in education and credentialing in professional psychology. Journal of Clinical Psychology, 60(7), 699–712. doi:10.1002/jclp.20016.

    Article  PubMed  Google Scholar 

  • Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. New Jersey: Prentice-Hall.

    Google Scholar 

  • Kolko, D. J., Cohen, J. A., Mannarino, A. P., Baumann, B. L., & Knudsen, K. (2009). Community treatment of child sexual abuse: A survey of practitioners in the National Child Traumatic Stress Network. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 37–49. doi:10.1007/s10488-008-0180-0.

    Article  PubMed  Google Scholar 

  • Laska, K. M., Gurman, A. S., & Wampold, B. E. (2014). Expanding the lens of evidence-based practice in psychotherapy: A common factors perspective. Psychotherapy, 51(4), 467–481. doi:10.1037/a0034332.

    Article  PubMed  Google Scholar 

  • Lau, N., & Weisz, J.R. (2012). Manual for the cognitive behavioral therapy competence observational measure of performance with youth depression (CBTCOMP-YD). Unpublished manuscript, Department of Psychology, Harvard University, Cambridge, MA.

  • Leahy, R. L., & Rego, S. A. (2012). Cognitive restructuring. In W. T. O’Donohue, J. E. Fisher, W. T. O’Donohue, & J. E. Fisher (Eds.), Cognitive behavior therapy: Core principles for practice (p. 133). Hoboken, NJ, US: Wiley. doi:10.1002/9781118470886.ch6.

    Chapter  Google Scholar 

  • McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodriguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20(1), 14–32. doi:10.1111/cpsp.12020.

    Google Scholar 

  • Milne, D. (2009). Evidence-based clinical supervision: Principles and practice. Leicester, England: Malden Blackwell Publishing.

    Google Scholar 

  • Nadeem, E., Gleacher, A., & Beidas, R. S. (2013). Consultation as an implementation strategy for evidence-based practices across multiple contexts: Unpacking the black box. Administration and Policy in Mental Health, 40(6), 439–450. doi:10.1007/s10488-013-0502-8.

    Article  PubMed  PubMed Central  Google Scholar 

  • Nakamura, B. J., Higa-McMillan, C. K., Okamura, K. H., & Shimabukuro, S. (2011). Knowledge of and attitudes towards evidence-based practices in community child mental health practitioners. Administration and Policy In Mental Health, 38(4), 287–300. doi:10.1007/s10488-011-0351-2.

    Article  PubMed  Google Scholar 

  • Nelson, T. D., & Steele, R. G. (2007). Predictors of practitioner self-reported use of evidence-based practices: practitioner training, clinical setting, and attitudes toward research. Administration and Policy in Mental Health and Mental Health Services Research, 34(4), 319–330. doi:10.1007/s10488-006-0111-x.

    Article  PubMed  Google Scholar 

  • Norcross, J. C., & Wampold, B. E. (2011). Evidence-based therapy relationships: Research conclusions and clinical practices. In J. C. Norcross, J. C. Norcross (Eds.), Psychotherapy relationships that work: Evidence-based responsiveness (2nd ed.) (pp. 423-430). New York, NY, US: Oxford University Press: 10.1093/acprof:oso/9780199737208.003.0021.

  • NREPP: The Substance Abuse and Mental Health Services Administration’s National Registry of Evidence-Based Programs and Practices. http://nrepp.samhsa.gov/Index.aspx. Accessed 15 Aug 2014.

  • R: A language and environment for statistical computing. (2015). R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/.

  • Rakovshik, S. G., & McManus, F. (2010). Establishing evidence-based training in cognitive behavioral therapy: A review of current empirical findings and theoretical guidance. Clinical Psychology Review, 30(5), 496–516. doi:10.1016/j.cpr.2010.03.004.

    Article  PubMed  Google Scholar 

  • Roth, A. D., Pilling, S., & Turner, J. (2010). Therapist training and supervision in clinical trials: Implications for clinical practice. Behavioural and Cognitive Psychotherapy, 38(3), 291–302. doi:10.1017/S1352465810000068.

    Article  PubMed  Google Scholar 

  • Schoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., et al. (2008). A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs). Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 84–97. doi:10.1007/s10488-007-0147-6.

    Article  PubMed  Google Scholar 

  • Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. doi:10.1007/s10488-010-0321-0.

    Article  PubMed  PubMed Central  Google Scholar 

  • Schoenwald, S. K., Mehta, T. G., Frazier, S. L., & Shernoff, E. S. (2013). Clinical supervision in effectiveness and implementation research. Clinical Psychology: Science and Practice, 20(1), 44–59. doi:10.1111/cpsp.12022.

    Google Scholar 

  • Schoenwald, S. K., Sheidow, A. J., & Chapman, J. E. (2009). Clinical supervision in treatment transport: effects on adherence and outcomes. Journal of Consulting and Clinical Psychology, 77(3), 410–421. doi:10.1037/a0013788.

    Article  PubMed  PubMed Central  Google Scholar 

  • Schwalbe, C. S., Oh, H. Y., & Zweben, A. (2014). Sustaining motivational interviewing: A meta-analysis of training studies. Addiction, 109(8), 1287–1294. doi:10.1111/add.12558.

    Article  PubMed  Google Scholar 

  • Silverman, W. K., & Hinshaw, S. P. (2008). The second special issue on evidence-based psychosocial treatments for children and adolescents: A 10-year update. Journal of Clinical Child and Adolescent Psychology, 37(1), 1–7. doi:10.1080/15374410701817725.

    Article  Google Scholar 

  • Spielmans, G. I., Gatlin, E. T., & McFall, J. P. (2010). The efficacy of evidence-based psychotherapies versus usual care for youths: Controlling confounds in a meta-reanalysis. Psychotherapy Research, 20(2), 234–246. doi:10.1080/10503300903311293.

    Article  PubMed  Google Scholar 

  • Stoltenberg, C. D. (2005). Enhancing professional competence through developmental approaches to supervision. American Psychologist, 60(8), 857–864. doi:10.1037/0003-066X.60.8.85.

    Article  PubMed  Google Scholar 

  • Stoltenberg, C. D., McNeill, B., & Delworth, U. (1998). IDM supervision: An integrated developmental model for supervising counselors and therapists. San Francisco, CA, US: Jossey-Bass.

    Google Scholar 

  • Watkins, C. E, Jr. (2014). Clinical supervision in the 21st century: revisiting pressing needs and impressing possibilities. American Journal of Psychotherapy, 68(2), 251–272.

    PubMed  Google Scholar 

  • Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78(2), 200–211. doi:10.1037/a0018912.

    Article  PubMed  PubMed Central  Google Scholar 

  • Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., et al. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A Randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. doi:10.1001/archgenpsychiatry.2011.147.

    Article  PubMed  Google Scholar 

  • Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70(7), 750–761. doi:10.1001/jamapsychiatry.2013.1176.

    Article  CAS  PubMed  Google Scholar 

  • Weisz, J. R., Ng, M. Y., & Bearman, S. K. (2014). Odd couple? Reenvisioning the relation between science and practice in the dissemination-implementation era. Clinical Psychological Science, 2(1), 58–74. doi:10.1177/2167702613501307.

    Article  Google Scholar 

  • Wheeler, S., & Richards, K. (2007). The impact of clinical supervision on counsellors and therapists, their practice, and their clients. A systematic review of the literature. Counselling and Psychotherapy Research, 7(1), 54–65. doi:10.1080/14733140601185274.

    Article  Google Scholar 

Download references

Acknowledgments

We acknowledge with thanks the research funding received by Sarah Kate Bearman from the National Institute of Mental Health (MH083887) and the Annie E. Casey Foundation. We are also grateful for the assistance of Adam Sales, Ph.D. and Daniel Swan, M.Ed. for their assistance with this manuscript.

Author information

Authors and Affiliations

  1. Department of Educational Psychology, The University of Texas at Austin, 504 SZB, 1 University Station, D5800, Austin, TX, 78712-0383, USA

    Sarah Kate Bearman

  2. Ferkauf Graduate School of Psychology, Department of School-Clinical Child Psychology, Yeshiva University, New York, USA

    Robyn L. Schneiderman & Emma Zoloth

Authors
  1. Sarah Kate Bearman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robyn L. Schneiderman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emma Zoloth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sarah Kate Bearman.

Ethics declarations

Conflict of Interest

The authors declare they have no conflict of interest.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Research Involving Human and Animal Rights

All procedures performed involving human participants were in accordance with the ethical standards of the Albert Einstein College of Medicine Institutional Review Board and with the 1964 Helsinki declaration and its later amendments.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bearman, S.K., Schneiderman, R.L. & Zoloth, E. Building an Evidence Base for Effective Supervision Practices: An Analogue Experiment of Supervision to Increase EBT Fidelity. Adm Policy Ment Health 44, 293–307 (2017). https://doi.org/10.1007/s10488-016-0723-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-016-0723-8

Keywords

Search

Navigation