Skip to main content

Advertisement

SpringerLink
Log in

Improvement in Symptoms Versus Functioning: How Do Our Best Treatments Measure Up?

  • Original Paper
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

We examined the effects of redefining standards of evidence for treatments targeting childhood mental health problems by expanding outcomes beyond symptom reduction to include functioning. Over 750 treatment protocols from 435 randomized controlled trials were rated based on empirical evidence. Nearly two-thirds (63.9%) demonstrated at least a minimum level of evidence for reducing symptoms; however, only 18.8% of treatments demonstrated evidence for reducing functional impairment. Of those treatments with empirical support for symptom reduction, the majority did not demonstrate empirical support for improvement in functioning because measures of functioning were not included in the studies in which these treatments were tested. However, even when measures of functioning were included, it was much more difficult for treatments to achieve improvement. Among treatments that achieved improvement in functioning, the most notable were Collaborative Problem Solving for disruptive behavior and Cognitive Behavioral Therapy plus Medication for traumatic stress because they demonstrated no support for symptom reduction but good support for improvement in functioning. Results are discussed within the context of evaluating the standards of evidence for treatments and the opportunity to move towards a multidimensional framework whose utility has the potential to exceed the sum of its parts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Achenbach, T. (1991). Manual for the child behavior checklist/4–18 and 1991 profile. Burlington: Department of Psychiatry, University of Vermont.

    Google Scholar 

  • American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders, (4th ed.), text revision. Washington, DC: American Psychiatric Association

  • Bacon, S., Collins, M., & Plake, E. (2002). Does the Global Assessment of Functioning assess functioning? Journal of Mental Health Counseling, 24, 202–212.

    Google Scholar 

  • Bates, M. (2001). The Child and Adolescent Functional Assessment Scale (CAFAS): Review and current status. Clinical Child and Family Psychology Review, 4, 63–84.

    Article  PubMed  CAS  Google Scholar 

  • Becker, K. D., Nakamura, B., Young, J., & Chorpita, B. (2009). What better place than here? What better time than now? ABCT’s burgeoning role in the dissemination and implementation of evidence-based practices. The Behavior Therapist, 32, 89–96.

    Google Scholar 

  • Bickman, L., Lambert, E., Karver, M., & Andrade, A. (1998). Two low-cost measures of child and adolescent functioning for services research. Evaluation and Program Planning, 21, 263–275.

    Article  Google Scholar 

  • Bickman, L., Lambert, E., & Schilling, S. (1999). Time patterns of mental health outcomes. Mental Health Services Research, 1, 67–82.

    Article  Google Scholar 

  • Bickman, L., Rosof-Williams, J., Salzer, M., Summerfelt, W., Noser, K., Wilson, S., et al. (2000). What information do clinicians value for monitoring adolescent client progress and outcomes? Professional Psychology: Research and Practice, 31, 70–74.

    Article  Google Scholar 

  • Bird, H. (1999). The assessment of functional impairment. In D. Shaffer, C. Lucas, & J. Richters (Eds.), Diagnostic assessment in child and adolescent psychopathology (pp. 209–229). NY: Guilford.

    Google Scholar 

  • Bird, H., Yager, T., Staghezza, B., & Gould, M. (1990). Impairment in the epidemiological measurement of childhood psychopathology in the community. Journal of the American Academy of Child & Adolescent Psychiatry, 29, 796–803.

    Article  CAS  Google Scholar 

  • Burlingame, G., Lambert, M., Reisinger, C., Neff, W., & Mosier, J. (1995). Pragmatics of tracking mental health outcomes in a managed care setting. Mental Health Services Research, 22, 226–236.

    CAS  Google Scholar 

  • Canino, G., Costello, E., & Angold, A. (1999). Assessing functional impairment for child mental health services research: A review of measures. Journal of Mental Health Services Research, 1, 93–108.

    Article  Google Scholar 

  • Chambless, D., Baker, M., Baucom, D., Beurtler, L., Calhoun, K., Crits-Cristoph, P., et al. (1998). Update on empirically validated therapies, II. The Clinical Psychologist, 51, 3–16.

    Google Scholar 

  • Chambless, D., Sanderson, W., Shoham, V., Bennett Johnson, S., Pope, K., Crits-Christoph, P., et al. (1996). An update on empirically validated therapies. The Clinical Psychologist, 49, 5–18.

    Google Scholar 

  • Chorpita, B. (2001). Reflections on clinical significance: What do our best treatments accomplish and how can we best find out? Clinical Psychology: Science and Practice, 8, 451–454.

    Article  Google Scholar 

  • Chorpita, B., & Daleiden, E. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77, 566–579.

    Article  PubMed  Google Scholar 

  • Chorpita, B., Daleiden, E., Ebesutani, C., Young, J., Becker, K. D., Nakamura, B. et al. (in press). Evidence based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice.

  • Chorpita, B., & Regan, J. (2009). Dissemination of effective mental health treatment procedures: Maximizing the return on a significant investment. Behaviour Research and Therapy, 47, 990–993.

    Article  PubMed  Google Scholar 

  • Chorpita, B., Yim, L., Donkervoet, J., Arensdorf, A., Amundsen, M., McGee, C., et al. (2002). Toward large-scale implementation of empirically supported treatments for children: A review and observations by the Hawaii Empirical Basis to Services Task Force. Clinical Psychology: Science and Practice, 9, 165–190.

    Article  Google Scholar 

  • Cohen, J., Mannarino, A., Perel, J., & Staron, V. (2007). A pilot randomized controlled trial of combined trauma-focused CBT and sertraline for childhood PTSD symptoms. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 811–819.

    Article  PubMed  Google Scholar 

  • De Los Reyes, A., & Kazdin, A. (2008). When the evidence says, ‘yes, no, and maybe so’: Attending to and interpreting inconsistent findings among evidence-based interventions. Current Directions in Psychological Science, 17, 47–51.

    Article  PubMed  Google Scholar 

  • Drake, R., Latimer, E., Leff, H., McHugo, G., & Burns, B. (2004). What is evidence? Child and Adolescent Psychiatric Clinics of North America, 13, 717–728.

    Article  PubMed  Google Scholar 

  • Drake, R., Rosenberg, S., Teague, G., Bartels, S., & Torrey, W. (2003). Fundamental principles of evidence-based medicine applied to mental health care. Psychiatric Clinics of North America, 26, 811–820.

    Article  PubMed  Google Scholar 

  • Dulcan, M. (2005). Practitioner perspectives on evidence-based practice. Child and Adolescent Psychiatric Clinics of North America, 14, 225–240.

    Article  PubMed  Google Scholar 

  • Goldman, H. (2005). Do you walk to school, or do you carry your lunch? Psychiatric Services, 56, 419.

    Article  Google Scholar 

  • Goldman, H., Ganju, V., Drake, R., Gorman, P., Hogan, M., Hyde, P., et al. (2001). Policy implications for implementing evidence-based practices. Psychiatric Services, 52, 1591–1597.

    Article  PubMed  CAS  Google Scholar 

  • Goldman, H., Skodol, A., & Lave, T. (1992). Revising axis V for DSM-IV: A review of measures of social functioning. American Journal of Psychiatry, 149, 1148–1156.

    PubMed  CAS  Google Scholar 

  • Greene, R., Ablon, J., Monuteaux, M., Goring, J., Henin, A., Raezer, L., et al. (2004). Effectiveness of collaborative problem solving in affectively dysregulated youth with oppositional defiant disorder: Initial findings. Journal of Consulting and Clinical Psychology, 72, 1157–1164.

    Article  PubMed  Google Scholar 

  • Hayes, S., Barlow, D., & Nelson-Gray, R. (1999). The scientist practitioner: Research and accountability in the age of managed care. Needham heights (2nd ed.). MA US: Allyn & Bacon.

    Google Scholar 

  • Hernandez, M., Hodges, S., & Cascardi, M. (1998). The ecology of outcomes: System accountability in children’s mental health. The Journal of Behavioral Health Services & Research, 25, 136–150.

    Article  CAS  Google Scholar 

  • Hoagwood, K., Jensen, P., Petti, T., & Burns, B. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 1055–1063.

    Article  CAS  Google Scholar 

  • Hodges, K., Doucette-Gates, A., & Kim, C. (2000). Predicting service utilization with the child and adolescent functional assessment scale in a sample of youths with serious emotional disturbance served by center for mental health Services-funded demonstrations. The Journal of Behavioral Health Services & Research, 27, 47–59.

    Article  CAS  Google Scholar 

  • Individuals with Disabilities Education Improvement Act (2004) H.R. 1350, 108th Congress.

  • Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.

    Google Scholar 

  • Jacobson, N., Roberts, L., Berns, S., & McGlinchey, J. (1999). Methods for defining and determining the clinical significance of treatment effects: Description, application, and alternatives. Journal of Consulting and Clinical Psychology, 67, 300–307.

    Article  PubMed  CAS  Google Scholar 

  • Kagan, J. (1964). Matching familiar figures test. Cambridge, MA: Harvard University.

    Google Scholar 

  • Karver, M., & Bickman, L. (2002). Positive functioning: Does it add validity to maladaptive functioning items? Evaluation and Program Planning, 25, 85–93.

    Article  Google Scholar 

  • Kazdin, A., & Bass, D. (1989). Power to detect differences between alternative treatments in comparative psychotherapy outcome research. Journal of Consulting and Clinical Psychology, 57, 138–147.

    Article  PubMed  CAS  Google Scholar 

  • Kazdin, A., & Kendall, P. (1998). Current progress and future plans for developing effective treatments: Comments and perspectives. Journal of Clinical Child Psychology, 27, 217–226.

    Article  PubMed  CAS  Google Scholar 

  • McGlinchey, J., Zimmerman, M., & Atkins, D. (2008). Clinical significance and remission in treating major depressive disorder: Parallels between related outcome constructs. Harvard Review of Psychiatry, 16, 25–34.

    Article  PubMed  Google Scholar 

  • McLeod, B., & Weisz, J. (2004). Using dissertations to examine potential bias in child and adolescent clinical trials. Journal of Consulting and Clinical Psychology, 72, 235–251.

    Article  PubMed  Google Scholar 

  • Nakamura, B., Chorpita, B., Hisch, M., Daleiden, E., Slavin, L., Amundson, M. et al. (2009). Large-scale implementation of evidence-based treatments for children ten years later: Hawaii’s evidence based services initiative in children’s mental health (submitted).

  • Pickles, A., Rowe, R., Simonoff, E., Foley, D., Rutter, M., & Silberg, J. (2001). Child psychiatric symptoms and psychosocial impairment: Relationship and prognostic significance. British Journal of Psychiatry, 179, 230–235.

    Article  PubMed  CAS  Google Scholar 

  • Plante, T., Couchman, C., & Diaz, A. (1995). Measuring treatment outcome and client satisfaction among children and families. Journal of Mental Health Administration, 22, 61–269.

    Article  Google Scholar 

  • PracticeWise. (2005). Psychosocial and combined treatments coding manual. Satellite Beach, FL: Author.

  • Rapp, C., Bond, G., Becker, D., Carpinello, S., Nikkel, R., & Gintoli, G. (2005). The role of state mental health authorities in promoting improved client outcomes through evidence-based practice. Community Mental Health Journal, 41, 347–363.

    Article  PubMed  Google Scholar 

  • Reynolds, C., & Richmond, B. (1985). Revised children’s manifest anxiety scale. Los Angeles: Western Psychological Services.

    Google Scholar 

  • Rogers, S., & Vismara, L. (2008). Evidence-based comprehensive treatments for early autism. Journal of Clinical Child & Adolescent Psychology, 37, 8–38.

    Article  Google Scholar 

  • Schaffer, D., et al. (1983). A children’s global assessment scale (CGAS). Archives of General Psychiatry, 40, 1228–1231.

    Google Scholar 

  • Schoenwald, S., Hoagwood, K., Atkins, M., Evans, S., & Ringeisen, H. (2010). Workforce development and the organization of work: The science we need. Administration and Policy In Mental Health, 37, 71–80.

    Article  PubMed  Google Scholar 

  • Sechrest, L., McKnight, P., & McKnight, K. (1996). Calibration of measures for psychotherapy outcome studies. American Psychologist, 51, 1065–1071.

    Article  PubMed  CAS  Google Scholar 

  • Silverman, W., & Hinshaw, S. (2008). The second special issue on evidence-based psychosocial treatments for children and adolescents: A 10-year update. Journal of Clinical Child and Adolescent Psychology, 37, 1–7.

    Article  Google Scholar 

  • Simonoff, E., et al. (1997). The Virginia twin study of adolescent behavioral development: Influences of age, gender and impairment on rates of disorder. Archives of General Psychiatry, 54, 801–808.

    PubMed  CAS  Google Scholar 

  • Sparrow, S., Balla, D., & Cicchetti, D. (1984). Vineland adaptive behavior scales: Interview edition survey form manual. Circle Pines, MN: American Guidance Service.

    Google Scholar 

  • Striley, C., Stiffman, A., & Spitznagel, E. (2003). Functioning mediates between symptoms and provider assessment. Mental Health Services Research, 5, 155–171.

    Article  PubMed  Google Scholar 

  • Swets, J. (1992). The science of choosing the right decision threshold in high-stakes diagnostics. American Psychologist, 47, 522–532.

    Article  PubMed  CAS  Google Scholar 

  • Task Force on Promotion, Dissemination of Psychological Procedures D. o. C. P. (1995). Training in and dissemination of empirically validated psychological treatments: Report and recommendations. The Clinical Psychologist, 48, 3–23.

    Google Scholar 

  • Wampold, B., Imel, Z., & Miller, S. (2009). Barriers to the dissemination of empirically supported treatments: Matching messages to the evidence. The Behavior Therapist, 32, 144–155.

    Google Scholar 

  • Weisz, J., Hawley, K., & Jensen-Doss, A. (2004). Empirically tested psychotherapies for youth internalizing and externalizing problems and disorders. Child and Adolescent Psychiatric Clinics of North America, 13, 729–815.

    Article  PubMed  Google Scholar 

  • Weisz, J., Weersing, V., & Henngeler, S. (2005). Jousting with straw men: Comment on Westen, Novotny, and Thompson-Brenner. Psychological Bulletin, 131, 418–426.

    Article  PubMed  Google Scholar 

  • Westen, D., & Bradley, R. (2005). Empirically supported complexity: Rethinking evidence-based practice in psychotherapy. Current Directions in Psychological Science, 14, 266–271.

    Article  Google Scholar 

  • Westen, D., Novotny, C., & Thompson-Brenner, H. (2004). The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin, 130, 631–663.

    Article  PubMed  Google Scholar 

  • Winters, N., Collett, B., & Myers, K. (2005). Ten-year review of rating scales, VII: Scales assessing functional impairment. Journal of the Academy of Child and Adolescent Psychiatry, 44, 309–338.

    Article  Google Scholar 

  • Zimmerman, M., McGlinchey, J. B., Posternak, M. A., Friedman, M., Attiullah, M., & Boerescu, D. (2006). How should remission from depression be defined? The depressed patient’s perspective. The American Journal of Psychiatry, 163, 148–150.

    Article  PubMed  Google Scholar 

Download references

Acknowledgment

During the preparation of this manuscript, Dr. Becker was supported by the National Institute of Mental Health (T32 MH18834).

Disclosure

Bruce F. Chorpita is President and Eric L. Daleiden is Chief Operating Officer of PracticeWise, LLC, a private behavioral health consulting corporation. Kimberly D. Becker is a consultant to PracticeWise, LLC.

Author information

Authors and Affiliations

  1. Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, 111 Market Place, Suite 901, Baltimore, MD, 21205, USA

    Kimberly D. Becker

  2. Department of Psychology, The University of California, UCLA, 1285 Franz Hall, Box 951563, Los Angeles, CA, 90095-1563, USA

    Bruce F. Chorpita

  3. PracticeWise, LLC, 285 Wilson Ave, Satellite Beach, FL, 32937, USA

    Eric L. Daleiden

Authors
  1. Kimberly D. Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruce F. Chorpita
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric L. Daleiden
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kimberly D. Becker.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Becker, K.D., Chorpita, B.F. & Daleiden, E.L. Improvement in Symptoms Versus Functioning: How Do Our Best Treatments Measure Up?. Adm Policy Ment Health 38, 440–458 (2011). https://doi.org/10.1007/s10488-010-0332-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-010-0332-x

Keywords

Search

Navigation