Skip to main content

Advertisement

SpringerLink
Log in

Formative Evaluation of a Framework for High Quality, Evidence-Based Services in School Mental Health

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

The purpose of this study was to test a three component framework for enhancing quality in school mental health (SMH), focusing on quality assessment and improvement (QAI), family engagement/empowerment, and modular evidence-based practice (EBP) implementation in three established SMH programs. The study involved a 2-year, multisite (Delaware, Maryland, Texas) formative evaluation with clinicians randomly assigned to participate in either the QAI (target) intervention or a Wellness Plus Information (WPI, comparison) intervention. As hypothesized, clinicians who participated in the QAI condition demonstrated significantly greater implementation of quality indicators and greater implementation of EBP as compared to clinicians in the WPI condition. However, contrary to original hypotheses, findings did not reveal differences between the conditions in knowledge or attitudes toward EBP, clinician self-efficacy, or student psychosocial outcomes. Implications for future research on quality improvement in SMH are discussed, with an emphasis on the need to examine the impact of increased implementation and resource support to SMH clinicians.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61–74.

    Article  PubMed  Google Scholar 

  • Attkisson, C. C., & Zwick, R. (1982). The Client Satisfaction Questionnaire: Psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and Program Planning, 5(3), 233–237.

    Article  PubMed  Google Scholar 

  • Barnard, W. M. (2004). Parent involvement in elementary school and educational attainment. Child and Youth Services Review, 26, 39–62.

    Article  Google Scholar 

  • Bickham, N., Pizarro, J., Warner, B., Rosenthal, B., & Weist, M. (1998). Family involvement in expanded school mental health. Journal of School Health, 68(10), 425–428.

    Article  PubMed  Google Scholar 

  • Chorpita, B. (2006). Modular cognitive-behavioral therapy for childhood anxiety disorders. New York: Guilford Press.

    Google Scholar 

  • Chorpita, B. F., Becker, K. D., & Daleiden, E. L. (2007). Understanding the common elements of evidence based practice: Misconceptions and clinical examples. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 647–652.

    Article  PubMed  Google Scholar 

  • Chorpita, B. F., & Daleiden, E. L. (2007). 2007 Biennial report: Effective psychosocial intervention for youth with behavioral and emotional needs. Hawaii: Child and Mental Health Division, Hawaii Department of Health.

    Google Scholar 

  • Chorpita, B. F., Taylor, A. A., Francis, S. E., Moifitt, C. E., & Austin, A. A. (2004). Efficacy of modular cognitive behavior therapy for childhood anxiety disorders. Behavior Therapy, 35, 263–287.

    Article  Google Scholar 

  • Cohen, P., Cohen, J., Aiken, L. S., & West, S. G. (1999). The problem of units and the circumstances for POMP. Multivariate Behavioral Research, 34, 315–346.

    Article  Google Scholar 

  • Cook, T., & Campbell, D. (1983). The design and conduct of quasi-experiments and true experiments in field settings. In M. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 223–326). Chicago: Rand McNally.

    Google Scholar 

  • Cook, T., Campbell, D., & Peracchio, L. (1990). Quasi experimentation. In M. Dunnette & L. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 1, pp. 491–576). Palo Alto, CA: Consulting Psychologists Press.

    Google Scholar 

  • Curry, J. F., & Reinecke, M. A. (2003). Modular cognitive behavior therapy for adolescents with major depression. In M. Reincke, F. Datillo, & A. Freeman (Eds.), Cognitive therapy with children and adolescents: A casebook for clinical practice (pp. 95–127). New York: Guilford Press.

    Google Scholar 

  • Evans, S. W., & Weist, M. D. (2004). Implementing empirically supported treatments in the schools: What are we asking? Clinical Child and Family Psychology Review, 7, 263–267.

    Article  PubMed  Google Scholar 

  • Evans, S. W., Weist, M. D., & Serpell, Z. (2007). Advances in school-based mental health interventions: Best practices and program models (Vol. II). New York: Civic Research Institute.

    Google Scholar 

  • Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature (FMHI publication #231). Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.

    Google Scholar 

  • Foster, S., Rollefson, M., Doksum, T., Noonan, D., & Robinson, G. (2005). School mental health services in the United States, 2002–2003. Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration.

    Google Scholar 

  • Glorfeld, L. W. (1995). An improvement on Horn’s parallel analysis methodology for selecting the correct number of factors to retain. Educational and Psychological Measurement, 55(3), 377–393.

    Article  Google Scholar 

  • Goodman, R., Meltzer, H., & Bailey, V. (1998). The strengths and difficulties questionnaire: A pilot study on the validity of the self-report version. European Child and Adolescent Psychiatry, 17(3), 125–130.

    Article  Google Scholar 

  • Graczyk, P. A., Domitrovich, C. E., & Zins, J. E. (2003). Facilitating the implementation of evidence-based prevention and mental health promotion efforts in schools. In M. Weist, S. Evans, & N. Lever (Eds.), Handbook of school mental health: Advancing practice and research (pp. 301–318). New York, NY: Kluwer Academic.

    Google Scholar 

  • Guadagnoli, E., & Velicer, W. F. (1988). Relation to sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265–275.

    Article  PubMed  Google Scholar 

  • Hazler, R., & Mellin, E. (2004). The developmental origins and treatment needs of female adolescents with depression. Journal of Counseling and Development, 82(1), 18–24.

    Google Scholar 

  • Hill, N. E., Castellino, D. R., Lansford, J. E., Nowlin, P., Dodge, K. A., Bates, J. E., et al. (2004). Parent academic involvement as related to school behavior, achievement, and aspirations: Demographic variations across adolescence. Child Development, 75(5), 1491–1509.

    Article  PubMed  Google Scholar 

  • Hoagwood, K. E. (2005). Family-based services in children’s mental health: A research review and synthesis. Journal of Child Psychology and Psychiatry, 46(7), 690–713.

    Article  PubMed  Google Scholar 

  • Hoagwood, K., Olin, S. S., Kerker, B., Kratochwill, T. R., Crowe, M., & Saka, N. (2007). Empirically-based school interventions targeted at academic and mental health functioning. Journal of Emotional and Behavioral Disorder, 15(2), 66–92.

    Article  Google Scholar 

  • Hunter, S. C., Boyle, J. M., & Warden, D. (2004). Help seeking amongst child and adolescent victims of peer-aggression and bullying: The influence of school-stage, gender, victimisation, appraisal, and emotion. British Journal of Educational Psychology, 74(3), 375–390.

    Article  PubMed  Google Scholar 

  • Jensen, P. S., & Hoagwood, K. E. (Eds.). (2008). Improving children’s mental health through parent empowerment: A guide to assisting families. New York: Oxford University Press.

    Google Scholar 

  • Jeynes, W. H. (2005). A meta-analysis of the relation of parental involvement to urban elementary school student academic achievement. Urban Education, 40, 237–269.

    Article  Google Scholar 

  • Kutash, K., Duchnowski, A. J., & Lynn, N. (2006). School-based mental health: An empirical guide for decision makers. Tampa, FL: University of South Florida, The Louis de la Parte Florida Mental Health Institute, Department of Child & Family Studies, Research and Training Center for Children’s Mental Health.

    Google Scholar 

  • Lowie, J. A., Lever, N. A., Ambrose, M. G., Tager, S. B., & Hill, S. (2003). Partnering with families in expanded school mental health programs. In M. Weist, S. Evans, & N. Lever (Eds.), Handbook of school mental health programs: Advancing practice and research. New York: Kluwer Academic.

    Google Scholar 

  • MacCallum, R. C., Widaman, K. F., Preacher, K. J., & Hong, S. (2001). Sample size in factor analysis: The role of model error. Multivariate Behavioral Research, 36(4), 611–637.

    Article  Google Scholar 

  • Marcon, R. A. (1999). Positive relationships between parent school involvement and public school inner-city preschoolers’ development and academic performance. School Psychology Review, 28(3), 395–412.

    Google Scholar 

  • McKay, M. (2004, March). Engagement of families in evidence-based trauma treatment: Preliminary findings. In K. Hoagwood (Eds.), Organizational efficacy: Impact of culture, climate, and family engagement on clinical care. Symposium conducted at the 17th annual Research Conference. A system of care for children’s mental health: Expanding the research base, Tampa, FL.

  • Paternite, C. E., Weist, M. D., Axelrod, J., Anderson-Butcher, D., & Weston, K. (2006). School mental health workforce issues. In M. Hoge, J. Morris, A. Daniels, N. Adams, L. Huey, & G. Stuart (Eds.), A thousand voices: The national action plan on behavioral health workforce development (pp. 199–210). Rockville, MD: U.S. Substance Abuse and Mental Health Services Administration.

    Google Scholar 

  • Power, T. J. (2003). Promoting children’s mental health: Reform through interdisciplinary and community partnerships. School Psychology Review, 32(1), 3–16.

    Google Scholar 

  • Power, T. J., Manz, P. H., & Leff, S. S. (2003). Training for effective practice in the schools. In M. Weist, S. Evans, & N. Lever (Eds.), Handbook of school mental health: Advancing practice and research (pp. 257–273). New York: Kluwer Academic.

    Google Scholar 

  • President’s New Freedom Commission on Mental Health. (2003). Achieving the Promise: Transforming Mental Health Care in America. Final Report for the President’s New Freedom Commission on Mental Health (SMA Publication No. 03-3832). Rockville, MD: President’s New Freedom Commission on Mental Health.

  • Rappaport, N., Osher, D., Garrison, E. G., Anderson-Ketchmark, C., & Dwyer, K. (2003). Enhancing collaboration within and across disciplines to advance mental health programs in schools. In M. Weist, S. Evans, & N. Lever (Eds.), Handbook of school mental health programs: Advancing practice and research (pp. 107–118). New York: Kluwer Academic.

    Google Scholar 

  • Rickwood, D. J., & Braithwaite, V. A. (1994). Social-psychological factors affecting help-seeking for emotional problems. Social Science and Medicine, 39, 563–572.

    Article  PubMed  Google Scholar 

  • Roberts, R. E., Atkisson, C. C., & Mendia, R. M. (1984). Assessing the Client Satisfaction Questionnaire in English and Spanish. Hispanic Journal of Behavioral Sciences, 6(4), 385–396.

    Article  Google Scholar 

  • Robinson, K. E. (2004). Advances in school-based mental health interventions: Best practices and program models. New York: Civic Research Institute.

    Google Scholar 

  • Schaeffer, C. M., Bruns, E., Weist, M., Stephan, S. H., Goldstien, J., & Simpson, Y. (2005). Overcoming challenges to using evidence-based interventions in schools. Journal of Youth and Adolescence, 34(1), 15–22.

    Article  Google Scholar 

  • Shernoff, E. S., Kratochwill, T. R., & Stoiber, K. C. (2003). Training in evidence-based interventions (EBI’s): What are school psychology programs teaching? Journal of School Psychology, 41, 467–483.

    Article  Google Scholar 

  • Stephan, S. H., Davis, E. T., Callan Burke, P., & Weist, M. D. (2006). Supervision in school mental health. In T. Neill (Ed.), Helping others help children: Clinical supervision of child psychotherapy. Washington, DC: American Psychological Association.

    Google Scholar 

  • Stumpf, R. E., Higa-McMillan, C. K., & Chorpita, B. F. (2009). Implementation of evidence based services for youth: Assessing provider knowledge. Behavior Modification, 33(1), 48–65.

    Google Scholar 

  • Sutton, J. M., & Fall, M. (1995). The relationship of school climate factors to counselor self-efficacy. Journal of Counseling & Development, 73(3), 331–336.

    Google Scholar 

  • Vanderbleek, L. M. (2004). Engaging families in school-based mental health treatment. Journal of Mental Health Counseling, 26, 211–224.

    Google Scholar 

  • Weist, M. D. (1997). Expanded school mental health services: A national movement on progress. In T. Ollendick & R. A. Prinz (Eds.), Advances in clinical child psychology (Vol. 19, pp. 319–352). New York, NY: Plenum Press.

    Google Scholar 

  • Weist, M. D., & Albus, K. E. (2004). Expanded school mental health: Exploring program details and developing the research base. Behavior Modification, 28, 463–471.

    Article  PubMed  Google Scholar 

  • Weist, M. D., Ambrose, M., & Lewis, C. (2006a). Expanded school mental health: A collaborative community/school example. Children & Schools, 28(1), 45–50.

    Google Scholar 

  • Weist, M. D., Evans, S. W., & Lever, N. A. (Eds.). (2003). Handbook of school mental health: Advancing practice and research. New York: Kluwer Academic.

    Google Scholar 

  • Weist, M. D., Lever, N. A., Stephan, S. H., Anthony, L. G., Moore, E. A., & Harrison, B. R. (2006, February). School mental health quality assessment and improvement: Preliminary findings from an experimental study. Paper presentation at the annual conference. A system of care for children’s mental health: Expanding the research base, Tampa, FL.

  • Weist, M. D., Sander, M. A., Walrath, C., Link, B., Nabors, L., Adelsheim, S., et al. (2005). Developing principles for best practice in expanded school mental health. Journal of Youth and Adolescence, 34(1), 7–13.

    Article  Google Scholar 

Download references

Acknowledgments

We extend our sincere appreciation to the following people who made critical contributions to the successful conduct of the study—Senior Clinicians: Patricia Cotton, Harold Burnett, and Gerald Grubbs; Site Managers: Ellie Davis, Rachel Grier, Phyllis Hazel, Jenni Jennings, and Glenn Pearson; Data Consultant: Mark Anderson; Site Coordinators: Skylar Woods-Nunley, and Nereyda Pederson, Research Associates: Linda Kinney, and Matthew Page; Advisory Panel: Alan Bellack, Maureen Black, Nick Ialongo, Karen Friedman, Philip Leaf, Richard Sarles, Susan Tager, Stuart Tiegel, Beth Warner; Consultants: Bruno Anthony, Bruce Chorpita, and David Pruitt.

Author information

Authors and Affiliations

  1. Division of Child and Adolescent Psychiatry, Center for School Mental Health, University of Maryland School of Medicine, 737 West Lombard Street, 4th Floor, Baltimore, MD, 21201, USA

    Mark Weist, Nancy Lever, Sharon Stephan, Elizabeth Moore, Bryan Harrison, Laura Anthony, Kenneth Rogers, Aya Ghunney, Krystal Lewis & Kerri Stiegler

  2. University of North Carolina, Chapel Hill, NC, USA

    Eric Youngstrom

  3. Columbia University, New York, NY, USA

    Kimberly Hoagwood

Authors
  1. Mark Weist
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nancy Lever
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sharon Stephan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eric Youngstrom
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elizabeth Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bryan Harrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Laura Anthony
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kenneth Rogers
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kimberly Hoagwood
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Aya Ghunney
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Krystal Lewis
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Kerri Stiegler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark Weist.

Additional information

Funded by the National Institute of Mental Health, US Department of Health and Human Services, #1R01MH71015-01A1; 2003-2007. Additional support for this project was provided by cooperative agreement U45 MC 00174-10-0 from the Office of Adolescent Health, Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Weist, M., Lever, N., Stephan, S. et al. Formative Evaluation of a Framework for High Quality, Evidence-Based Services in School Mental Health. School Mental Health 1, 196–211 (2009). https://doi.org/10.1007/s12310-009-9018-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-009-9018-5

Keywords

Search

Navigation