Swipe om te navigeren naar een ander artikel
The Child and Adolescent Functional Assessment Scale (CAFAS) is widely used for outcome management, for providing real time client and program level data, and the monitoring of evidence-based practices. Methods of reliability training and the assessment of rater drift are critical for service decision-making within organizations and systems of care. We assessed two approaches for CAFAS training: external technical assistance and internal technical assistance. To this end, we sampled 315 practitioners trained by external technical assistance approach from 2,344 Ontario practitioners who had achieved reliability on the CAFAS. To assess the internal technical assistance approach as a reliable alternative training method, 140 practitioners trained internally were selected from the same pool of certified raters. Reliabilities were high for both practitioners trained by external technical assistance and internal technical assistance approaches (.909–.995, .915–.997, respectively). 1 and 3-year estimates showed some drift on several scales. High and consistent reliabilities over time and training method has implications for CAFAS training of behavioral health care practitioners, and the maintenance of CAFAS as a global outcome management tool in systems of care.
Accreditation Working Group, Children’s Mental Health Ontario. (2004). BCFPI and CAFAS: Guidelines for use of the measurement tools in accreditation. Toronto: Author.
Barwick, M. A., Boydell, K. M., & Omrin, C. (2002). A knowledge transfer infrastructure for children’s mental health in Ontario: Building capacity for research and practice. Toronto, ON: The Hospital for Sick Children.
Barwick, M. A., Schmidt, J., & Hodges, K. (2004). Culturally competent evaluation: Clinical considerations for rating The Child and Adolescent Functional Assessment Scale with Aboriginal children and youth. Toronto, ON: The Hospital for Sick Children.
Boydell, K. M., Barwick, M. A., Ferguson, H. B., & Haines, R. (2004). A feasibility study to assess service providers’ perspectives regarding use of The Child and Adolescent Functional Assessment Scale in Ontario. Journal of Behavioral Health Services and Research, 32(1), 105–109. CrossRef
Chorpita, B. F., Bernstein, A. D., Daleiden, E. L., & The Research Network on Youth Mental Health. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 114–123. doi: 10.1007/s10488-007-0151-x. PubMedCrossRef
Fallon, T., Pumariega, A., Sowers, W., Klaehn, R., Huffine, C., Vaughan, T., et al. (2006). A level of care instrument for children’s systems of care: Construction, reliability and validity. Journal of Child and Family Studies, 15, 140–152. CrossRef
Franco, E., Holden, W., & Hodges, K. (2002). An exploratory study of factors influencing reliability and drift of the Child and Adolescent Functional Assessment scale ratings (unpublished).
Friesen, B. J., Giliberti, M., Katz-Leavy, J., Osher, T., & Pullmann, M. D. (2003). Research in the service of policy change: The “custody problem”. Journal of Emotional and Behavioral Disorders, 11(1), 39–47. CrossRef
Garland, A. F., Bickman, L., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research, 37(1–2), 15–26. doi: 10.1007/s10488-010-0279-y. PubMedCentralPubMedCrossRef
Hodges, K. (2003). Child and Adolescent Functional Assessment Scale (3rd ed.). Ypsilanti, MI: Eastern Michigan University.
Hodges, K. (2005). CAFAS manual for training coordinators, clinical administrators, and data managers (3rd ed.). Ann Arbor, MI: Kay Hodges.
Hodges, K. (2006). CAFAS self training manual and blank scoring forms. Ypsilanti, MI: Eastern Michigan University.
Hodges, K., Doucette-Gates, A., & Liao, Q. (1999). The relationship between the Child and Adolescent Functional Assessment Scale (CAFAS) and indicators of functioning. Journal of Child and Family Studies, 8, 109–122. CrossRef
Hodges, K., Latessa, M., Pernice, F., Wong, M., Doucette-Gates, A., & Liao, Q. (1997). Practical issues in using the CAFAS for clinical and administrative outcome. Symposium presented at the 10th Annual Research Conference on a System of Care for Children’s Mental Health: Expanding the Research Base. Tampa: FL.
Hodges, K., & Wong, M. M. (1996). Psychometric characteristics of a multidimensional measure to assess impairment: The Child and Adolescent Functional Assessment Scale (CAFAS). Journal of Child and Family Studies, 5, 445–467. CrossRef
Hodges, K., & Wotring, J. (2000). Client typology based on functioning across domains using the CAFAS: Implications for service planning. The Journal of Behavioral Health Services & Research, 27, 257–270. CrossRef
Hodges, K., Xue, Y., & Wotring, J. (2004). Use of the CAFAS to evaluate outcome for youths with SED served by public mental health. Journal of Child and Family Studies, 13, 325–339. CrossRef
Kaplan, R. M., & Saccuzzo, D. P. (1997). Psychological testing. Principles, applications, and issues (4th ed.). Pacific Grove, CA: Brooks/Cole.
Manteuffel, B., Stephens, R. L., & Santiago, R. (2002). Overview of the national evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program and summary of current findings. Children’s Services: Social Policy, Research, and Practice, 5(1), 3–20. CrossRef
Maruish, M. E. (Ed.). (2004). The use of psychological testing for treatment planning and outcomes assessment, 3rd edn, vol. 2. Mahwah: NJ: Lawrence Erlbaum Associates, Inc.
Ogles, B. M., Davis, D., & Lunnen, K. M. (1999). Inter-rater reliability of four measures of youth functioning. In J. Willis, C. Liberton, K. Kutash, & R. Friedman (Eds.), Proceedings of the 11th annual research conference on a system of care for children’s mental health (pp. 321–325). Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, Research and Training Center for Children’s Mental Health.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunder, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76. doi: 10.1007/s10488-010-0319-7. PubMedCentralPubMedCrossRef
Raphael, D., Weir, P., Weston, A., Lines, R., & Pettingill, P. (1999). Outcome measurement in children’s mental health. Report commissioned by the Ontario Ministry of Community and Social Services. Toronto, ON.
Roy, K. M., Roberts, M. C., Vernberg, E. M., & Randall, C. J. (2008). Measuring treatment outcome for children with serious emotional disturbances: Discriminant validity and clinical significance of the Child and Adolescent Functioning Assessment Scale. Journal of Child and Family Studies, 17, 232–240. CrossRef
Vernberg, E. M., Roberts, M. C., & Nyre, J. E. (2008). The Intensive Mental Health Program: Development and structure of the model of intervention for children with serious emotional disturbances. Journal of Child and Family Studies, 17, 169–177. CrossRef
Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003). Improving the effects of psychotherapy: The use of early identification of treatment and problem-solving strategies in routine practice. Journal of Counseling Psychology, 50, 59–68. CrossRef
Wilson, M., & Case, H. (2000). An examination of variation in rater severity over time: A study in rater drift. In M. Wilson & G. Engelhard (Eds.), Objective measurement: Theory into practice (Vol. 5, pp. 113–133). Stamford, CT: Ablex.
- Training and Maintaining System-Wide Reliability in Outcome Management
Melanie A. Barwick
Diana J. Urajnik
Julia E. Moore
- Springer US