Abstract
This paper describes the reliability and validity of the service assessment for children and adolescents (SACA) for use among Spanish-speaking respondents. The test-retest reliability of the instrument was assessed in a randomly selected clinical sample of 146 Puerto Rican children and adolescents aged 4–17. Both parents and children were administered the SACA twice by independent interviewers over an average 12-day follow-up period. The accuracy of parental and youth self-reports was assessed by comparing these reports to information obtained from medical records. The results showed that parents and children (aged 11–17) were able to report with fair to moderate reliability any last year use of mental health services, any outpatient mental health services, and school services. Residential and hospitalization services were reported by both informants with substantial test-retest reliability. Slight or no test-retest reliability was obtained for parent and child on the use of the specific type of mental health professionals, as well as parental reports of several treatment modalities. Substantial sensitivity of the SACA was obtained when comparing medical records to parental and child reports to lifetime use of any service and outpatient mental health service. Moderate sensitivity was obtained for last year use of mental health services for both parent and child informants.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Angold, A., Costello, E. J., Burns, B. J., Erkanli, A., & Farmer, E. M. Z. (2000). Effectiveness of nonresidential specialty mental health services for children and adolescents in the “Real World.” Journal of the American Academy of Child and Adolescent Psychiatry, 39, 154-160.
Ascher, B. H., Farmer, E. M. Z., Burns, B. J., & Angold, A. (1996). The child and adolescent services assessment (CASA): Description and psychometrics. Journal of Emotional and Behavioral Disorders, 4, 12-20.
Bravo, M., Ribera, J., Rubio-Stipec, M., Canino, G., Shrout, P., Alegría, M., et al. (2001). Test-retest reliability of the Spanish version of the diagnostic interview schedule for children (DISC-IV). Journal of Abnormal Child Psychology, 29, 433-444.
Bravo, M., Woodbury-Fariña, M., Canino, G., & Rubio-Stipec, M. (1993). The Spanish translation and cultural adaptation of the diagnostic interview schedule for children (DISC) in Puerto Rico. Culture, Medicine and Psychiatry, 17, 329-344.
Canino, G., Bravo, M., Ramírez, R., Febo, V. E., Rubio-Stipec, M., Lewis-Fernández, R., et al. (1999). The spanish alcohol use disorder and associated disabilities interview schedule (AUDADIS): Reliability and concordance with clinical diagnoses in a Hispanic population. Journal of Studies on Alcohol, 60, 790-799.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
Donner, A., & Eliasziw, M. (1992). A goodness-of-fit approach to inference procedures for the kappa statistic: Confidence interval construction, significance-testing and sample size estimation. Statistics in Medicine, 11, 1511-1519.
Farmer, E. M. Z., Angold, A., Burns, B. J., & Costello, E. J. (1994). Reliability of self-reported service use: Test-retest consistency of children's responses to the child and adolescent services assessment (CASA). Journal of Child and Family Studies, 3, 307-325.
Fendrich, M., Johnson, T., Wislar, J. S., & Jageotte, C. (1999). Accuracy of parent mental health service reporting: Results from a reverse record-check study. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 147-155.
Flaherty, J. A. (1987). Appropriate and inappropriate research methodologies for Hispanic mental health. In M. Gaviria (Ed.), Health and behavior: Research agenda for Hispanics (pp. 177-186). Chicago: University of Illinois Press.
Goodman, S. H., Hoven, C. W., Narrow, W. E., Cohen, P., Fieldfing, B., Alegria, M., et al. (1998). Measurement of risk for mental disorders and competence in a psychiatric epidemiologic community survey: The national institute of mental health methods for the epidemiology of child and adolescent mental disorders (MECA) study. Social Psychiatry and Psychiatric Epidemiology, 33, 162-173.
Hoagwood, K. (2000). Commentary: The dose effect in children's mental health services. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 172-175.
Hoagwood, K., Horwitz, S. M., Stiffman, A. R., et al. (2000). Concordance between parent reports of children's mental health services and services records: The services assessment for children and adolescents (SACA). Journal of Child and Family Services, 9, 315-331.
Hoagwood, K., Jensen, P., Petti, T., & Burns, B. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 1055-1063.
Horwitz, S. M., Hoagwood, K., Stiffman, A. R., Summerfelt, T., Weisz, J. R., & Costello, E. J. (in press). Measuring youth's use of mental health services: Reliability of the services assessment for children and adolescents (SACA). Psychiatric Services.
Jensen, P., Roper, M., Fisher, P., Piacentini, J., Canino, G., Richters, J., et al. (1995). Test-retest reliability of the diagnostic interview schedule for children (DISC 2.1). Archives of General Psychiatry, 52, 61-71.
Leaf, P. J., Alegría, M., Cohen, P., Goodman, S. H., Horwitz, S. M., Hoven, C. W., et al. (1996). Mental health service use in the community and schools: Results from the four-community MECA study. Journal of American Academy of Child and Adolescent Psychiatry, 35, 889-897.
Lucas, C. P., Fisher, P., Piacentini, J., Zhang, H., Jensen, P. S., Shaffer, D., et al. (1999). Features of interviews questions associated with attenuation of symptom reports. Journal of Abnormal Child Psychology, 27, 429-437.
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12, 153-157.
Ribera, J.C., Canino, G. J., Rubio-Stipec, M., Bravo, M., Bird, H. R., Freeman, D., et al. (1996). The diagnostic interview schedule for children (DISC 2.1) in Spanish: Reliability in a Hispanic Population. Journal of Child Psychology and Psychiatry, 37, 195-204.
Roberts, R. E. (2000). Depression and suicidal behaviors among adolescents: The role of ethnicity. In I. Cuellar & F. H. Peniagua (Eds.), Handbook of multi-cultural mental health (pp. 360-380). San Diego, CA: Academic Press.
Shaffer, D., Fisher, P., Lucas, C. P., Dulcan, M. K., & Schwab-Stone, M. E. (2000). NIMH diagnostic interview schedule for children version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 28-38.
Shrout, P. E. (1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7, 301-317.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.
Stiffman, A. R., Horwitz, S. M., Hoagwood, K., Compton, W., III, Cottler, L., Bean, D. L., et al. (2000). The service assessment for children and adolescents (SACA): Adult and child reports. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 1032-1039.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Canino, G., Shrout, P.E., Alegría, M. et al. Methodological Challenges in Assessing Children's Mental Health Services Utilization. Ment Health Serv Res 4, 97–107 (2002). https://doi.org/10.1023/A:1015252217154
Issue Date:
DOI: https://doi.org/10.1023/A:1015252217154