Swipe om te navigeren naar een ander artikel
The Childhood Autism Rating Scale (CARS) is a popular behavior-observation instrument that was developed more than 34 years ago and has since been adopted in a wide variety of contexts for assessing the presence and severity of autism symptomatology in both children and adolescents. This investigation of the reliability of CARS scores involves meta-analysis and meta-regression of empirical data from reports of original research that made use of CARS between 1980 and 2012. Findings of good internal consistency (.896, 95 % CI .877–.913) and good interrater reliability (.796, 95 % CI .736–.844) support use of CARS at least in early-phase, exploratory research. Evidence of heterogeneity among literature data indicates that reliability is a property of CARS scores and is not intrinsic to the instrument itself. As the first of its kind pertaining to autism, this investigation provides guidance for reviews of other instruments’ ratings.
Log in om toegang te krijgen
Met onderstaand(e) abonnement(en) heeft u direct toegang:
Arizona Software. (2008). GraphClick graph-digitization software: Release 3.0 [computer software]. Neuchâtel: Arizona Software.
Baghdadli, A., Assouline, B., Sonié, S., Pernon, E., Darrou, C., Michelon, C., et al. (2012). Developmental trajectories of adaptive behaviors from early childhood to adolescence in a cohort of 152 children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42, 1314–1325. PubMedCrossRef
Bornmann, L., Mutz, R., & Daniel, H. D. (2010). A reliability-generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS One, 5, 1–10.
Botella, J., Suero, M., & Gambara, H. (2010). Psychometric inferences from a meta-analysis of reliability and internal consistency coefficients. Psychological Methods, 4, 386–397. CrossRef
Campbell, J. S. (2003). Meta-analysis and reliability generalization of the Psychopathy Checklist applied to youth. Unpublished doctoral dissertation from University of Northern Colorado.
Chen, G. M., Yoder, J. J., Ganzei, B. L., Goodwin, M. S., & Belmonte, M. K. (2012). Harnessing repetitive behaviours to engage attention and learning in a novel therapy for autism: An exploratory analysis. Frontiers in Psychology, doi: 10.3389/fpsyg.2012.00012.
Chez, M. G., & Buchanan, C. P. (2000). Comment: Reply: Reply to B. Rimland’s “Comments on ‘Secretin and autism: A two-part clinical investigation’”. Journal of Autism and Developmental Disorders, 30, 97–98. CrossRef
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. CrossRef
Connelly, B. S. (2008). The reliability, convergence, and predictive validity of personality ratings: An other perspective. Unpublished doctoral dissertation from University of Minnesota.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. CrossRef
Duhachek, A., & Iacobucci, D. (2004). Alpha’s standard error (ASE): An accurate and precise confidence interval estimate. Journal of Applied Psychology, 5, 792–808. CrossRef
Hakstian, A. R., & Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. Psychometrika, 41, 219–231. CrossRef
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1, 39–65. CrossRef
Howell, R. T., & Shields, A. L. (2008). The file drawer problem in reliability generalization: A strategy to compute fail-safe N with reliability coefficients. Educational and Psychological Measurement, 68, 120–128. CrossRef
Konstantareas, M. M., Homatidis, S., & Plowright, C. M. S. (1992). Assessing resources and stress in parents of severely dysfunctional children through the Clarke modification of Holroyd’s Questionnaire on Resources and Stress. Journal of Autism and Developmental Disorders, 22, 217–234. PubMedCrossRef
Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9, 202–220. CrossRef
LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852. CrossRef
Li, A., & Bagger, J. (2007). The balanced inventory of desirable responding (BIDR): A reliability generalization study. Educational and Psychological Measurement, 67, 525–544. CrossRef
Nassen, L. M. (2008). An internal consistency reliability generalization of the Child Behavior Checklist. Unpublished doctoral dissertation from University of Northern Colorado.
Pereira, T. V., Patsopoulos, N. A., Salanti, G., & Ioannidis, J. P. A. (2010). Critical interpretation of Cochran’s Q test depends on power and prior assumptions about heterogeneity. Research Synthesis Methods, 1, 149–161. CrossRef
Rodriguez, M. C., & Jung, T. (2003). The replicability of reliability generalization. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL. Accessed May 27, 2011 from http://www.edmeasurement.net/aera/papers/rg%20replication%20aera.pdf.
Ryngala, D. J., Shields, A. L., & Caruso, J. C. (2005). Reliability generalization of the revised Children’s Manifest Anxiety Scale. Educational and Psychological Measurement, 65, 259–271. CrossRef
Samtani, A., Sterling-Lewis, K., Scholten, R. J. P. M., Woolfenden, S., Hooft, L., & Williams, K. (2011). Diagnostic tests for autism spectrum disorder (ASD) in preschool children. Cochrane Database for Systematic Reviews Collaboration,. doi: 10.1002/14651858.cd009044.
Sawilowsky, S. S. (2000). Psychometrics versus datametrics: Comment on Vacha-Haase’s “reliability generalization” method and some Epm [sic] editorial policies. Educational and Psychological Measurement, 60, 157–173. CrossRef
Schlosser, R. W., Wendt, O., & Sigafoos, J. (2007). Not all systematic reviews are created equal: Considerations for appraisal. Evidence-Based Communication Assessment and Intervention, 1, 138–150. CrossRef
Schopler, E., Reichler, R. J., & Renner, B. R. (1988). The Childhood Autism Rating Scale (CARS). Los Angeles, CA: Western Psychological Services.
StataCorp. (2012). Stata statistical software: Release 12.1 [computer software]. College Station, TX: StataCorp LP.
Sun, S. (2011). Meta-analysis of Cohen’s kappa. Health Services Outcomes Research Methodology, 11, 145–163. CrossRef
Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174–195.
Vacha-Haase, T., Kogan, L. R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60, 509–522. CrossRef
Vacha-Haase, T., & Thompson, B. (2011). Score reliability: A retrospective look back at 12 years of reliability generalization studies. Measurement and Evaluation in Counseling and Development, 44, 159–168. CrossRef
Vassar, M., & Hale, W. (2007). Reliability reporting practices in youth life satisfaction research. Social Indicators Research, 83, 487–496. CrossRef
Vassar, M., Ridge, J. W., & Hill, A. D. (2008). Inducing score reliability from previous reports: An examination of life satisfaction studies. Social Indicators Research, 87, 27–45. CrossRef
Viding, E., & Blakemore, S. J. (2007). Endophenotype approach to developmental psychopathology: Implications for autism research. Behavioral Genetics, 37, 51–60. CrossRef
Wilkinson, L., & American Psychological Association (APA) Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594–604.
Woolfenden, S., Sarkozy, V., Ridley, G., & Williams, K. (2012). A systematic review of the diagnostic stability of autism spectrum disorder. Research in Autism Spectrum Disorders, 6, 345–354. CrossRef
- Reliability Generalization for Childhood Autism Rating Scale
Tim J. Croudace
- Springer US