Skip to main content
Top
Gepubliceerd in:

05-04-2023 | Original Paper

Reliability of the Commonly Used and Newly-Developed Autism Measures

Auteurs: Thomas W. Frazier, Andrew J. O. Whitehouse, Susan R. Leekam, Sarah J. Carrington, Gail A. Alvares, David W. Evans, Antonio Y. Hardan, Mirko Uljarević

Gepubliceerd in: Journal of Autism and Developmental Disorders | Uitgave 6/2024

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Purpose

The aim of the present study was to compare scale and conditional reliability derived from item response theory analyses among the most commonly used, as well as several newly developed, observation, interview, and parent-report autism instruments.

Methods

When available, data sets were combined to facilitate large sample evaluation. Scale reliability (internal consistency, average corrected item-total correlations, and model reliability) and conditional reliability estimates were computed for total scores and for measure subscales.

Results

Generally good to excellent scale reliability was observed for total scores for all measures, scale reliability was weaker for RRB subscales of the ADOS and ADI-R, reflecting the relatively small number of items for these measures. For diagnostic measures, conditional reliability tended to be very good (> 0.80) in the regions of the latent trait where ASD and non-ASD developmental disability cases would be differentiated. For parent-report scales, conditional reliability of total scores tended to be excellent (> 0.90) across very wide ranges of autism symptom levels, with a few notable exceptions.

Conclusions

These findings support the use of all of the clinical observation, interview, and parent-report autism symptom measures examined, but also suggest specific limitations that warrant consideration when choosing measures for specific clinical or research applications.
Literatuur
go back to reference Alexander, L. M., Escalera, J., Ai, L., Andreotti, C., Febre, K., Mangone, A., Vega Potler, N., Langer, N., Alexander, A., Kovacs, M., Litke, S., O’Hagan, B., Andersen, J., Bronstein, B., Bui, A., Bushey, M., Butler, H., Castagna, V., Camacho, N., Chan, E., Citera, D., Clucas, J., Cohen, S., Dufek, S., Eaves, M., & Gregory, C. (2017). The healthy Brain Network Biobank: An open resource for transdiagnostic research in pediatric mental health and learning disorders. Cold Spring Harbor Laboratory. https://doi.org/10.1101/149369.CrossRef Alexander, L. M., Escalera, J., Ai, L., Andreotti, C., Febre, K., Mangone, A., Vega Potler, N., Langer, N., Alexander, A., Kovacs, M., Litke, S., O’Hagan, B., Andersen, J., Bronstein, B., Bui, A., Bushey, M., Butler, H., Castagna, V., Camacho, N., Chan, E., Citera, D., Clucas, J., Cohen, S., Dufek, S., Eaves, M., & Gregory, C. (2017). The healthy Brain Network Biobank: An open resource for transdiagnostic research in pediatric mental health and learning disorders. Cold Spring Harbor Laboratory. https://​doi.​org/​10.​1101/​149369.CrossRef
go back to reference Carrington, S., Leekam, S., Kent, R., Maljaars, J., Gould, J., Wing, L., Le Couteur, A., Van Berckelaer-Onnes, I., & Noens, I. (2015). Signposting for diagnosis of Autism Spectrum disorder using the diagnostic interview for Social and Communication Disorders (DISCO). Research in Autism Spectrum Disorders, 9, 45–52. https://doi.org/10.1016/j.rasd.2014.10.003.CrossRef Carrington, S., Leekam, S., Kent, R., Maljaars, J., Gould, J., Wing, L., Le Couteur, A., Van Berckelaer-Onnes, I., & Noens, I. (2015). Signposting for diagnosis of Autism Spectrum disorder using the diagnostic interview for Social and Communication Disorders (DISCO). Research in Autism Spectrum Disorders, 9, 45–52. https://​doi.​org/​10.​1016/​j.​rasd.​2014.​10.​003.CrossRef
go back to reference Carrington, S. J., Barrett, S. L., Sivagamasundari, U., Fretwell, C., Noens, I., Maljaars, J., & Leekam, S. R. (2019). Describing the Profile of Diagnostic features in autistic adults using an Abbreviated Version of the diagnostic interview for Social and Communication Disorders (DISCO-Abbreviated). Journal of Autism and Developmental Disorders, 49(12), 5036–5046. https://doi.org/10.1007/s10803-019-04214-7.CrossRefPubMedPubMedCentral Carrington, S. J., Barrett, S. L., Sivagamasundari, U., Fretwell, C., Noens, I., Maljaars, J., & Leekam, S. R. (2019). Describing the Profile of Diagnostic features in autistic adults using an Abbreviated Version of the diagnostic interview for Social and Communication Disorders (DISCO-Abbreviated). Journal of Autism and Developmental Disorders, 49(12), 5036–5046. https://​doi.​org/​10.​1007/​s10803-019-04214-7.CrossRefPubMedPubMedCentral
go back to reference Constantino, J. N., & Gruber, C. P. (2012). The social responsiveness scale manual, second edition (SRS-2). Western Psychological Services. Constantino, J. N., & Gruber, C. P. (2012). The social responsiveness scale manual, second edition (SRS-2). Western Psychological Services.
go back to reference Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates.
go back to reference Frazier, T. W., Dimitropoulos, A., Abbeduto, L., Armstrong-Brine, M., Kralovic, S., Shih, A., Hardan, A. Y., Youngstrom, E. A., & Uljarevic, M. (2023). The Autism Symptom Dimensions Questionnaire: Development and psychometric evaluation of a new, open-source measure of autism symptomatology. Developmental Medicine and Child Neurology. https://doi.org/10.1111/dmcn.15497. & Quadrant Biosciences - As You Are Frazier, T. W., Dimitropoulos, A., Abbeduto, L., Armstrong-Brine, M., Kralovic, S., Shih, A., Hardan, A. Y., Youngstrom, E. A., & Uljarevic, M. (2023). The Autism Symptom Dimensions Questionnaire: Development and psychometric evaluation of a new, open-source measure of autism symptomatology. Developmental Medicine and Child Neurology. https://​doi.​org/​10.​1111/​dmcn.​15497. & Quadrant Biosciences - As You Are
go back to reference Geschwind, D. H., Sowinski, J., Lord, C., Iversen, P., Shestack, J., Jones, P., Ducat, L., & Spence, S. (2001). The autism genetic resource exchange: A resource for the study of autism and related neuropsychiatric conditions. American Journal of Human Genetics, 69, 463–466.CrossRefPubMedPubMedCentral Geschwind, D. H., Sowinski, J., Lord, C., Iversen, P., Shestack, J., Jones, P., Ducat, L., & Spence, S. (2001). The autism genetic resource exchange: A resource for the study of autism and related neuropsychiatric conditions. American Journal of Human Genetics, 69, 463–466.CrossRefPubMedPubMedCentral
go back to reference Grzadzinski, R., Carr, T., Colombi, C., McGuire, K., Dufek, S., Pickles, A., & Lord, C. (2016). Measuring changes in Social Communication Behaviors: Preliminary Development of the brief Observation of Social Communication Change (BOSCC). Journal of Autism and Developmental Disorders, 46(7), 2464–2479. https://doi.org/10.1007/s10803-016-2782-9.CrossRefPubMed Grzadzinski, R., Carr, T., Colombi, C., McGuire, K., Dufek, S., Pickles, A., & Lord, C. (2016). Measuring changes in Social Communication Behaviors: Preliminary Development of the brief Observation of Social Communication Change (BOSCC). Journal of Autism and Developmental Disorders, 46(7), 2464–2479. https://​doi.​org/​10.​1007/​s10803-016-2782-9.CrossRefPubMed
go back to reference Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Sage. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Sage.
go back to reference Hayes, A. F., & Coutts, J. J. (2020). Use omega rather than Cronbach’s alpha for estimating reliability. But… Communication Methods and Measures, 14(1), 1–24.CrossRef Hayes, A. F., & Coutts, J. J. (2020). Use omega rather than Cronbach’s alpha for estimating reliability. But… Communication Methods and Measures, 14(1), 1–24.CrossRef
go back to reference IBM Corp (2021). IBM SPSS Statistics for Windows. In (Version 28.0) IBM Corp. IBM Corp (2021). IBM SPSS Statistics for Windows. In (Version 28.0) IBM Corp.
go back to reference Kanne, S. M., Mazurek, M. O., Sikora, D., Bellando, J., Branum-Martin, L., Handen, B., Katz, T., Freedman, B., Powell, M. P., & Warren, Z. (2014). The Autism Impact measure (AIM): Initial development of a new tool for treatment outcome measurement. Journal of Autism and Developmental Disorders, 44(1), 168–179. https://doi.org/10.1007/s10803-013-1862-3.CrossRefPubMed Kanne, S. M., Mazurek, M. O., Sikora, D., Bellando, J., Branum-Martin, L., Handen, B., Katz, T., Freedman, B., Powell, M. P., & Warren, Z. (2014). The Autism Impact measure (AIM): Initial development of a new tool for treatment outcome measurement. Journal of Autism and Developmental Disorders, 44(1), 168–179. https://​doi.​org/​10.​1007/​s10803-013-1862-3.CrossRefPubMed
go back to reference Kent, R. G., Carrington, S. J., Le Couteur, A., Gould, J., Wing, L., Maljaars, J., Noens, I., van Berckelaer-Onnes, I., & Leekam, S. R. (2013). Diagnosing autism spectrum disorder: Who will get a DSM-5 diagnosis? Journal of Child Psychology and Psychiatry and Allied Disciplines, 54(11), 1242–1250. https://doi.org/10.1111/jcpp.12085.CrossRefPubMed Kent, R. G., Carrington, S. J., Le Couteur, A., Gould, J., Wing, L., Maljaars, J., Noens, I., van Berckelaer-Onnes, I., & Leekam, S. R. (2013). Diagnosing autism spectrum disorder: Who will get a DSM-5 diagnosis? Journal of Child Psychology and Psychiatry and Allied Disciplines, 54(11), 1242–1250. https://​doi.​org/​10.​1111/​jcpp.​12085.CrossRefPubMed
go back to reference Leekam, S., Libby, S., Wing, L., Gould, J., & Gillberg, C. (2000). Comparison of ICD-10 and Gillberg’s criteria for Asperger syndrome. Autism, 4(1), 11–28.CrossRef Leekam, S., Libby, S., Wing, L., Gould, J., & Gillberg, C. (2000). Comparison of ICD-10 and Gillberg’s criteria for Asperger syndrome. Autism, 4(1), 11–28.CrossRef
go back to reference Leekam, S. R., Libby, S. J., Wing, L., Gould, J., & Taylor, C. (2002a). The diagnostic interview for Social and Communication Disorders: Algorithms for ICD-10 childhood autism and Wing and Gould autistic spectrum disorder. Journal of Child Psychology and Psychiatry and Allied Disciplines, 43(3), 327–342. https://doi.org/10.1111/1469-7610.00024.CrossRefPubMed Leekam, S. R., Libby, S. J., Wing, L., Gould, J., & Taylor, C. (2002a). The diagnostic interview for Social and Communication Disorders: Algorithms for ICD-10 childhood autism and Wing and Gould autistic spectrum disorder. Journal of Child Psychology and Psychiatry and Allied Disciplines, 43(3), 327–342. https://​doi.​org/​10.​1111/​1469-7610.​00024.CrossRefPubMed
go back to reference Lidstone, J., Uljarevic, M., Sullivan, J. P., Rodgers, J., McConachie, H., Freeston, M., Le Couteur, A., Prior, M., & Leekam, S. (2014). Relations among restricted and repetitive behviors, anxiety and sensory features in children with autism spectrum disorders. Research in Autism Spectrum Disorders, 8(2), 82–92.CrossRef Lidstone, J., Uljarevic, M., Sullivan, J. P., Rodgers, J., McConachie, H., Freeston, M., Le Couteur, A., Prior, M., & Leekam, S. (2014). Relations among restricted and repetitive behviors, anxiety and sensory features in children with autism spectrum disorders. Research in Autism Spectrum Disorders, 8(2), 82–92.CrossRef
go back to reference Lord, C., Pickles, A., McLennan, J., Rutter, M., Bregman, J., Folstein, S., Fombonne, E., Leboyer, M., & Minshew, N. (1997). Diagnosing autism: Analyses of data from the Autism Diagnostic interview. Journal of Autism and Developmental Disorders, 27(5), 501–517.CrossRefPubMed Lord, C., Pickles, A., McLennan, J., Rutter, M., Bregman, J., Folstein, S., Fombonne, E., Leboyer, M., & Minshew, N. (1997). Diagnosing autism: Analyses of data from the Autism Diagnostic interview. Journal of Autism and Developmental Disorders, 27(5), 501–517.CrossRefPubMed
go back to reference Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (2002). Autism Diagnostic Observation schedule: ADOS manual. Western Psychological Services. Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (2002). Autism Diagnostic Observation schedule: ADOS manual. Western Psychological Services.
go back to reference Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. L. (2012). Autism Diagnostic Observation schedule, Second Edition (ADOS-2) manual (part 1): Modules 1–4. Western Psychological Services. Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. L. (2012). Autism Diagnostic Observation schedule, Second Edition (ADOS-2) manual (part 1): Modules 1–4. Western Psychological Services.
go back to reference Lord, C., Rutter, M., & LeCouteur, A. (1994). ADI-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 569–685.CrossRef Lord, C., Rutter, M., & LeCouteur, A. (1994). ADI-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 569–685.CrossRef
go back to reference Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus User’s Guide. Eighth Edition Muthén & Muthén. Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus User’s Guide. Eighth Edition Muthén & Muthén.
go back to reference Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill, Inc. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill, Inc.
go back to reference Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114(3), 552–566.CrossRefPubMed Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114(3), 552–566.CrossRefPubMed
go back to reference Rutter, M., Bailey, A., & Lord, C. (2003). The Social Communication Questionnaire Manual. Western Psychological Services. Rutter, M., Bailey, A., & Lord, C. (2003). The Social Communication Questionnaire Manual. Western Psychological Services.
go back to reference Schopler, E., Van Bourgondien, M., Wellman, G., & Love, S. (2010). Childhood Autism Rating Scale – 2nd Edition. Western Psychological Services. Schopler, E., Van Bourgondien, M., Wellman, G., & Love, S. (2010). Childhood Autism Rating Scale – 2nd Edition. Western Psychological Services.
go back to reference Streiner, D. L., & Norman, G. R. (1995). Health Measurement Scales: A practical guide to their development and use (2nd ed.). Oxford University Press. Streiner, D. L., & Norman, G. R. (1995). Health Measurement Scales: A practical guide to their development and use (2nd ed.). Oxford University Press.
go back to reference Taylor, B. P., Liu, J., Mowrey, W., Eule, E., Bolognani, F., & Hollander, E. (2022). Development, administration, reliability, and validity in child and adult Autism Spectrum Disorder (ASD). Journal of Psychiatric Research, 147, 142–147. https://doi.org/10.1016/j.jpsychires.2021.12.055. The Montefiore-Einstein Rigidity Scale-Revised (MERS-R). Taylor, B. P., Liu, J., Mowrey, W., Eule, E., Bolognani, F., & Hollander, E. (2022). Development, administration, reliability, and validity in child and adult Autism Spectrum Disorder (ASD). Journal of Psychiatric Research, 147, 142–147. https://​doi.​org/​10.​1016/​j.​jpsychires.​2021.​12.​055. The Montefiore-Einstein Rigidity Scale-Revised (MERS-R).
go back to reference Thissen, D. (2000). Reliability and measurement precision (2nd ed.). Lawrence Erlbaum Associates. Thissen, D. (2000). Reliability and measurement precision (2nd ed.). Lawrence Erlbaum Associates.
go back to reference Uljarevic, M., Arnott, B., Carrington, S. J., Meins, E., Fernyhough, C., McConachie, H., Le Couteur, A., & Leekam, S. R. (2017). Development of restricted and repetitive behaviors from 15 to 77 months: Stability of two distinct subtypes? Developmental Psychology, 53(10), 1859–1868. https://doi.org/10.1037/dev0000324.CrossRefPubMed Uljarevic, M., Arnott, B., Carrington, S. J., Meins, E., Fernyhough, C., McConachie, H., Le Couteur, A., & Leekam, S. R. (2017). Development of restricted and repetitive behaviors from 15 to 77 months: Stability of two distinct subtypes? Developmental Psychology, 53(10), 1859–1868. https://​doi.​org/​10.​1037/​dev0000324.CrossRefPubMed
go back to reference Uljarević, M., Frazier, T. W., Jo, B., Scahill, L., Youngstrom, E. A., Spackman, E., Phillips, J. M., Billingham, W., & Hardan, A. Y. (2022). Dimensional Assessment of Restricted and repetitive behaviors: Development and preliminary validation of a new measure. Journal of American Academy of Child and Adolescent Psychiatry. https://doi.org/10.1016/j.jaac.2022.07.863.CrossRef Uljarević, M., Frazier, T. W., Jo, B., Scahill, L., Youngstrom, E. A., Spackman, E., Phillips, J. M., Billingham, W., & Hardan, A. Y. (2022). Dimensional Assessment of Restricted and repetitive behaviors: Development and preliminary validation of a new measure. Journal of American Academy of Child and Adolescent Psychiatry. https://​doi.​org/​10.​1016/​j.​jaac.​2022.​07.​863.CrossRef
go back to reference Uljarevic, M., Frazier, T. W., Phillips, J. M., Jo, B., Littlefield, S., & Hardan, A. Y. (2020). Quantifying Research Domain Criteria Social Communication Subconstructs using the Social Communication Questionnaire in Youth. Journal Of Clinical Child And Adolescent Psychology : The Official Journal For The Society Of Clinical Child And Adolescent Psychology, American Psychological Association, Division 53, 1–11. https://doi.org/10.1080/15374416.2019.1669156. Uljarevic, M., Frazier, T. W., Phillips, J. M., Jo, B., Littlefield, S., & Hardan, A. Y. (2020). Quantifying Research Domain Criteria Social Communication Subconstructs using the Social Communication Questionnaire in Youth. Journal Of Clinical Child And Adolescent Psychology : The Official Journal For The Society Of Clinical Child And Adolescent Psychology, American Psychological Association, Division 53, 1–11. https://​doi.​org/​10.​1080/​15374416.​2019.​1669156.
go back to reference Uljarević, M., Jo, B., Frazier, T. W., Scahill, L., Youngstrom, E. A., & Hardan, A. Y. (2021). Using the big data approach to clarify the structure of restricted repetitive behaviors across the most commonly used autism spectrum disorder measures. Molecular Autism. Uljarević, M., Jo, B., Frazier, T. W., Scahill, L., Youngstrom, E. A., & Hardan, A. Y. (2021). Using the big data approach to clarify the structure of restricted repetitive behaviors across the most commonly used autism spectrum disorder measures. Molecular Autism.
go back to reference Wetherby, A. M., & Prizant, B. M. (2003). Communication and symbolic behavior scales (CSBS), Normed Edition. Paul A. Brookes Publishing Co. Wetherby, A. M., & Prizant, B. M. (2003). Communication and symbolic behavior scales (CSBS), Normed Edition. Paul A. Brookes Publishing Co.
Metagegevens
Titel
Reliability of the Commonly Used and Newly-Developed Autism Measures
Auteurs
Thomas W. Frazier
Andrew J. O. Whitehouse
Susan R. Leekam
Sarah J. Carrington
Gail A. Alvares
David W. Evans
Antonio Y. Hardan
Mirko Uljarević
Publicatiedatum
05-04-2023
Uitgeverij
Springer US
Gepubliceerd in
Journal of Autism and Developmental Disorders / Uitgave 6/2024
Print ISSN: 0162-3257
Elektronisch ISSN: 1573-3432
DOI
https://doi.org/10.1007/s10803-023-05967-y