Abstract
The assessment literature cites several instruments used to assess the skills of children with an autism spectrum disorder (ASD) diagnosis, but many lack adequate empirical support for their psychometric properties. The Assessment of Basic Language and Learning Skills-Revised (ABLLS-R) is a popular assessment used by clinicians to measure the skills of children with ASD. Despite its widespread use, the ABLLS-R contains limited research on its psychometric properties. The current study sought to extend the recent research on the psychometric properties of the ABLLS-R by using data obtained from two separate panels of expert raters to evaluate its content validity and the inter-rater reliability of its scores. Our results demonstrate evidence of content validity as at least five out of six expert panel members rated 441 out of the 544 ABLLS-R items (or 81% of the items in the assessment) as “essential.” We also found evidence of excellent inter-rater reliability (intraclass correlation coefficient = .95, p < .001) across the ABLLS-R scores obtained from a second panel of expert raters. These findings extend the existing literature and further document the ABLLS-R as a valid instrument that yields reliable scores.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aman, M.G., Novotny, S., Samango-Sprouse, C., Lecavalier, L., Leonard, E., Gadow, K.D…Chez, M. (2004). Outcome measures for clinical drug trials in autism. CNS Spectrums, 9, 36–47. https://doi.org/10.1017/S1092852900008348.
American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing. (2014). Standards for educational and psychological testing. Washington, DC: AERA.
American Medical Association. (2014). CPT assistant. Chicago: American Medical Association.
Center for Disease Control and Prevention (2015). Data and statistics. Retrieved from http://www.cdc.gov/NCBDDD/autism/data.html
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. https://doi.org/10.1037/1040-3590.6.4.284.
Every Student Succeeds Act, Pub. L. No. 114–95 § 114 Stat. 1177 (2015).
Guldberg, K. (2010). Educating children on the autism spectrum: Preconditions for inclusion and notions of “best autism practice” in the early years. British Journal of Special Education, 37, 168–170. https://doi.org/10.1111/j.1467-8578.2010.00482.x.
Individuals with Disabilities Education Act, 20 U.S.C. § 1400 (2004).
Landers, R. N. (2011). Computing intraclass correlations (ICC) as estimates of inter-rater reliability in SPSS. Retrieved from http://neoacademic.com/2011/11/16/computing-intraclass-correlations-icc-as-estimates-of-inter-rater-reliability-in-spss/#.U6z9uChhmGl. doi:10.15200/winn.143518.81744.
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x.
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35, 382–385. https://doi.org/10.1097/00006199-198611000-00017.
Malkin, A., Dixon, M. R., Speelman, R. C., & Luke, N. (2016). Evaluating the relationships between the PEAK Relations Training System-Direct Training Module, Assessment of Basic Language and Learning Skills-Revised, and the Vineland Adaptive Behavior Scales-II. Journal of Developmental and Physical Disabilities. https://doi.org/10.1007/s10882-016-9527-8.
Partington, J. W. (2010a). The Assessment of Basic Language and Learning Skills-Revised. Pleasant Hill: Behavior Analysts, Inc..
Partington, J. W. (2010b). The Assessment of Basic Language and Learning Skills-Revised: Scoring Instructions and IEP Development Guide. Pleasant Hill: Behavior Analysts, Inc..
Partington, J. W., Bailey, A., & Partington, S. W. (2016). A pilot study examining the test-retest and internal consistency reliability of the ABLLS-R. Journal of Psychoeducational Assessment. https://doi.org/10.1177/0734282916678348.
Salkind, N. (2013). Encyclopedia of research design: Content validity. Thousand Oaks: SAGE Publications, Inc..
Schwartz, I. S., Boulware, G., McBride, B. J., & Sandall, S. R. (2001). Functional assessment strategies for young children with autism. Focus on Autism and Other Developmental Disabilities, 16, 222–227. https://doi.org/10.1177/108835760101600404.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. https://doi.org/10.1037/0033-2909.86.2.420.
Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). Vineland Adaptive Behavior Scales (2nd ed.). Bloomington, MN: Pearson. https://doi.org/10.1037/t15164-000.
Sundberg, M. (2008). VB-MAPP Verbal Behavior Milestones Assessment and Placement Program: A language and social skills assessment program for children with autism or other developmental disabilities: Guide. Concord: AVB Press.
Thompson, T. (2007). Making Sense of Autism. Baltimore: Paul H. Brookes Publishing Co..
Thompson, T. (2011). Individualized Autism Intervention for Young Children. Baltimore: Paul H. Brookes Publishing Co..
Wilson, F. R., Pan, W., & Schumsky, D. A. (2012). Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development, 45, 197–210. https://doi.org/10.1177/0748175612440286.
Wolak, M. E., Fairbairn, D. J., & Paulsen, Y. R. (2012). Guidelines for estimating repeatability. Methods in Ecology and Evolution, 3, 129–137.
Funding
This study did not receive any funding.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Conflict of Interest
The first and second authors declare that they have no conflict of interest. The third author owns stock in the company that publishes the ABLLS-R.
Rights and permissions
About this article
Cite this article
Usry, J., Partington, S.W. & Partington, J.W. Using Expert Panels to Examine the Content Validity and Inter-Rater Reliability of the ABLLS-R . J Dev Phys Disabil 30, 27–38 (2018). https://doi.org/10.1007/s10882-017-9574-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10882-017-9574-9