Elsevier

Research in Autism Spectrum Disorders

Volume 6, Issue 4, October–December 2012, Pages 1265-1272
Research in Autism Spectrum Disorders

Validation of an interview-based rating scale developed in Japan for pervasive developmental disorders

https://doi.org/10.1016/j.rasd.2012.04.002Get rights and content

Abstract

The pervasive developmental disorders (PDDs) Autism Society Japan Rating Scale (PARS), an interview-based instrument for evaluating PDDs, has been developed in Japan with the aim of providing a method that (1) can be used to evaluate PDD symptoms and related support needs and (2) is simpler and easier than the currently used “gold standard” instruments such as the Autism Diagnostic Interview-Revised (ADI-R). We examined the reliability and validity of PARS on the basis of data from 572 participants (277 PDD patients and 295 nonclinical controls). Inter-rater reliability was sufficient at both the item and scale level. Factor analysis extracted four subscales, for which internal consistency was found to be high. The sub and total scores of PARS showed correlations with the domain and total scores of ADI-R, in line with theoretical prediction, indicating the convergent validity of PARS. A receiver operating characteristic analysis showed that PARS has good discriminative validity in differentiating between PDD patients and nonclinical controls, regardless of intellectual capacity. Considering that PARS can be easily implemented by professionals with appropriate knowledge regarding PDDs, PARS may be superior to the existing instruments in terms of cost performance.

Highlights

PARS, a simple interview-based evaluation instrument for PDD, was validated. ► Inter-interviewer reliability of PARS was sufficient at both the item and scale level. ► Factor analysis extracted four subscales for which internal consistency was high. ► PARS subscores showed correlations with ADI-R domain scores in line with prediction. ► PARS discriminates PDD patients from normal controls as well as the ADI-R.

Introduction

Over the course of many years, several instruments have been developed for the diagnosis, evaluation, and screening of pervasive development disorders (PDD). In recent years, the Autism Diagnostic Interview-Revised (ADI-R; Le Couteur et al., 1989, Lord et al., 1994) has been broadly accepted as a standardized interview-based diagnostic instrument for PDD. The Autism Diagnostic Observational Schedule (ADOS; Lord et al., 2000, Lord et al., 1989) is also widely used as an observation-based diagnostic instrument. These instruments have a high level of discriminative validity with respect to the differentiation of PDD from non-PDD and are useful in reaching a definitive diagnosis; however, their implementation requires special training and significant time, leading to the development of numerous simpler evaluation scales in recent years.

The Modified Checklist for Autism in Toddlers (M-CHAT; Robins, Fein, Barton, & Green, 2001), which has been broadly accepted as a screening instrument, is a unique tool that comprises a combination of questionnaires, telephone interviews, and structured follow-up interviews. Although it is a highly useful tool, its use is limited to toddlers because it was developed with the aim of early identification of PDD. In countries such as Japan and other Asian countries lacking the medical and governmental services for PDD that exist in the United States and Europe, it is believed that many people with undiagnosed PDD exist in a broad age group. In fact, Kawamura, Takahashi, and Ishii (2008) reported that in Toyota City, Japan, where a new systematic PDD screening system has been implemented, there were 11 times more detections of PDD compared with that observed in a survey done 20 years ago. However, few regions in the world have an adequate PDD detection system of this kind. Considering this, the development of a simple and practical evaluation scale that can be applied to a wide age group is an important and pressing issue.

The Autism Spectrum Screening Questionnaire (ASSQ; Ehlers, Gillberg, & Wing, 1999), Autism Screening Questionnaire (ASQ; Berument, Rutter, Lord, Pickles, & Bailey, 1999), and Social Responsiveness Scale (Constantino et al., 2003) have been developed as PDD evaluation scales that can be applied to a relatively broad age group. As all of these evaluation tools are in the format of a questionnaire that can be evaluated by parents or teachers, they have the advantage of being fairly easy to implement. However, in most cases, parents lack the specialized knowledge needed to understand PDD, so the standards for rating individual items can vary greatly depending on the individual conducting the evaluation, possibly leading to a deterioration of the reliability of evaluation results. Furthermore, though teachers generally have more PDD-related knowledge than do parents, they have less specific knowledge of each individual child; hence, their evaluations tend to be less reliable than those of parents. In practice, the sensitivity (true positive rate) and specificity (one minus false positive rate) of the ASSQ in distinguishing PDD and non-PDD was .91 and .77, respectively, for the parent evaluation and .90 and .58, respectively, for the teacher evaluation (Ehlers et al., 1999). Considering that the sensitivity and specificity of the ADI-R were 1.00 and .90, respectively (Lord et al., 1997), the level of accuracy of the ASSQ in distinguishing PDD from non-PDD was insufficient in the hands of both parents and teachers. Furthermore, in a simultaneous comparison conducted by Charman et al. (2007), sensitivity and specificity in identifying autistic spectrum disorders was .86 and .78, respectively, for the ASQ and .78 and .67, respectively, for the SRS, thereby indicating its insufficient precision in practical use.

To resolve this dilemma between accuracy and simplicity, the PDDs Autism Society Japan Rating Scale (PARS) has been developed in Japan as an instrument for evaluating PDDs (Adachi et al., 2006, Kamio et al., 2006, Tsujii et al., 2006). This scale was developed with the aim of providing an instrument that is simpler to use than the ADI-R and ADOS; is applicable to any age group, unlike the M-CHAT; and has better reliability and validity than questionnaire scales such as the ASSQ and ASQ. While PARS uses an interview format similar to ADI-R, the procedures, which are briefly summarized in the manual, can be implemented after simple training. Furthermore, because the criteria for rating each item is clearly defined in PARS, a more reliable and valid evaluation is possible than with questionnaire scales. In order to ease the rating process and shorten the evaluation time, the evaluator assigns values at three levels—none (0 points), somewhat apparent (1 point), and apparent (2 points)—for the 34 items listed as typical behavioral symptoms of PDD. This innovation ensures that the time required to implement PARS is kept to 30–90 min, depending on the interviewer's proficiency and the target's age and symptoms.

There is no international literature on the psychometric properties of PARS, although PARS is now widely used in Japan. This study examined the reliability and validity of PARS and involved a study population of 628 test subjects that included 302 people with PDD and 326 people without PDD. Specifically, we evaluated the inter-rater reliability, factor structure, internal consistency, correlation with the ADI-R, and the ability to distinguish subjects with PDD from a nonclinical sample.

Section snippets

PARS

The PARS instrument has been developed (Adachi et al., 2006, Kamio et al., 2006, Tsujii et al., 2006) and published (PARS Committee, 2008) in Japan. It involves the evaluation of PDD symptoms through a semi-structured interview conducted with a parent or family member of the subject as the target. This tool can be used to assess not only the risk of PDD but also the need for support pertaining to administrative and medical services. PARS comprises both an evaluation of symptoms when they were

Inter-rater reliability

Spearman's rank correlation coefficients between the scores of two interviewers were significant for all items (p < .05 in item 27; p < .01 in remaining items), with an average value of .68 (SD = .11). For the total score, the Pearson's correlation coefficient between the scores of the interviewers was r = .78 (p < .01).

Factor structure and internal consistency

Table 2 shows the corrected item-total correlation for each item and the results of factor analysis. Based on a scree plot (9.25, 3.76, 2.36, 2.02, 1.68, 1.62, …) that showed a

Discussion

The objective of this study was to validate PARS, a scale developed for (1) the evaluation of PDD symptoms in a simpler manner than “gold standard” instruments, such as the ADI-R and ADOS, and (2) more objective evaluation than questionnaire scales, such as the ASSQ and ASQ. As long as the interviewer has a certain level of expertise pertaining to PDDs, PARS can be used after brief training and can be administered in an hour on an average by simplifying and structuring the interview procedure

Acknowledgement

This study was supported by a grant from the Foundation for Children's Future.

References (28)

  • J. Hattori et al.

    Are pervasive developmental disorders and attention-deficit/hyperactivity disorder distinct disorders?

    Brain & Development

    (2006)
  • J. Adachi et al.

    Reliability and validity of the childhood part of the PARS (PDD-Autism Society Japan Rating Scale)

    Rinsho Seishin Igaku (Clinical Psychiatry)

    (2006)
  • American Psychiatric Association

    Diagnostic and statistical manual of mental disorders (4th ed., Text Revision) (DSM-IV-TR)

    (2000)
  • S.K. Berument et al.

    Autism screening questionnaire: Diagnostic validity

    British Journal of Psychiatry

    (1999)
  • R.B. Cattell

    The scree test for the number of factors

    Multivariate Behavioral Research

    (1966)
  • T. Charman et al.

    Efficacy of three screening instruments in the identification of autistic-spectrum disorders

    British Journal of Psychiatry

    (2007)
  • J.N. Constantino et al.

    Validation of a brief quantitative measure of autistic traits: Comparison of the Social Responsiveness Scale with the Autism Diagnostic Interview-Revised

    Journal of Autism and Developmental Disorders

    (2003)
  • S. Ehlers et al.

    A screening questionnaire for Asperger syndrome and other high-functioning autism spectrum disorders in school age children

    Journal of Autism and Developmental Disorders

    (1999)
  • F.P. Holgado-Tello et al.

    Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables

    Quality & Quantity

    (2010)
  • Japanese WISC-III Publication Committee

    Nihonban WISCIII chinou kensahou

    (1998)
  • Y. Kamio et al.

    Reliability and validity of the pervasive developmental disorders (PDD) Autism Society Japan rating scale: A behavior checklist for adolescents and adults with PDDs

    Seishin Igaku (Psychiatry)

    (2006)
  • Kaufman et al.

    K-ABC Shinri Kyoiku Asesument Batteri [Kaufman Assessment Battery for Children]

    (1993)
  • Y. Kawamura et al.

    Reevaluating the incidence of pervasive developmental disorders: Impact of elevated rates of detection through implementation of an integrated system of screening in Toyota, Japan

    Psychiatry and Clinical Neurosciences

    (2008)
  • A. Le Couteur et al.

    Autism diagnostic interview: A standardized investigator-based instrument

    Journal of Autism and Developmental Disorders

    (1989)
  • Cited by (0)

    View full text