Validation of an interview-based rating scale developed in Japan for pervasive developmental disorders
Highlights
► PARS, a simple interview-based evaluation instrument for PDD, was validated. ► Inter-interviewer reliability of PARS was sufficient at both the item and scale level. ► Factor analysis extracted four subscales for which internal consistency was high. ► PARS subscores showed correlations with ADI-R domain scores in line with prediction. ► PARS discriminates PDD patients from normal controls as well as the ADI-R.
Introduction
Over the course of many years, several instruments have been developed for the diagnosis, evaluation, and screening of pervasive development disorders (PDD). In recent years, the Autism Diagnostic Interview-Revised (ADI-R; Le Couteur et al., 1989, Lord et al., 1994) has been broadly accepted as a standardized interview-based diagnostic instrument for PDD. The Autism Diagnostic Observational Schedule (ADOS; Lord et al., 2000, Lord et al., 1989) is also widely used as an observation-based diagnostic instrument. These instruments have a high level of discriminative validity with respect to the differentiation of PDD from non-PDD and are useful in reaching a definitive diagnosis; however, their implementation requires special training and significant time, leading to the development of numerous simpler evaluation scales in recent years.
The Modified Checklist for Autism in Toddlers (M-CHAT; Robins, Fein, Barton, & Green, 2001), which has been broadly accepted as a screening instrument, is a unique tool that comprises a combination of questionnaires, telephone interviews, and structured follow-up interviews. Although it is a highly useful tool, its use is limited to toddlers because it was developed with the aim of early identification of PDD. In countries such as Japan and other Asian countries lacking the medical and governmental services for PDD that exist in the United States and Europe, it is believed that many people with undiagnosed PDD exist in a broad age group. In fact, Kawamura, Takahashi, and Ishii (2008) reported that in Toyota City, Japan, where a new systematic PDD screening system has been implemented, there were 11 times more detections of PDD compared with that observed in a survey done 20 years ago. However, few regions in the world have an adequate PDD detection system of this kind. Considering this, the development of a simple and practical evaluation scale that can be applied to a wide age group is an important and pressing issue.
The Autism Spectrum Screening Questionnaire (ASSQ; Ehlers, Gillberg, & Wing, 1999), Autism Screening Questionnaire (ASQ; Berument, Rutter, Lord, Pickles, & Bailey, 1999), and Social Responsiveness Scale (Constantino et al., 2003) have been developed as PDD evaluation scales that can be applied to a relatively broad age group. As all of these evaluation tools are in the format of a questionnaire that can be evaluated by parents or teachers, they have the advantage of being fairly easy to implement. However, in most cases, parents lack the specialized knowledge needed to understand PDD, so the standards for rating individual items can vary greatly depending on the individual conducting the evaluation, possibly leading to a deterioration of the reliability of evaluation results. Furthermore, though teachers generally have more PDD-related knowledge than do parents, they have less specific knowledge of each individual child; hence, their evaluations tend to be less reliable than those of parents. In practice, the sensitivity (true positive rate) and specificity (one minus false positive rate) of the ASSQ in distinguishing PDD and non-PDD was .91 and .77, respectively, for the parent evaluation and .90 and .58, respectively, for the teacher evaluation (Ehlers et al., 1999). Considering that the sensitivity and specificity of the ADI-R were 1.00 and .90, respectively (Lord et al., 1997), the level of accuracy of the ASSQ in distinguishing PDD from non-PDD was insufficient in the hands of both parents and teachers. Furthermore, in a simultaneous comparison conducted by Charman et al. (2007), sensitivity and specificity in identifying autistic spectrum disorders was .86 and .78, respectively, for the ASQ and .78 and .67, respectively, for the SRS, thereby indicating its insufficient precision in practical use.
To resolve this dilemma between accuracy and simplicity, the PDDs Autism Society Japan Rating Scale (PARS) has been developed in Japan as an instrument for evaluating PDDs (Adachi et al., 2006, Kamio et al., 2006, Tsujii et al., 2006). This scale was developed with the aim of providing an instrument that is simpler to use than the ADI-R and ADOS; is applicable to any age group, unlike the M-CHAT; and has better reliability and validity than questionnaire scales such as the ASSQ and ASQ. While PARS uses an interview format similar to ADI-R, the procedures, which are briefly summarized in the manual, can be implemented after simple training. Furthermore, because the criteria for rating each item is clearly defined in PARS, a more reliable and valid evaluation is possible than with questionnaire scales. In order to ease the rating process and shorten the evaluation time, the evaluator assigns values at three levels—none (0 points), somewhat apparent (1 point), and apparent (2 points)—for the 34 items listed as typical behavioral symptoms of PDD. This innovation ensures that the time required to implement PARS is kept to 30–90 min, depending on the interviewer's proficiency and the target's age and symptoms.
There is no international literature on the psychometric properties of PARS, although PARS is now widely used in Japan. This study examined the reliability and validity of PARS and involved a study population of 628 test subjects that included 302 people with PDD and 326 people without PDD. Specifically, we evaluated the inter-rater reliability, factor structure, internal consistency, correlation with the ADI-R, and the ability to distinguish subjects with PDD from a nonclinical sample.
Section snippets
PARS
The PARS instrument has been developed (Adachi et al., 2006, Kamio et al., 2006, Tsujii et al., 2006) and published (PARS Committee, 2008) in Japan. It involves the evaluation of PDD symptoms through a semi-structured interview conducted with a parent or family member of the subject as the target. This tool can be used to assess not only the risk of PDD but also the need for support pertaining to administrative and medical services. PARS comprises both an evaluation of symptoms when they were
Inter-rater reliability
Spearman's rank correlation coefficients between the scores of two interviewers were significant for all items (p < .05 in item 27; p < .01 in remaining items), with an average value of .68 (SD = .11). For the total score, the Pearson's correlation coefficient between the scores of the interviewers was r = .78 (p < .01).
Factor structure and internal consistency
Table 2 shows the corrected item-total correlation for each item and the results of factor analysis. Based on a scree plot (9.25, 3.76, 2.36, 2.02, 1.68, 1.62, …) that showed a
Discussion
The objective of this study was to validate PARS, a scale developed for (1) the evaluation of PDD symptoms in a simpler manner than “gold standard” instruments, such as the ADI-R and ADOS, and (2) more objective evaluation than questionnaire scales, such as the ASSQ and ASQ. As long as the interviewer has a certain level of expertise pertaining to PDDs, PARS can be used after brief training and can be administered in an hour on an average by simplifying and structuring the interview procedure
Acknowledgement
This study was supported by a grant from the Foundation for Children's Future.
References (28)
- et al.
Are pervasive developmental disorders and attention-deficit/hyperactivity disorder distinct disorders?
Brain & Development
(2006) - et al.
Reliability and validity of the childhood part of the PARS (PDD-Autism Society Japan Rating Scale)
Rinsho Seishin Igaku (Clinical Psychiatry)
(2006) Diagnostic and statistical manual of mental disorders (4th ed., Text Revision) (DSM-IV-TR)
(2000)- et al.
Autism screening questionnaire: Diagnostic validity
British Journal of Psychiatry
(1999) The scree test for the number of factors
Multivariate Behavioral Research
(1966)- et al.
Efficacy of three screening instruments in the identification of autistic-spectrum disorders
British Journal of Psychiatry
(2007) - et al.
Validation of a brief quantitative measure of autistic traits: Comparison of the Social Responsiveness Scale with the Autism Diagnostic Interview-Revised
Journal of Autism and Developmental Disorders
(2003) - et al.
A screening questionnaire for Asperger syndrome and other high-functioning autism spectrum disorders in school age children
Journal of Autism and Developmental Disorders
(1999) - et al.
Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables
Quality & Quantity
(2010) Nihonban WISCIII chinou kensahou
(1998)