Diagnosing autism spectrum disorder (ASD) is a complex process that requires standardized collection of information through both child observations and parental interviews, as well as other information concerning the functioning of the child (Falkmer et al. 2013
; Ozonoff et al. 2005
). Standardized and validated instruments are available to aid professionals in this process, but can be costly to administer, since they are often time consuming and require specific expertise to correctly administer and interpret. While the use of standardized parental interviews is becoming more and more common in specialized centers (Ashwood et al. 2014
), only 50 % of UK child development teams make use of them (Palmer et al. 2010
). One of the main reasons for this relates to feasibility, i.e. the required time investment (Matson et al. 2007
). Standardized parental interviews, such as the ADI-R or DISCO (Rutter et al. 2003
; Wing et al. 2002
), commonly require up to 3 h to administer. This constitutes a significant time burden on both parents and clinicians.
To meet this need for a clinically feasible standardized and valid parental interview, the Developmental Diagnostic Dimensional Interview-short version (3Di-sv; Santosh et al. 2009
) was developed. This is a 45-min version of the original 3Di (Skuse et al. 2004
). Like its longer equivalent, the 3Di-sv is a computerized parental interview for ASD assessment. It offers dimensional scores on the three pervasive developmental disorders (PDD) domains of social reciprocity, communication and repetitive and stereotyped behavior (RSB), as defined in the DSM-IV-TR (American Psychiatric Association 2000
), as well as PDD classifications based on validated cut-off scores on these domains. Items were selected based on existing 3Di research data, and scores and classifications from the shorter interview showed excellent agreement with those on the longer version as well as ADI-R classifications (Santosh et al. 2009
). This original study found sensitivity and specificity values of over .85 when comparing 3Di classifications to clinical diagnoses.
While these results are promising, it should be noted that they were based on existing data obtained from the full interview, which were then rescored based on the new algorithm. Consequently, scores on the selected items might have been influenced by information obtained by asking additional questions as part of the longer interview that were subsequently removed in the shortening process. The validity of the 3Di-sv as a stand-alone instrument was supported by a study in Thailand (Chuthapisith et al. 2012
), which resulted in fair to good areas under the ROC curves for all three scales compared to clinical DSM-IV-TR diagnoses (.79–.89). Lai et al. (2014
) found excellent (.95) sensitivity for the Chinese translation of the 3Di-sv and fair specificity (.77). For the Dutch version, preliminary explorations pointed towards similar results (Slappendel et al. 2013
), with moderate sensitivity (.60) and fair (.75) specificity compared to clinical DSM-IV-TR diagnosis, and moderate to strong correlations (.25–.65) between 3Di-sv domain scores and scores on the Social Responsiveness Scale (SRS; Constantino and Gruber 2005
So far, research on the validity of the 3Di-sv as a standalone instrument has been performed using typically developing children (Chuthapisith et al. 2012
), or children referred for symptoms unrelated to ASD (Lai et al. 2014
) as controls. This is not representative of the reality of clinical work, where ASD specific instruments will not be used unless a child shows elevated levels of ASD symptoms according to parent and/or teacher report. Using controls who show no reasons for suspicion of ASD likely inflates sensitivity and specificity scores, by including children that would not normally be tested. Therefore, this study only included children with elevated levels of ASD symptoms as reported by parents on the SRS, indicating that parents feel their child shows significant ASD symptoms.
Beyond the regular validation concerns, the recent change from DSM-IV-TR to DSM-5 poses extra challenges. The DSM-5 draws on an increasing amount of research on the symptom domains of ASD (e.g. Boomsma et al. 2008
; Frazier et al. 2008
; Mandy et al. 2012
; Snow et al. 2009
; van Lang et al. 2006
), which has led to a reformulation of the diagnostic requirements for ASD. The model of ASD has been changed from three dimensions to a two dimensional model that merges the reciprocity and communication domains into one social communication domain. Additionally, the RSB domain has been expanded to include both stereotypical communication and sensory hyper and/or hyporeactivity. These changes have prompted the developers of several ASD assessment instruments to revise both their item content and their scoring algorithms to better address these new criteria (e.g. Carrington et al. 2014
; Kent et al. 2013
; Lord et al. 2012
). In doing so, the need to expand this research to the 3Di has already been expressed (Carrington et al. 2014
). While studies have been done to relate the full 3Di ASD interview to the DSM-5 model, with positive results (Mandy et al. 2012
), the 3Di-sv has not yet been studied with this aim. This means little is known about whether its factor structure conforms to the DSM-5 ASD model, and whether its items adequately cover the new criteria.
In order to fill the above-mentioned gaps in the literature on the 3Di-sv, the current study has three main aims. Firstly, to assess the validity of the 3Di-sv as a standalone instrument in a sample of children at high risk for ASD (aim 1). Secondly, to determine, through confirmatory factor analysis (CFA), whether the 3Di-sv fits the two factor (DSM-5) structure of ASD as well as the three factor (DSM-IV-TR) structure (aim 2). Finally, we explored content validity to clarify whether the items of the 3Di-sv represent all DSM-5 ASD criteria and exemplars (i.e. construct under or overrepresentation) (aim 3a), and subsequently explored possibilities to overcome construct underrepresentation by collecting perspectives of a panel of ASD expert clinicians frequently using the 3Di (aim 3b).
In this study, we aimed to explore the utility of the 3Di-sv, by determining its validity, examining its DSM-5 factor structure, exploring its DSM-5 construct representation and examining ways to improve construct representation.
Criterion Validity of the 3Di-sv
While the 3Di-sv showed good sensitivity compared to ADOS-2-confirmed clinical diagnoses, specificity was low. This low specificity for the 3Di-sv is atypical. Other studies (Chuthapisith et al. 2012
; Lai et al. 2014
) have found similarly high sensitivities, but found better specificity values. While we did a post hoc analysis to determine whether this might be explained by the selection of a sample of high ASD risk children, specificity in our full clinically referred sample was still low compared to other studies. The remaining difference may well be due to the stringency of our criterion. Lai et al. (2014
) and Chuthapisith et al. (2012
) both used clinical diagnoses as a criterion, not confirmed by any standardized assessment. Taking this approach would have increased the number of ASD-positive children in our sample, and thus would have increased specificity to be more in line with these other studies.
While results did not change when the sample was stratified by age or by type of center, the wide confidence intervals indicate that this may well be due to the remaining sample sizes after stratification; only 28 children under 6 were available for this analysis, and only 31 children from specialized ASD centers. Future studies might look more closely into these kinds of subgroups with larger samples, in order to learn more about the samples in which the 3Di-sv is most useful.
Factor Structure of the 3Di-sv
Confirmatory factor analyses found a better fit against a DSM-5 model than a DSM-IV-TR model of ASD. The positive results for the DSM-5 model are in line with those from previous studies using the full version of the interview (Mandy et al. 2014
, 2012), as well as studies using other measures to investigate the structure of the ASD phenotype (Boomsma et al. 2008
; Frazier et al. 2008
; Guthrie et al. 2013
; Norris et al. 2012
; Snow et al. 2009
; van Lang et al. 2006
). This suggests that the structure of the 3Di-sv is not changed by the removal of the extra items that make up the full interview, and supports the research from Santosh et al. (2009
) suggesting that the 3Di-sv is a valid alternative for the full 3Di interview in situations where time restrictions are in play. Internal consistency of the preliminary DSM-5 domains was also confirmed by Cronbach’s alpha values.
Finally, while the underlying structure of the instrument may align well with the DSM-5, the 3Di-sv does not fully cover all symptom groups described in the new ASD criteria. Sensory hyper and hyporeactivity, inflexible adherence to routines and insistence on sameness are not adequately covered by the current question set of the 3Di-sv, and repetitive language use is scored under communication rather than RSB. This is likely to lead to under inclusion because some relevant symptoms are not recognized. In fact, sensory behaviors have previously been found to be particularly discriminative for ASD caseness (Carrington et al. 2014
), making this an important gap to fill to create an optimally functional instrument.
In order to adapt the 3Di-sv to the DSM-5, we explored items that may be added to fill these gaps. Results from our questionnaire provide a good starting point for which particular items might be added, based on the perspectives of experienced, 3Di-trained clinicians specialized in ASD. However, in the longer run, we argue for an additional in-depth, data driven approach, based on data collected with the recently developed new DSM-5 3Di full version, using analytical techniques such as those used by the developers of the DISCO DSM-5 algorithm (Carrington et al. 2014
) or the AQ-10 (Allison et al. 2012
Another concern for the concept representation of the 3Di-sv are the items in the current version that are not linked to the DSM-5 exemplars. While it may seem that these items can simply be removed, more consideration should go into this decision. The DSM-5 criteria explicitly state that the exemplars listed are not to be considered exhaustive. This means that items that cannot be matched to a specific exemplar in the DSM-5 might still be considered good examples of behavior indicative of a limitation as intended by the DSM-5 criteria. This particularly seems to be the case for the items concerning the sharing of food treats, and imitative play. While neither fits closely to any of the defined exemplars, both do seem indicative of an understanding of social relationships and social reciprocity that could clinically be considered part of the criteria for the reciprocal communication domain as the DSM-5 defines it. This is less obviously the case for the items concerning solo fantasy play, which do not have a direct relation to social behavior. However, solitary fantasy play is one of many ways in which children practice social scenarios and how to behave in social situations, and thus could still be considered an important part of the process through which children develop adequate social skills (Hobson et al. 2013
). Item selection approaches such as those mentioned above may well offer further guidance in deciding whether and, if so, where, these items should be retained.
Some methodological notes should be considered when interpreting the results of this study. For the validation of the 3Di-sv, we used ADOS-2-confirmed clinical diagnosis as a criterion. This standard combines clinical judgment, which is the gold standard for ASD diagnosis, with the clarity and replicability of standardized assessment. However, it does exclude children who were diagnosed with ASD but did not obtain an ASD classification on the ADOS-2 on the assumption that these are not true cases. The recommendation to combine child observation and parental interview for ASD diagnosis is made particularly because the results from the two sources complement each other, rather than overlapping perfectly. Thus, some of the false positives in our study may actually have been children with ASD who were missed by the ADOS-2.
For the CFA, we followed Mandy et al. (2014
) in rearranging 3Di scales, rather than items. While this mostly works out well, it does lead to some concerns about specific items in the subscales. In particular, while the movement of the repetitive and stereotyped language scale to the RSB domain is in line with the changes in the DSM-5, this scale also includes items about issues such as asking inappropriate questions and mixing up pronouns, which may better fit the social communication domain, and thus may be misspecified on an item level. However, since the results of the Cronbach’s alpha calculations, which were done at an item level, support the idea that the factor analysis yielded two coherent domains, this does not seem to have unduly influenced the results. Conversely, even without changing the content of the scales, analyzing the same data on a different level—that is, using items or subscales rather than the scales used in our study—may change the results (e.g. De la Marche et al. 2015
), so further analysis is needed to determine the best factor structure for the data on an item level. The current dataset lacks the power to perform the CFA with the larger number of variables this would involve. CFA should therefore be repeated within a larger dataset, so individual items can be entered into the analysis.
Finally, construct representation was based purely on a theory-driven, face validity perspective. In the future, it would be a valuable addition to determine which items would best add to the 3Di-sv in order to construct a route that fully represents the DSM-5 based on a data-driven procedure.
Conclusion and Clinical Implications
The 3Di-sv, in this study, appears to be somewhat over inclusive in its classifications. While the confidence intervals for the sensitivity and specificity of the 3Di-sv in specialized ASD centers were wide, resulting in the differences with general centers being non-significant, the values do seem to indicate that the 3Di-sv may perform better in general centers with a less complex or severe population. However, future research with larger samples will have to confirm whether or not this is indeed the case.
While the current results indicate that the 3Di-sv may be a solid basis upon which to build to create a similar route that is compatible with the DSM-5, creating a new DSM-5 version of the 3Di-sv will require some adjustments. First, items will need to be added to better cover newly introduced criteria, such as insistence on sameness and sensory abnormalities. The current full interview already contains some of the items needed to make this adjustment, but a data-driven approach to determine which items best represent the relevant criteria would be a valuable addition to the research on the 3Di. Secondly, decisions will need to be made on how to deal with those items that over represent the ASD construct under the DSM-5. Items concerning related symptoms that do not quite fit the exemplars may still add diagnostically important information. More research is needed to determine their usefulness in order to decide how best to deal with these items. Finally, the current scoring algorithm still leads to scores on DSM-IV-TR domains, with stereotypical language symptoms scored under communication rather than RSB. Thus, the scoring algorithm and format for final scores will need to be adjusted in order to represent the new ASD conceptualization.
Finally, while changes to the 3Di-sv may improve the specificity, any single instrument is likely to over or under identify ASD cases in some samples. Therefore, standardized parental interview information should always be complemented by alternative sources of information, such as direct observation and/or school report.
We gratefully acknowledge the contribution of all graduate students, PhD-students, and research assistants involved in the study, as well as the clinical professionals, management and administrative staff of the participating mental health care centers: Emergis, Erasmus MC-Sophia Children’s Hospital, GGZ WNB, Lucertis, Riagg Rijnmond, Yulius. We would particularly like to mention the work of Richard Warrington in developing and coding the Dutch version of the 3Di. We thank all children and parents who participated in the study.
GS participated in the coordination of the Social Spectrum Study, performed the measurements and statistical analyses and drafted the manuscript; WM developed the original analysis plan for the confirmatory factor analysis, provided necessary details on the 3Di routes and development and helped in the allocation of items to the DSM-5 scales; JvdE contributed to the statistical analyses plan and correct interpretation of the data; FV participated in the design of the Social Spectrum Study and helped draft the manuscript; AvdS conceived of the translation and validation of a Dutch 3Di; JD participated in the design and coordination of and measurements for the Social Spectrum Study and helped to draft the manuscript; DS designed the 3Di and provided necessary detail about its functioning and background, and helped in the allocation of items to the DSM-5 scales; KGL conceived of the Social Spectrum Study, participated in its design and coordination, and helped to draft the manuscript. All authors read and approved the final manuscript.
This research was supported by a Grant from the Sophia Foundation for Scientific Research (SSWO; project number 958).