Elsevier

Psychiatry Research

Volume 246, 30 December 2016, Pages 520-526
Psychiatry Research

The use of the SDQ with Chinese adolescents in the clinical context

https://doi.org/10.1016/j.psychres.2016.10.034Get rights and content

Highlights

  • The SDQ is widely used screening instrument for child and adolescent behavioural and psychological problems.

  • Its use in clinical contexts remains relatively unexamined.

  • We found that its psychometric properties are weak in the Chinese clinical context.

Abstract

This study investigated the psychometric properties of the Strengths and Difficulties Questionnaire when it was applied to a sample of 443 outpatients in China. Included in the sample were 88 adolescents diagnosed with depression, 96 with schizophrenia, 98 with generalized anxiety, 70 with OCD, and 91 with ADHD. Each patient and one of their parents completed the SDQ at intake. Confirmatory Factor Analyses provided limited support for the proposed five factor structure of the scale over other models. Internal reliabilities of the subscales for both self and parent report were weak, and inter-rater agreement between self- and parent-report was moderate. The specificity and sensitivity for the Total Difficulties scores were weak. Overall, these findings suggest that the use of the SDQ in clinical samples in China may be limited.

Introduction

Recent research (e.g., Lawrence et al., 2015) and meta-analyses indicate that there is a high prevalence rate of mental health problems among children and adolescents around the world. For example, Polanczyk et al.'s (2015) meta-analysis of 41 studies conducted in 27 countries found that the worldwide prevalence of child and adolescent mental disorders was 13.4% (CI 95% 11.3–15.9), with anxiety disorders (6.5%), depressive disorders (2.6%), attention-deficit hyperactivity disorder (3.4%), and disruptive disorders (5.7%) being the most common. Although various methodological factors impacted on the prevalence figures reported in any study, estimates did not vary by geographic region. These findings, and those of studies that show that many children and adolescents with mental health problems do not access/receive treatment (e.g., Sawyer et al., 2001) suggest the need for early detection and referral.

The Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997) is a brief screening questionnaire designed to assess child and adolescent mental health and behavioral difficulties. It consists of 25 items that make up five five-item subscales assessing conduct problems, hyperactivity-inattention, emotional symptoms, peer problems and prosocial behavior. An extended version includes an impact supplement that asks if the respondent thinks that the target child has a problem, and explores its chronicity, distress, social impairment and burden for others. The instrument has been widely adopted and researched, and there are now 75 language variations of the three informant versions (self, parent, and teacher) available at the SDQ website (sdqinfo.org).

Considerable research has been conducted in numerous community contexts to establish the psychometric properties of the SDQ. Early studies demonstrated good internal reliability for the subscales and total difficulties scale across the three informant versions, as well as sound inter-informant reliability and good test-retest reliability across the parent, teacher and child versions in the UK (e.g., Goodman et al., 2000, 2003), the Netherlands (Muris et al., 2003), Germany (Klasen et al., 2000), Finland (Koskelainen at al., 2000), and Sweden (Smedje et al., 1999). However, other studies have reported reliability coefficients for informants that are lower than what is usually considered to be adequate (e.g., 0.70, Cicchetti, 1994), particularly for the conduct and peer problems subscales (Australia, Mellor, 2004; Italy, Di Riso et al., 2010; China, Du et al., 2008; Mellor et al., 2011; Japan, Matsuishi et al., 2008; and the Netherlands, Muris et al., 2003).

Studies that have focused on the structural validity of the SDQ across various countries have also been completed with community samples, and they too have reported mixed findings. Most of these studies (e.g., Koskelainen et al., 2001; Smedje et al., 1999) have factor-analyzed data obtained from just one of the three informant versions (i.e., parent, teacher or self-report) and have failed to reproduce the expected simple five factor solution. Rather, while the factor structure is typically partially replicated, there are several items that seem to be problematic. These problematic items cross-load on more than one factor, or load more strongly on a factor other than that onto which they are expected to load. Dickey and Blumberg’s (2004) combination of exploratory and confirmatory factor analysis on split-halves of their national representative sample of 9574 children and adolescents in the United States supported a three-factor solution made up of internalizing (emotional symptoms and peer problems), externalizing (conduct problems and hyperactivity/inattention), and prosocial behavior factors. Di Riso et al.'s (2010) confirmatory factor analysis of self-report data also identified three similar interpretable factors, while Palmieri and Smith (2007) found support for a different three-factor model when the parent version of the SDQ was administered over the telephone to 773 custodial grandparents. Based on parent-, teacher- and youth-report data from a large representative sample in Britain, Goodman et al. (2010) proposed a more complex five factor model that included Dickey and Blumberg’s (2004) internalizing and externalizing factors as second order factors. However, they suggested that the broader internalizing and externalizing subscales should only be used with low risk samples.

Other studies in Spain by Ortuño-Sierra et al. (2015a) and Ortuño-Sierra et al. (2015b) using self-report by large samples of adolescents have supported the five factor model and demonstrated general measurement invariance across age and gender. In the second of these studies a five point response format was used rather than the standard three point format. Finally, Ortuño-Sierra et al. (2015c) examined the structural and measurement invariance of the self-report version of the SDQ across five European contexts. In this study they found that five-factor model had partial measurement invariance across countries, and that 11 of the 25 items were non-invariant across samples, suggesting that the subscales may need to be modified.

Despite the above issues, the SDQ is widely used in many contexts as a screening tool and outcome measure. In Australia for example, has been adopted as one of a suite of outcome measures in child and adolescent mental health services. However, one aspect of the SDQ that has been the subject of less research has been its ability to discriminate between various clinical groups of children and adolescents. Early studies (Goodman et al., 1999; Klasen et.al., 2000) reported that it is comparable with, or better than, the Child Behavior Checklist (CBCL) in detecting disorders diagnosed through standardised semi-structured interview, with Klasen et al. concluding that the SDQ and the CBCL are equally valid for most clinical and research purposes. Goodman et al. (2000) subsequently developed a computerized algorithm based on multi-informant SDQ symptom and impact scores to identify four broad categories of problems: conduct disorder, emotional disorder, hyperactivity disorder, and ‘any psychiatric disorder’. They applied this algorithm to data derived from clinical samples in the UK (London) and Bangladesh (Dhaka) and compared the results with diagnoses established by clinicians according to ICD-10 criteria. The sensitivity for the different categories ranged from 81% to 90%, and apart from the conduct disorder category in the UK (47%), the specificity ranged from 78% to 84%. On the basis of these findings Goodman et al. argued that the SDQ algorithm is good at detecting psychiatric disorders, although other analyses suggested that it might over-diagnose slightly. Using the multi-informant algorithm in a large sample of children in care in the UK, Goodman et al. (2004) later found that against an independent psychiatric diagnoses, the SDQ identified children and adolescents with a diagnosis with a specificity of 80% and a sensitivity of 85%.

Consistent with the above, are the findings of a study in Germany, in which Becker et al. (2004a) applied the SDQ with 214 adolescent in- and out-patients attending a child and adolescent psychiatry clinic. They found that adolescent-reported SDQ scores were sensitive to caseness and demonstrated good specificity in that they differentiated between various psychiatric disorders. In a further study, Becker et al. (2004b) reported on the application of the SDQ with 543 inpatients and outpatients at the same clinic, 380 of whom were considered to exhibit a DSM Axis 1 disorder after psychiatric examination. They found that the SDQ discriminated well between clinical and non-clinical subgroups, and between clinically defined subgroups (emotional disorders, conduct disorders, hyperactivity/attention deficit disorders). Becker et al. (2004b) also reproduced the factor structure in ¨exactly the proposed pattern¨ (p. II/5) using CFA on the parent-reported data, concluding that the German SDQ maintains the purported structure when used in a clinical as well as community samples.

In contrast to the above studies, various others in different parts of the world have questioned the use of the SDQ in clinical contexts. In Sydney, Australia Mathai et al. (2004) found that the level of agreement between the SDQ-generated diagnoses and clinician diagnoses was moderate to high, ranging from 0.39 to 0.56. Sensitivities of 44% for hyperactivity disorder, 93% for conduct disorder, and 36% for emotional disorder were reported. Mathai et al. concluded that the ´SDQ is a useful instrument to aid clinicians in diagnosis and could be used as part of the initial assessment process´(p. 639). Similarly, Brøndbo et al. (2011) also investigated the SDQ in a clinical sample in Norway, with a focus on its sensitivity and specificity. For the diagnostic categories included sensitivity was 0.47–0.85 (‘probable’ dichotomisation level) and 0.81–1.00 (‘possible’ dichotomisation level). Specificity was 0.52–0.87 (‘probable’ level) and 0.24–0.58 (‘possible’ level). The discriminative ability, as measured by ORD, was in the interval for potentially useful tests for hyperactivity disorders and conduct disorders when dichotomised on the ‘possible’ level. Brøndbo et al. concluded that the SDQ is better suited for use as a screening instrument rather than a tool to diagnose clinical problems.

With few studies of the clinical utility of the SDQ, further research is required. In addition, while the SDQ has been used in a number of countries, the majority of published studies report on Western samples. The current study aims to determine the psychometric properties of the SDQ in a clinical context in China. Previous work with the SDQ in China has been reported by Du et al. (2008) who collected parent and teacher reports on 1965 children from the general community between the ages of 3 and 17 years in Shanghai. In addition, 690 of the target children completed the self-report version of the SDQ. Using principle components factor analysis, Du et al. only partially replicated the original five subscales, with the Conduct Problems and Peer Problems having the least support. Internal consistencies ranged between 0.30 and 0.83, and only two subscales had alpha co-efficients greater than 0.7. These were Hyperactivity–Inattention (parent and teacher versions) and Prosocial Behavior (teacher version). Inter-rater correlations were between 0.23–0.49, while test-retest reliability was variable (range 0.40–0.79). Du et al. investigated convergent and discriminant validity in relation to other scales and a small sample of children with a clinical diagnosis of ADHD, and reported that it was adequate. However, no major study in a Chinese clinical setting has been reported.

Our study was also conducted in Shanghai at a Child and Adolescent clinic at a major mental health facility. We investigated the factor structure, the internal consistency, the inter-rater reliability, as well as the sensitivity and specificity of the SDQ when self and parent report versions were applied among adolescents with a diagnosed psychiatric disorder in this tertiary clinical setting.

Section snippets

Participants

A total of 443 adolescent and their parents participated in the study. There were 249 males and 194 females in the adolescent sample, and all were outpatients attending a major Child and Adolescent clinic in Shanghai. The inclusion criteria were: 1) being between 12 and 18 years of age; 2) meeting one the five most commonly diagnosed DSM-IV diagnoses at the Center as determined by the treating psychiatrist and confirmed by another psychiatrist. These diagnoses were generalized anxiety disorder,

Analytic strategy

Before data were analyzed missing values (<1%) were replaced with the mean value for the variable. To evaluate the fit of factorial structures in use for the SDQ, Confirmatory Factor Analyses (CFA) were conducted separately for parent and child report, using in Mplus version 7.3 (Muthén and Muthén, 2012). Given the ordinal response format, a robust weighted least squares estimator (Muthén, 1984) was used. Four previously described models were evaluated: the conventional five-factor structure;

Factor structure

CFA was used to evaluate the fit of the previously described measurement models. Item 23 (“Gets on better with adults than with other children”) was found to not associate with its designated factor in any of the solutions for both parent and child report, duly models were evaluated both with and without this item; fit statistics for these models are presented in Table 1.

Overall, fit was below conventional criteria (Hu and Bentler, 1999) for all solutions, with fit being substantially worse for

Discussion

There have been limited studies of the SDQ in clinical samples, and relatively few studies have been reported on the use of the SDQ in China. In this study we addressed both of these issues, and examined a variety of psychometric properties of the scale. Our sample of more than 400 was drawn from a large outpatient child and adolescent clinic, and included adolescents diagnosed with five disorders: depression, schizophrenia, AHDH, anxiety, and OCD. We were able to access both self-report and

References (35)

  • Cossey, K., 2008. Hypothesis Driven Assessment of an NMR Curriculum....
  • D. Di Riso et al.

    The Strengths and Difficulties Questionnaire (SDQ). Early evidence of its reliability and validity in a community sample of Italian children

    Pers. Individ. Differ.

    (2010)
  • Y. Du et al.

    The validity, reliability and normative scores of the parent, teacher and self report versions of the Strengths and Difficulties Questionnaire in China

    Child. Adolesc. Psychiatry Ment. Health

    (2008)
  • T.J. Dunn et al.

    From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation

    Br. J. Psychol.

    (2014)
  • G.E. Gignac

    On the inappropriateness of using items to calculate total scale score reliability via coefficient alpha for multidimensional scales

    Eur. J. Psychol. Assess.

    (2014)
  • A. Goodman et al.

    When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): data from British parents, teachers and children

    J. Abnorm. Child Psychol.

    (2010)
  • R. Goodman

    The Strengths and Difficulties questionnaire: a research note

    J. Child Psychol. Psychiatry

    (1997)
  • Cited by (0)

    1

    Now at Australian Catholic University, Melbourne, Australia.

    View full text