Hostname: page-component-7c8c6479df-nwzlb Total loading time: 0 Render date: 2024-03-28T01:22:15.648Z Has data issue: false hasContentIssue false

Clinical utility of the parent-reported Strengths and Difficulties Questionnaire as a screen for emotional and behavioural difficulties in children and adolescents with intellectual disability

Published online by Cambridge University Press:  17 November 2020

Caitlin A. Murray*
Affiliation:
Centre for Educational Development, Appraisal and Research, University of Warwick, Coventry, UK
Richard P. Hastings
Affiliation:
Centre for Educational Development, Appraisal and Research, University of Warwick, Coventry, UK; and Centre for Developmental Psychiatry and Psychology, Department of Psychiatry, Monash University, Melbourne, Australia
Vasiliki Totsika
Affiliation:
Centre for Educational Development, Appraisal and Research, University of Warwick, Coventry, UK; and Centre for Developmental Psychiatry and Psychology, Department of Psychiatry, Monash University, Melbourne, Australia; and Division of Psychiatry, University College London, UK
*
Correspondence: Caitlin A. Murray. Email: C.Murray.7@warwick.ac.uk
Rights & Permissions [Opens in a new window]

Summary

We assessed the clinical utility of the parent-reported Strengths and Difficulties Questionnaire (SDQ) as a screen for emotional and behavioural difficulties in 626 children and young people with intellectual disability. Using the Developmental Behavior Checklist (DBC2-P) to determine clinical caseness, the area under the curve for the SDQ total difficulties score was 0.876 (95% CI 0.841–0.911), indicating that it is a good measure for identifying significant emotional and behavioural difficulties requiring further investigation. Analyses supported the use of the same SDQ cut-off for those with and without intellectual disability, which may assist with consistent and comparable assessment in clinical practice.

Type
Short report
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press on behalf of the Royal College of Psychiatrists

Children with intellectual disability have increased emotional and behavioural difficulties and mental health problems compared with typically developing children,Reference Kinnear, Rydzewska, Dunn, Hughes-McCormack, Melville and Henderson1 with a 36% prevalence for any psychiatric disorder (versus 8% in those without intellectual disability).Reference Emerson and Hatton2 Although prevalence rates are high, there can be challenges in screening and diagnosis, especially when considering whether mental health problems present differently in children with intellectual disability. The UK's National Institute for Health and Care Excellence (NICE) suggests that the gold standard for assessing emotional and behavioural difficulties in children and young people with intellectual disability3 is the Developmental Behavior Checklist – Parent ReportReference Gray, Tonge, Einfeld, Gruber and Klein4 (DBC2-P). However, clinical settings in the UK and across the world, especially those serving children with a range of intellectual ability, often rely on shorter screening measures developed for the general population before full assessment and/or acceptance to services.Reference Wright, Wellsted, Gratton, Besser and Midgley5 Often used is the Strengths and Difficulties QuestionnaireReference Goodman6 (SDQ), a brief measure with parent, teacher and child self-report versions freely available. Although there has been strong evidence for the capacity of the SDQ to identify those who require a full clinical assessment in the general population,Reference Goodman, Ford, Simmons, Gatward and Meltzer7 there is less evidence for children and young people with intellectual disability. A previous study involving 83 young people with intellectual disability found that SDQ and DBC-P total scores correlated well, and the SDQ ‘borderline’ cut-off identified 86% of those who met the DBC-P clinical cut-off.Reference Rice, Emerson, Gray, Howlin, Tonge and Warner8 The aim of the present study was to examine whether the SDQ can be used as a screen for emotional and behavioural problems in children and young people with intellectual disability, using data in comparison with the DBC2-P in a community research sample of children and young people with intellectual disability.

Method

Participants were from the first wave of the Cerebra 1000 Families study, a UK cohort of families of children with intellectual disability.Reference Hastings, Totsika, Hayden, Murray, Jess and Langley9 The National Health Service West Midlands–South Birmingham Research Ethics Committee granted ethical approval. Written informed consent was obtained from all participants. The present sample were primary carers from 626 families who completed the SDQ and the DBC2-P for their child with intellectual disability. Children's ages ranged from 4 years to 15 years 11 months (mean age 8.98 years, s.d. = 2.97). The majority of the children were male (67.9%; n = 425; female 31.8%, n=199; missing 0.3%, n=2), 51.1% had a diagnosis of autism, based on parent report; 87.1% of primary carers were White British. Intellectual disability was determined by parental report only. The Vineland Adaptive Behavior Scales (VABS-II) conducted alongside the DBC2-P indicated that the children's intellectual disability ranged from mild to severe (Adaptive Behavior Composite (ABC) scores: mean 58, range 25–94).

Measures

The SDQ parent reportReference Goodman6 is a 25-item measure for children 4–17 years old with five key scales: emotional problems, conduct problems, hyperactivity/inattention, peer relationship problems and prosocial behaviour. A total difficulties score is obtained by summing the four problem scales. For the current four-band scoring, total scores 0–13 indicate ‘close to average’, 14–16 indicate ‘slightly raised’, 17–19 indicate ‘high’ and 20–40 indicate ‘very high’ problem levels. Previous three-band scoring used scores 14–16 as ‘borderline’ and 17–40 as ‘abnormal’.10

The DBC2-PReference Gray, Tonge, Einfeld, Gruber and Klein4 is a 96-item (each item scored 0–2) measure of emotional and behavioural problems in 4- to 18-year-olds with intellectual or developmental disabilities. The subscales are: disruptive/antisocial behaviour, self-absorbed, communication disturbance, anxiety and social relating. The total behaviour problem score sums all items and a cut-off score of 46 can be used to identify clinically significant levels of emotional and behavioural disturbance.

Statistical analysis

The clinical cut-off for the DBC2-P was used as the gold standard criterion for emotional and behavioural problems. Receiver operator characteristic (ROC) analysis examined the association between SDQ and DBC caseness. Area under the curve (AUC) effect sizes indicated the magnitude of this association. We estimated sensitivity, specificity and optimal cut-off values for the SDQ total difficulties score.

Results

The AUC was 0.876 (P < 0.001, s.e. = 0.018, 95% CI 0.841–0.911), indicating that the SDQ total score is a good measure for indicating when children and young people with intellectual disability have clinically significant emotional and behavioural difficulties (Fig. 1). When considering both sensitivity and specificity, an SDQ cut-off score of 17 (0.819 sensitivity, 0.787 specificity) was optimal (matching ‘high’ problem levels). The SDQ cut-off of 14 (‘slightly raised’ problems) showed high sensitivity (0.931) but low specificity (0.481). The SDQ ‘very high’ cut-off score of 20 showed low sensitivity (0.670) but high specificity (0.898).

Fig. 1 Receiver operator characteristic (ROC) curve for the parent-reported Strengths and Difficulties Questionnaire total difficulties score. Diagonal segments are produced by ties.

Using the ‘high’ cut-off of 17, the SDQ identified 81.9% of children who met the clinical cut-off on the DBC2-P, with a false-positive rate of 21.3%. Given that the SDQ is for brief screening, and the children scoring above the SDQ cut-off are still likely to be demonstrating difficulties that would benefit from identification and support, this false-positive rate may be acceptable.

The majority (69.4%) of children with intellectual disability scored above 17 on the SDQ, indicating clinically significant emotional and behavioural difficulties. Correlations between the SDQ and DBC2-P were as expected, with a strong correlation between total scores (r = 0.72, P < 0.001) and the expected correlations between the respective subscales: SDQ emotional subscale and DBC2-P anxiety subscale (r = 0.60, P < 0.001), SDQ conduct problems and DBC2-P disruptive/antisocial subscale (r = 0.66, P < 0.001) and SDQ peer problems and DBC2-P social relating subscales (r = 0.47, P < 0.001).

Some clinicians have queried whether the SDQ is suitable for use across the range of intellectual disability. Using VABS-IIReference Sparrow, Cicchetti and Balla11 ABC scores to define groups, children with milder intellectual disability (ABC score ≥55; n = 414) had an AUC for SDQ total scores of 0.878 (P < 0.001, s.e. = 0.022, 95% CI 0.834–0.921) and those with more severe (ABC score <55; n = 204) an AUC of 0.876 (P < 0.001, s.e. = 0.031, 95% CI 0.814–0.937). The ‘high’ SDQ cut-off identified 83.7% of those with milder intellectual disability and 78.3% of those with more severe intellectual disability, with false-positive rates of 21.4 and 21.1% respectively.

Boys (n = 425) and girls (n = 199) showed similar results, with an AUC for SDQ total scores of 0.849 (P < 0.001, s.e. = 0.027, 95% CI 0.795–0.903) and 0.904 (P < 0.001, s.e. = 0.024, 95% CI 0.857–0.950) respectively. The ‘high’ cut-off identified 82.0% of boys and 81.9% of girls, with false-positive rates of 26.4 and 16.4% respectively.

In relation to an additional autism diagnosis, AUC scores indicated that the SDQ is a fair to good measure. For the autism group (n = 320), the AUC was 0.755 (P = 0.03, s.e. = 0.081, 95% CI 0.596–0.913); and for the non-autism group (n = 306), the AUC was 0.831 (P < 0.001, s.e. = 0.024, 95% CI 0.784–0.878). The ‘high’ cut-off identified 90.4% of the autism group and 68.6% of the non-autism group, with false-positive rates of 66.7 and 18.6% respectively. This high false-positive rate in the autism group may be acceptable as it is based on very low numbers of negative cases (n = 6).

Discussion

Findings indicated that the SDQ is a clinically useful screen for emotional and behavioural difficulties for children and young people with intellectual disability prior to formal assessment, given the strong association between SDQ total scores and the DBC2-P clinical cut-off. Unlike previous research that supported the older ‘borderline’ cut-off for children with intellectual disability,Reference Rice, Emerson, Gray, Howlin, Tonge and Warner8 the best balance of specificity and sensitivity was found using the existing SDQ ‘high’ cut-off.10 Using the same cut-off enables children with intellectual disability to be compared with other children and consistent clinical use in services for children across the range of intellectual ability. The SDQ total score also appeared to be suitable for use in both milder and more severe intellectual disability groups, although additional testing of subgroups is needed based on more robust classification than VABS-II ABC scores. The SDQ appears suitable for both boys and girls. The SDQ is a fair measure for children with intellectual disability and autism, with excellent sensitivity, although further investigation may be useful.

This study is based on cross-sectional data. Longitudinal data would allow researchers to examine the predictive validity of the SDQ in the identification of emotional and behavioural problems in children with intellectual disability. Multiple informants, particularly child self-report, would be useful to consider in the future, as multi-informant report increased the sensitivity of the SDQ in a child community sample.Reference Goodman, Ford, Simmons, Gatward and Meltzer7 Previous research in small groups of children with intellectual disability indicate that the self-report SDQ may be an appropriate measure in this populationReference Emerson12,Reference Haynes, Gilmore, Shochet, Campbell and Roberts13 given modified wording.Reference Haynes, Gilmore, Shochet, Campbell and Roberts13 The 1000 Families study relied on caregiver report only, and so clinical interviews and diagnoses were not available for this study. Further research using diagnostic interviews would allow for further examination of the clinical utility of the SDQ for children with intellectual disability. In addition, attention to other factors that may be associated with the utility of the SDQ (e.g. ethnicity) is needed.

Data availability

Data from this study are not available for sharing owing to ethical approval requirements. Researchers interested in collaboration should contact the corresponding author with their expression of interest.

Acknowledgements

We thank all the families who participated in this study. We acknowledge the support of Cerebra and their collaboration on the overall 1000 Families study, including study design and support with data recruitment. We also acknowledge the support of researchers involved with the 1000 Families study.

Author contributions

C.A.M. contributed to the design of the study, analysed the data, drafted the manuscript for publication, reviewed and revised the manuscript, and read and approved the final manuscript. R.P.H. and V.T. contributed to the design of the study, reviewed and revised the manuscript, and read and approved the final manuscript.

Funding

The authors acknowledge the funding support of Cerebra, UK, and the University of Warwick through the PhD studentship of C.A.M.. Cerebra had no involvement or restrictions regarding publication.

Declaration of interest

None.

References

Kinnear, D, Rydzewska, E, Dunn, K, Hughes-McCormack, LA, Melville, C, Henderson, A, et al. Relative influence of intellectual disabilities and autism on mental and general health in Scotland: a cross-sectional study of a whole country of 5.3 million children and adults. BMJ Open 2019; 9(8): e029040.CrossRefGoogle ScholarPubMed
Emerson, E, Hatton, C. Mental health of children and adolescents with intellectual disabilities in Britain. Br J Psychiatry 2007; 191: 493–9.CrossRefGoogle ScholarPubMed
National Institute for Health and Care Excellence. Mental Health Problems in People with Learning Disabilities: Prevention, Assessment and Management (NICE Guideline NG54). NICE, 2016.Google Scholar
Gray, K, Tonge, B, Einfeld, S, Gruber, C, Klein, A. DBC2: Developmental Behavior Checklist 2. Manual. Western Psychological Services, 2018.Google Scholar
Wright, H, Wellsted, D, Gratton, J, Besser, SJ, Midgley, N. Use of the Strengths and Difficulties Questionnaire to identify treatment needs in looked-after children referred to CAMHS. Dev Child Welf 2019; 1: 159–76.CrossRefGoogle Scholar
Goodman, R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry 1997; 38: 581–6.CrossRefGoogle ScholarPubMed
Goodman, R, Ford, T, Simmons, H, Gatward, R, Meltzer, H. Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Br J Psychiatry 2000; 177: 534–9.CrossRefGoogle Scholar
Rice, LJ, Emerson, E, Gray, KM, Howlin, P, Tonge, BJ, Warner, GL, et al. Concurrence of the Strengths and Difficulties Questionnaire and Developmental Behaviour Checklist among children with an intellectual disability. J Intellect Disabil Res 2018; 62: 150–5.CrossRefGoogle ScholarPubMed
Hastings, RP, Totsika, V, Hayden, NK, Murray, CA, Jess, M, Langley, E, et al. 1000 Families Study, a UK multi-wave cohort investigating the wellbeing of families of children with intellectual disabilities. BMJ Open 2020; 10(2): e032919.CrossRefGoogle Scholar
Youth in Mind. Scoring the Strengths and Difficulties Questionnaire for ages 4–17. In Scoring the SDQ. Youth in Mind, 2016 (https://www.sdqinfo.org/py/sdqinfo/c0.py).Google Scholar
Sparrow, SS, Cicchetti, VD, Balla, AD. Vineland Adaptive Behavior Scales (2nd edn). American Guidance Service, 2005.Google Scholar
Emerson, E. Use of the Strengths and Difficulties Questionnaire to assess the mental health needs of children and adolescents with intellectual disabilities. J Intellect Dev Disabil 2005; 30: 1423.CrossRefGoogle Scholar
Haynes, A, Gilmore, L, Shochet, I, Campbell, M, Roberts, C. Factor analysis of the self-report version of the strengths and difficulties questionnaire in a sample of children with intellectual disability. Res Dev Disabil 2013; 34: 847–54.CrossRefGoogle Scholar
Figure 0

Fig. 1 Receiver operator characteristic (ROC) curve for the parent-reported Strengths and Difficulties Questionnaire total difficulties score. Diagonal segments are produced by ties.

Submit a response

eLetters

No eLetters have been published for this article.