Content overlap between youth OCD scales: Heterogeneity among symptoms probed and implications

Highlights

• Substantial heterogeneity exists among scales purportedly assessing the same OCD construct in youth.

• There is greater content overlap between compulsion symptoms probed by youth OCD scales than between obsession symptoms.

• Incorporating both convergent validity and content overlap data gives the best picture of the similarity of two given instruments.

Abstract

Assessing obsessive compulsive disorder (OCD) in paediatric populations is complex, and a number of scales have been developed or validated for this purpose. As is the case with instruments measuring other mental health disorders, it is often assumed that youth OCD scales assess the same construct and can be used interchangeably. However, the extent to which this is true is unknown. One way to assess this – beyond calculating convergent validity – is to measure the extent of content overlap between scales. In the current paper, we apply a recently-developed method of content overlap analysis to a set of freely-available, self-report instruments for OCD measurement that were developed for or validated in paediatric populations. We found a high level of heterogeneity among scales purportedly assessing the same construct. Implications for clinical practice and research are discussed, as well as what further research is required.

Introduction

Obsessive-compulsive disorder (OCD) is characterised by the presence of obsessions and/or compulsions, which impact on daily functioning or cause psychological distress (American Psychiatric Association, 2013). However, there are complications in the disorder's diagnosis and measurement given the heterogeneous nature of OCD and the various combinations of potential symptoms present (Lewin, Park, & Storch, 2013). Additionally, psychopathologies such as anxiety and autism spectrum disorders may present similarly to OCD (Berman and Abramowitz, 2010, Lewin and Piacentini, 2010), and OCD is highly comorbid with these and other disorders, including tic, depressive, attention-deficit/hyperactivity, and behavioural disorders (Lewin et al., 2013, Storch et al., 2008).

The measurement of the disorder in youth populations has its own complexities, especially as children often lack insight into their OCD (Lewin et al., 2013). In childhood, prevalence estimates of OCD range from 0.25% (5–15 years; Heyman et al., 2003) to 2.7% (9–17 years; Rapoport et al., 2000), and up to 4% in adolescents specifically (18 years; Douglass, Moffitt, Dar, McGee, & Silva, 1995). The disorder is associated with significant functional impairment in those aged 5–17 (Piacentini, Bergman, Keller, & McCracken, 2003). Importantly, the DSM-5 (American Psychiatric Association, 2013) notes that presentation of OCD in youth may differ to that in adults. For example, the content of obsessions and compulsions may vary, children may struggle to articulate obsessions and so they may be less obvious than observable compulsions, and younger children may find it difficult to describe the aims of their behaviours or thoughts (American Psychiatric Association, 2013). There are also disorder characteristics specific to children, e.g., an especially high prevalence of family accommodation, i.e., involving relatives in rituals (Lewin et al., 2013, Peris et al., 2008). As such, in addition to having strong psychometric properties, it is important that OCD scales administered to paediatric populations have been specifically created for them and are developmentally appropriate or have at least been validated in youth samples.

There are several scales available for use with paediatric populations, including both those specifically assessing OCD, and those that examine OCD among other psychopathology. A recent review identified 14 OCD-specific measures whose psychometric properties had been assessed in youth samples (Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar, & Storch, 2014). As is often the case with instruments used to assess specific mental health disorders, it is generally assumed that these OCD scales measure the same construct, and as such, scales are used interchangeably and results from one compared against another. Whether the former is true, and the latter a sound practice, is important to verify. If scales do not assess the same construct, this hinders our ability to draw conclusions about epidemiological data and intervention results as they relate to a given disorder. Rather, we may only be able to speak in terms of scores on a given instrument (Fried, 2017). A scale's symptom coverage also has implications for the ability of that instrument to discern disorder subtypes, which is of particular concern for OCD given a presentation heterogeneity that is more marked than in other disorders (Storch et al., 2010). Considering evidence in adult populations that certain symptom profiles may respond differently to therapy (Sookman et al., 2005, Williams et al., 2013), and similar findings based on limited research in youth populations as well (Storch et al., 2008), the ability of an instrument to identify symptom profiles/subtypes is key to selecting appropriate treatments.

There are several reasons to suspect that youth OCD scales may not measure the same construct – scales have been created by different research groups, for different research or clinical purposes, and with different populations in mind. Additionally, some assessments are biased towards obsessions over compulsions (or vice versa), and existing scales vary markedly in length. One way to assess whether scales are measuring the same construct is to calculate convergent validity, i.e., how actual scores on different tests correlate. There is mixed evidence for the convergent validity of youth OCD assessments, with some scales demonstrating strong convergent validity, some weak, and others lacking in research (Iniesta-Sepúlveda et al., 2014).

However, it is possible that despite two scales displaying high convergent validity, they may not actually be assessing the same construct. This could be due to high rates of comorbidity (and so people that score highly on a scale measuring construct A may also score highly on a scale measuring construct B). More probable in the case of youth OCD, scales may be assessing different aspects of the same broader construct, thus not overlapping well in terms of item content but displaying high convergent validity due to inter-relatedness of constructs or some common underlying construct. To explore this possibility, a complementary method of comparing instruments is to assess the overlap of item content between the scales themselves, irrespective of actual human performance on those scales. Developing a method of content overlap analysis that utilises the Jaccard similarity coefficient, Fried (2017) assessed the overlap between seven widely-used depression scales and found considerable heterogeneity among the instruments, concluding that results from depression research may only be interpretable in terms of the specific scale used. He found a moderate correlation between number of symptoms captured by a scale and how well it overlapped with other scales, suggesting longer scales were more representative. To the best of the authors’ knowledge, his is the only study to implement such analysis.

This paper aims to apply the method outlined by Fried (2017) to a set of freely-available, self-report instruments that exclusively measure OCD and were developed for or validated in paediatric populations. Our focus here is on content overlap of items that probe the presence of specific symptoms, as opposed to items assessing symptom severity or functional impairment. For the reasons outlined earlier, we expect to find some level of heterogeneity among youth OCD scales.