Testing gender differential item functioning for ordinal and binary scored parent rated ADHD symptoms

Abstract

The study used the ordinal logistic regression method to examine gender differential item functioning (DIF) for ratings of the ADHD symptoms. This was examined for two different response scoring methods: ordinal and binary. Parents from the general community completed ratings for 742 and 733 primary school-aged girls and boys, respectively. Results indicate no gender DIF for all the ordinal scored inattention (IA) and hyperactivity/impulsivity (HI) symptoms, and all the binary scored IA symptoms. For the binary scored HI symptoms, there was DIF for one symptom, while DIF could not be computed for three symptoms. Overall, these findings suggest minimal gender DIF for the ADHD symptoms. The implications of the findings for the use of ADHD rating scales in clinical practice are discussed.

Introduction

According to the latest Diagnostic and Statistical Manual (DSM-IV TR, American Psychiatric Association, APA, 2000), attention deficit-hyperactivity disorder (ADHD) is one of the most common childhood psychological disorders. The diagnostic symptoms in this manual are identical to those in the original edition, DSM-IV (APA, 1994). These manuals have proposed the same diagnostic criteria for diagnosing ADHD for boys and girls. This implies that there is gender invariance for the ADHD symptoms. At present, rating scales of the ADHD symptoms have an important role in the assessment and diagnosis of ADHD (e.g., Anastopoulos and Shelton, 2001, Power et al., 1998). Given the assumption of gender invariance in the ADHD symptoms, it is important to evaluate if such scales demonstrate gender invariance. The aim of the current study was to examine gender invariance for one such scale in terms of two scoring procedures (ordinal and binary) that are currently used by researchers and clinicians. Gender invariance addresses the extent to which there is equivalency across rating scores provided by parents of boys and parents of girls when the individuals rated on have the same level of the underlying latent trait scores (Reise, Widaman, & Paugh, 1993). Invariance is inferred when the raw scores are the same, while non-invariance is inferred if this is not the case.

At present there are a number of DSM-IV based ADHD rating scales. These include the ADHD Rating Scale-IV (DuPaul et al., 1998), DSM-IV ADHD Rating Scale (Gomez, Harvey, Quick, Scharer, & Harris, 1999), and the Disruptive Behavior Rating Scale (Barkley & Murphy, 1998). In general, most of the DSM-IV based ADHD rating scales have the 18 DSM-IV ADHD symptoms, with almost the exact wordings from the DSM-IV. Thus there is an inattention (IA) scale with the nine IA symptoms, and a hyperactivity/impulsivity (HI) scale with the nine HI symptoms. For most of the ADHD rating scales, the symptoms are rated on a 4-point interval, ranging from 0 (indicating little or no sign of the symptoms) to 3 (indicating a very high level of the symptoms).

In terms of the construct validity of ADHD rating scales, a number of studies have examined their factor structure using confirmatory factor analysis (CFA; Burns et al., 2003, DuPaul et al., 1998, DuPaul et al., 1997, Gomez et al., 2005, Gomez et al., 2003, Gomez et al., 1999). Consistent with the way the ADHD symptoms are grouped in DSM-IV, these studies have found that a 2-factor model consisting of the IA and HI symptoms in separate factors provides good fit for the organization of parent and teacher ratings.

For both clinical and research purposes, the responses provided for ADHD rating scales are scored in one of two different ways: the ordinal scoring method, and the binary scoring method (Barkley & Murphy, 1998). In the ordinal scoring method, the original 4 ordinal levels for the 9 IA and 9 HI symptoms are added to give total scores for the IA and HI scales. Thus the intervals between 0 and 1, 1 and 2, and 2 and 3 are treated for this purpose as equal-interval magnitudes. Total scores of at least 1.5 SD above the normative mean score have been used to infer the possibility of the presence of the relevant symptom group. In the binary scoring method, the first two response options of the original responses (i.e., options 0 and 1) are recoded as the symptom being absent (recoded as 0), while the next two response options (i.e., options 2 and 3) are recoded as the symptom being present (recoded as 1). Since the diagnoses for the IA and HI groups of symptoms require at least 6 symptoms to be present in each case, a score of at least 6 is used to infer the presence of the symptom groups. Given the use of the ordinal and the binary scoring methods, it will be desirable to examine gender invariance for ADHD symptom ratings for both the scoring methods. So far this has not been done for either teacher or parent ratings.

For diagnosis, DSM-IV has proposed the same diagnostic criteria for boys and girls. This implies that there needs to be gender invariance for the symptoms presented in ADHD rating scales. To test this, Reid et al. (2000) used a multiple group CFA approach to evaluate gender invariance for teacher ratings of an ADHD rating scale that was scored using the ordinal method. The results supported gender invariance for factor numbers, factor loadings, and the correlation between the IA and HI latent factors. More recently, Burns, Walsh, and Gomez (in press) reported similar findings for parent ratings of the ADHD symptoms, also scored using the ordinal method. The study also found gender invariance for item intercepts, the variances of the IA and HI factors, and differences for the IA and HI latent scores. Thus existing data for both parent and teacher ratings, scored using the ordinal method, do provide good support for gender invariance for the ADHD symptoms presented in the ADHD rating scales. Despite this, it is argued here that the issue of gender invariance for such scales needs to be evaluated further. This is because DSM-IV based ADHD diagnosis is based on a binary scoring method of the symptoms that requires simply establishing whether they are present or absent. Gender invariance for the ADHD symptoms involving binary scores has not been established so far.

One method that can be used for testing group invariance across both ordinal scores and binary scores is Zumbo’s ordinal logistic regression (OLR) method. This method can test if members belonging to different groups (such as parents of boys and parents of girls) differ in the probabilities of endorsing an item after being equated on the underlying trait that the item is intended to measure. This is referred to as differential item functioning (DIF). When items show DIF it means that they are not invariant across the groups. When items do not show DIF it means that they are invariant across the groups. Recently Gelin and Zumbo (2003) used the OLR method to examine gender DIF for the 20 items in the Center for Epidemiologic Studies Depression scale (CES-D; Radloff, 1977), scored using two binary methods (presence and persistence) and one ordinal method. Their results showed DIF for one same item for the ordinal and presence scoring methods, and two other items for the persistence scoring method. Thus, the scoring method had an effect on DIF, thereby indicating that DIF of an item can vary by scoring methods. Such findings highlight further the importance of examining the DIF in a measure in terms of all its scoring methods.

The aim of this study was to use the OLR method to determine if there was any gender DIF for parent ratings of the DSM-IV ADHD IA and HI symptoms. DIF was examined for both the ordinal and binary scoring methods. Thus the study also examined if DIF for the ADHD symptoms varies by scoring methods.