Validity, responsiveness, minimal detectable change, and minimal clinically important change of the Pediatric Motor Activity Log in children with cerebral palsy

Abstract

This study examined criterion-related validity and clinimetric properties of the Pediatric Motor Activity Log (PMAL) in children with cerebral palsy. Study participants were 41 children (age range: 28–113 months) and their parents. Criterion-related validity was evaluated by the associations between the PMAL and criterion measures at baseline and posttreatment, including the self-care, mobility, and cognition subscale, the total performance of the Functional Independence Measure in children (WeeFIM), and the grasping and visual-motor integration of the Peabody Developmental Motor Scales. Pearson correlation coefficients were calculated. Responsiveness was examined using the paired t test and the standardized response mean, the minimal detectable change was captured at the 90% confidence level, and the minimal clinically important change was estimated using anchor-based and distribution-based approaches. The PMAL-QOM showed fair concurrent validity at pretreatment and posttreatment and predictive validity, whereas the PMAL-AOU had fair concurrent validity at posttreatment only. The PMAL-AOU and PMAL-QOM were both markedly responsive to change after treatment. Improvement of at least 0.67 points on the PMAL-AOU and 0.66 points on the PMAL-QOM can be considered as a true change, not measurement error. A mean change has to exceed the range of 0.39–0.94 on the PMAL-AOU and the range of 0.38–0.74 on the PMAL-QOM to be regarded as clinically important change.

Introduction

Cerebral palsy (CP) is defined as nonprogressive motor impairment caused by brain injury at very early human development and is often accompanied with psychiatric or behavioral problems and restrictions in learning and perceptual development (Rosenbaum, Paneth, Leviton, Goldstein, & Bax, 2007). The prevalence of CP is approximately 2–3 per 1000 births (Winter, Autry, Boyle, & Yeargin-Allsopp, 2002) and affects 3.6 per 1000 school-aged children (Yeargin-Allsopp et al., 2008). Many individuals with CP have hemiplegia or deficit in upper extremity (UE) (Himmelmann, Hagberg, Beckung, Hagberg, & Uvebrant, 2005) that is more involved than in the lower limb, with reduced strength and dysfunction in spasticity and in sensation (Sakzewski, Ziviani, & Boyd, 2009). Impaired arm function often limits children's participation in education and leisure activities as well as in vocational roles. The treatment and the health care over their lifespan are costly (Centers for Disease Control and Prevention [CDC], 2004). It is therefore necessary to validate existing measures of arm and hand function to determine which therapy method is effective for rehabilitation in children with CP (DeMatteo et al., 1993).

The Pediatric Motor Activity Log (PMAL) is a widely used outcome measure in research investigating the efficacy of constraint-induced movement therapy (CIT) for children with CP (Deluca et al., 2006, Lin et al., 2011, Sutcliffe et al., 2007). The PMAL was derived from the Motor Activity Log (MAL) in adult patients with stroke (Taub, Ramey, DeLuca, & Echols, 2004). The adult MAL is a patient-reported evaluative tool for assessing the use of affected UE in 30 tasks of daily living (Taub et al., 1993). The adult MAL-QOM has acceptable to excellent psychometric and clinimetric properties: internal consistency (Cronbach's alpha > 0.81), test–retest reliability (r > 0.91), intraclass correlations (ICC) between patients and caregivers (0.52–0.70), and responsiveness (ratio > 1.9) (Hammer and Lindmark, 2010, Uswatte et al., 2006). The MAL-AOU was found to be not as reliable as the QOM scale, yielding inconsistent findings in the rehabilitation research (Uswatte et al., 2005, van der Lee et al., 2004). Evidence also supports the predictive capacity of the adult MAL-QOM on the outcome after CIT in the Extremity Constraint Induced Therapy Evaluation (EXCITE) trial (Park, Wolf, Blanton, Winstein, & Nichols-Larsen, 2008).

The PMAL, a parent-reported outcome measure (Taub et al., 2004), consists of 22 daily activities, including unilateral and bilateral tasks, to capture fine to gross motor functions in children with CP. To date, only one study has investigated the psychometric properties of the PMAL and showed that it has a unidimensional construct with a logical item difficulty hierarchy and good test–retest reliability (Wallen, Bundy, Pont, & Ziviani, 2009). However, its criterion-related validity and ability to detect changes after treatments (e.g., responsiveness) have not been examined yet. Previous research often cited the psychometric and clinimetric findings in the adult MAL for stroke patients to support the use of the PMAL (e.g., Deluca et al., 2006, Taub et al., 2004), but these findings cannot be extrapolated to the PMAL because the item contents and populations are different. The PMAL, as most measures used for research investigating interventions in children with CP, has been criticized in the lack of sufficient study of metric properties (Gordon et al., 2005, Huang et al., 2009). Further explorations in the properties of the PMAL are needed before its wide use in outcome evaluations of interventions for children with CP (Sakzewski et al., 2009).

Criterion-related validity focuses on the degrees of consistency between an instrument and the criterion measures. It includes concurrent and predictive validity. Concurrent validity assesses the consistency at the same assessment time. Predictive validity indicates the ability of an instrument to predict subsequent consequences and is important for prognosis and setting long-term goals in rehabilitation. Because the primary focus of the UE function training for children with CP is to improve their independence in ADL (McMahon, Pruitt, & Vargus-Adams, 2010) and to increase the rate of acquisition of specific developmental milestones, performance in the ADLs and in motor function are good indicators of the treatment outcomes. ADL instruments for children with disabilities, such as the Functional Independence Measure in children (WeeFIM) (Ottenbacher et al., 2000) and standardized developmental assessments that measure motor skills (e.g., the Peabody Developmental Motor Scales-Second Edition [PDMS-2]; Folio & Fewell, 2000) might be adequate criterion measures to establish the criterion-related validity of the PMAL.

Clinimetric properties focus on important changes after a patient receives a treatment. Change scores from pretreatment to posttreatment reflect the effect of treatment and undesired measurement error. Three indices have been advocated to signify an important difference in patient symptoms: responsiveness, minimal detectable change (MDC), and minimal clinically important difference (MCID). Responsiveness refers to the ability of an instrument to detect the efficacy of a treatment. It can be conceptualized as longitudinal validity, indicating longitudinal change over a defined interval during or after treatment (Lauridsen, Hartvigsen, Manniche, Korsholm, & Grunnet-Nilsson, 2006). The MDC indicates the smallest amount of change beyond measurement error that reflects a true difference due to a specific intervention. An instrument with greater reliability usually yields a smaller value of the MDC compared with a measure with lower reliability (Portney & Watkins, 2009). The MCID goes further to provide additional information of changes—the smallest change that is important by a respondent.

Researchers have advocated two approaches (distribution-based and anchor-based) to estimate an important change that is necessary to establish clinical meaningfulness (Dawson et al., 2007, Lemieux et al., 2007). A distribution-based approach expresses changes underlying a specific sampling distribution using statistical significance, sample variability, and measurement precision to estimate the MCID (Haley & Fragala-Pinkham, 2006). An anchor-based approach compares change scores with an external standard to determine the meaning of clinical importance (Copay, Subach, Glassman, Polly, & Schuler, 2007). Because there is no consensus for a standard method to determine the MCID, combinations of distribution- and anchor-based methods to triangulate a range of values to indicate the clinical importance is recommended (Copay et al., 2007).

Clinicians and researchers use measurement as a way to understand, evaluate, and differentiate characteristics of patients and treatments. An instrument with sound psychometric and clinimetric properties plays a significant role for making decisions or drawing conclusions in rehabilitation research. The purpose of this investigation was therefore to examine the criterion-related validity of the PMAL and its clinimetric properties, including responsiveness, MDC, and MCID, in children with CP.