Article
Score comparability of short forms and computerized adaptive testing: simulation study with the activity measure for post-acute care 1,

https://doi.org/10.1016/j.apmr.2003.08.097Get rights and content

Abstract

Haley SM, Coster WJ, Andres PL, Kosinski M, Ni P. Score comparability of short forms and computerized adaptive testing: simulation study with the Activity Measure for Post-Acute Care. Arch Phys Med Rehabil 2004;85:661–666.

Objective

To compare simulated short-form and computerized adaptive testing (CAT) scores to scores obtained from complete item sets for each of the 3 domains of the Activity Measure for Post-Acute Care (AM-PAC).

Design

Prospective study.

Setting

Six postacute health care networks in the greater Boston metropolitan area, including inpatient acute rehabilitation, transitional care units, home care, and outpatient services.

Participants

A convenience sample of 485 adult volunteers who were receiving skilled rehabilitation services.

Interventions

Not applicable.

Main outcome measures

Inpatient and community-based short forms and CAT applications were developed for each of 3 activity domains (physical & mobility, personal care & instrumental, applied cognition) using item pools constructed from new items and items from existing postacute care instruments.

Results

Simulated CAT scores correlated highly with score estimates from the total item pool in each domain (4- and 6-item CAT r range, .90–.95; 10-item CAT r range, .96–.98). Scores on the 10-item short forms constructed for inpatient and community settings also provided good estimates of the AM-PAC item pool scores for the physical & movement and personal care & instrumental domains, but were less consistent in the applied cognition domain. Confidence intervals around individual scores were greater in the short forms than for the CATs.

Conclusions

Accurate scoring estimates for AM-PAC domains can be obtained with either the setting-specific short forms or the CATs. The strong relationship between CAT and item pool scores can be attributed to the CAT’s ability to select specific items to match individual responses. The CAT may have additional advantages over short forms in practicality, efficiency, and the potential for providing more precise scoring estimates for individuals.

Section snippets

Participants

This study included a sample of 485 participants who were receiving rehabilitation services from 6 health provider networks in the greater Boston area for neurologic (eg, stroke, Parkinson’s disease), orthopedic (eg, hip fracture), or complex medical conditions. Further details of the sample are provided elsewhere on the AM-PAC short forms.15

AM-PAC item pools

Three distinct activity item pools were developed from a combination of new items and items from existing functional instruments currently used in

Score comparability

Correlations between simulated score estimates of the CATs and the total item pools indicate a high degree of correspondence. The 10-item CATs all performed exceedingly well across the 3 activity domains, with an r range of correlations between .96 and .98. The correlations between scores from the 4- to 6-item CATs and the total item pool scores (r) were all greater than or equal to .90. The applied cognition CAT correlations were slightly lower than the physical & mobility and personal care &

Discussion

The results of these simulation analyses suggest that, using the 1-parameter Rasch model to score persons, both the short forms and CAT applications yield good estimates of the underlying functional domain scores with various subsets of the total item pool. The ability of either the short forms or the CAT to perform in this manner, however, depends largely on how well the range of difficulty of the total item pool corresponds to the functioning ability of the sample. As reported previously, the

Conclusions

Score comparability and precision of simulated short forms and CAT applications were compared with scores generated from the entire item pool across 3 domains of functional activities. We demonstrated good to excellent short-form and CAT score comparability with the entire item pool. Short forms were less precise than CATs for estimation of individual scores. Future research should examine how these short forms and CAT programs work in practice to monitor functional recovery over time, in order

Acknowledgements

We thank all the clinical sites who participated in our study: Boston Medical Center, Spaulding Rehabilitation Hospital Network, New England Baptist Hospital, Northeast Rehabilitation Health Network, HealthSouth/New England Rehabilitation Hospital, and Jewish Memorial Hospital and Rehabilitation Center.

References (23)

  • M.E Cohen et al.

    The tools of disability outcomes research functional status measures

    Arch Phys Med Rehabil

    (2000)
  • S.M Haley et al.

    Short-form Activity Measure for Post-Acute Care

    Arch Phys Med Rehabil

    (2004)
  • Classifying and reporting functional status

    (2002)
  • A.M Jette et al.

    A comparison of functional outcome instruments used in postacute care

    Health Care Serv Rev

    (2003)
  • S.M Haley et al.

    RRTC for measuring rehabilitation outcomesextending the frontier of rehabilitation outcome measurement and research

    J Rehabil Outcome Meas

    (2000)
  • Development for MDS-module for postacute care

    (1999)
  • D.L Wilkerson et al.

    Clinical program monitoring systemscurrent capability and future directions

  • C.A Velozo et al.

    The use of Rasch analysis to produce scale-free measurement of functional ability

    Am J Occup Ther

    (1999)
  • M.V Johnston et al.

    Outcomes research in medical rehabilitationfoundations from the past and directions for the future

  • W Fisher

    Physical disability construct convergence across instrumentstowards a universal metric

    J Outcome Meas

    (1997)
  • C McHorney

    Generic health measurementpast accomplishments and a measurement paradigm for the 21st century

    Ann Intern Med

    (1997)
  • Cited by (82)

    View all citing articles on Scopus

    Supported in part by the National Institute on Disability and Rehabilitation Research (grant no. H133B990005), the National Institute of Child Health and Human Development (grant no. R01 HD43568), and the Agency for Healthcare Research and Quality. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the funders.

    1

    No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the author(s) or upon any organization with which the author(s) is/are associated.

    View full text