Development of an Instrument to Measure Behavioral Health Function for Work Disability: Item Pool Construction and Factor Analysis

doi:10.1016/j.apmr.2013.03.013

Archives of Physical Medicine and Rehabilitation

Volume 94, Issue 9, September 2013, Pages 1670-1678

https://doi.org/10.1016/j.apmr.2013.03.013 Get rights and content

Abstract

Objectives

To develop a broad set of claimant-reported items to assess behavioral health functioning relevant to the Social Security disability determination processes, and to evaluate the underlying structure of behavioral health functioning for use in development of a new functional assessment instrument.

Design

Cross-sectional.

Setting

Community.

Participants

Item pools of behavioral health functioning were developed, refined, and field tested in a sample of persons applying for Social Security disability benefits (N=1015) who reported difficulties working because of mental or both mental and physical conditions.

Interventions

None.

Main Outcome Measure

Social Security Administration Behavioral Health (SSA-BH) measurement instrument.

Results

Confirmatory factor analysis (CFA) specified that a 4-factor model (self-efficacy, mood and emotions, behavioral control, social interactions) had the optimal fit with the data and was also consistent with our hypothesized conceptual framework for characterizing behavioral health functioning. When the items within each of the 4 scales were tested in CFA, the fit statistics indicated adequate support for characterizing behavioral health as a unidimensional construct along these 4 distinct scales of function.

Conclusions

This work represents a significant advance both conceptually and psychometrically in assessment methodologies for work-related behavioral health. The measurement of behavioral health functioning relevant to the context of work requires the assessment of multiple dimensions of behavioral health functioning. Specifically, we identified a 4-factor model solution that represented key domains of work-related behavioral health functioning. These results guided the development and scale formation of a new SSA-BH instrument.

Section snippets

Behavioral control item pool development

The SSA-BH item pool examined in this study was grounded on a theoretic framework that characterizes behavioral health functioning across 5 hypothesized domains: behavioral control, temperament and personality, adaptability, basic interactions, and workplace behaviors.²⁶ The initial content model served as the conceptual foundation for constructing items and was developed using a structured literature review, expert interviews, and stakeholder feedback. The item pool consisted of 165 questions

Results

Demographic data from the calibration study characterizing the SSA claimant and normative samples are presented in table 2. The SSA claimant sample was approximately 56% women, predominantly (61%) white, 73% from urban areas, and had an average age of 44±11 years. With regard to the work-related disability that was motivating their application for benefits, 20% of the SSA claimant sample reported having a mental condition only (eg, depression, anxiety, bipolar, posttraumatic stress disorder),

Discussion

Empirical evidence, conceptual elements, and utility of the instrument were all important components of the model refinement process. We consulted an expert panel to impart conceptual meaning and clinical relevance to the items specified in each solution. In addition, the goal of maintaining the ability to crosswalk with legacy instruments was a priority. With feedback from the experts, we reached consensus on the modified 4-factor model representing the solution that aligned best with the

Conclusions

The development of the SSA-BH instrument represents a groundbreaking effort in moving SSDI assessment processes forward both conceptually and psychometrically. The items used in the SSA-BH instrument offer an opportunity to assess functioning from a “whole-person” perspective as compared with traditional assessment methods based on the medical model of disability. This conceptual shift provides a unique opportunity to assess behavioral health functioning across a variety of key domains that may

Suppliers

a.
Muthén & Muthén, 3463 Stoner Ave, Los Angeles, CA 90066.
b.
SAS Institute Inc, 100 SAS Campus Dr, Cary, NC 27513.
c.
SPSS Inc, 233 S Wacker Dr, 11th Fl, Chicago, IL 60606.

References (37)

D.S. Tulsky et al.
Spinal Cord Injury-Functional Index: item banks to measure physical functioning in individuals with spinal cord injury
Arch Phys Med Rehabil
(2012)
D. Cella et al.
The neurology quality-of-life measurement initiative
Arch Phys Med Rehabil
(2011)
S.M. Haley et al.
Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank
J Clin Epidemiol
(2006)
E.E. Marfeo et al.
Scale refinement and initial evaluation of a behavioral health function measurement tool for work disability evaluation
Arch Phys Med Rehabil
(2013)
Social Security Administration. Annual statistical report on the Social Security Disability Insurance program, 2011....
A. Cieza et al.
The ICF as a conceptual platform to specify and discuss health and health-related concepts
Gesundheitswesen
(2008)
R. Escorpizo et al.
A conceptual definition of vocational rehabilitation based on the ICF: building a shared global model
J Occup Rehabil
(2011)
K. Giannangelo et al.
ICF: representing the patient beyond a medical classification of diagnoses
Perspect Health Inf Manag
(2005)
W.A. Anthony et al.
Relationships between psychiatric symptomatology, work skills, and future vocational performance
Psychiatr Serv
(1995)
H. Tsang et al.
Predictors of employment outcome for people with psychiatric disabilities: a review of the literature since the mid '80s
J Rehabil
(2000)

K. MacDonald-Wilson et al.

Unique issues in assessing work function among individuals with psychiatric disabilities

J Occup Rehabil

(2001)

Reeve B. Applications of item response theory (IRT) modeling for building and evaluating questionnaires measuring...

D.E. Brandt et al.

Connecting contemporary paradigms to Social Security Administration's disability evaluation process

J Disabil Policy Stud

(2011)

Social Security Administration. Social Security Administration: titles II and XVI: residual functional capacity for...

Social Security Administration

Administrative review process for adjudicating initial disability claims. Final rule

Fed Regist

(2006)

Social Security Administration. Annual statistical report on the Social Security Disability Insurance program, 2010....

K. Sanderson et al.

Mental health in the workplace: using the ICF to model the prospective associations between symptoms, activities, participation and environmental factors

Disabil Rehabil

(2008)

L.F. Bufka et al.

The International Classification of Functioning, Disability and Health (ICF) is emerging as the principal framework for the description of health and health related status

Can J Occup Ther

(2008)

Cited by (25)

National Academies Consensus Report on Functional Assessment for Adults With Disabilities
2019, Archives of Physical Medicine and Rehabilitation
The Work Disability Functional Assessment Battery (WD-FAB)
2019, Physical Medicine and Rehabilitation Clinics of North America
Citation Excerpt :
The WD-FAB Mental Health Function content model presented in Fig. 2 suggests there are 5 major domains: (1) Behavior control, (2) Basic interactions, (3) Temperament and personality, (4) Adaptability, and (5) Workplace behaviors. These 5 key domains were developed based on the ICF, other selected models of work disability, and more theoretic literature discussing aspects of human behavior, personality, and social skills.14,15 Subdomains of the primary mental health concept represent individual components, such as mood and emotions, behavioral modulation adaptation, adaptability, temperament and personality, and interpersonal interaction skills that may act independently or interact to characterize mental health functioning.
Psychometric Evaluation of the Improved Work-Disability Functional Assessment Battery
2019, Archives of Physical Medicine and Rehabilitation
To assess psychometric properties of the improved Work Disability Functional Assessment Battery (WD-FAB 2.0).
Longitudinal study.
Community.
Three samples of working-age (21-66) adults (N=1006): (1) unable to work because of a physical condition (n=375); (2) unable to work because of a mental health condition (n=296); (3) general United States working age sample (n=335).
NA.
All samples completed the WD-FAB 2.0; the second administration came 5 days after the first. Construct validity was examined by convergent and divergent correlational analysis using legacy measures. Test-retest reliability was assessed by intraclass correlation coefficients (ICC_3,1). Standard error of the mean (SEM) and minimal detectable change (MDC₉₀) were calculated to measure scale precision and sensitivity.
Physical function ICCs ranged from 0.69 to 0.77 in the general sample, and 0.66 to 0.86 in the disability sample. Mental health function scales ICCs ranged from 0.62 to 0.73 in the general sample, and 0.74 to 0.76 in the disability sample. SEMs for all scales indicated good discrimination; those for the physical function scales were generally lower than those for the mental health scales. MDC₉₀ values ranged from 3.41 to 10.55. Correlations between all WD-FAB 2.0 scales and legacy measures were in the expected direction.
The study provides substantial support for the reliability and construct validity of the WD-FAB 2.0 among 3 diverse samples. Although initially developed for use within the Social Security Administration (SSA), these results suggest that the WD-FAB 2.0 could be used for assessment and measurement of work-related physical and mental health function in other contexts as well.
Validation of the Work-Disability Physical Functional Assessment Battery
2018, Archives of Physical Medicine and Rehabilitation
Citation Excerpt :
Thus, the scores from this sample are not representative of SSA claimants, and should not be used more broadly for score interpretation. Prior studies of large claimant and general population samples were designed to guide score interpretation.4-6,8,10 The characteristics of the sample limited the variance in scores for the WD-FAB and ratings for the PWPE.
To examine the validity of the self-report Work-Disability Functional Assessment Battery (WD-FAB) physical function scales relative to clinician ratings of function and a performance-based functional capacity evaluation called the Physical Work Performance Evaluation (PWPE).
Cross-sectional.
Outpatient rehabilitation.
Adults (N=50) participating in physical therapy for musculoskeletal conditions.
Not applicable.
Patients completed the PWPE and the WD-FAB physical function scales including Changing and Maintaining Body Position, Whole Body Mobility, Upper Body Function, and Upper Extremity Fine Motor. The physical therapist also answered the WD-FAB questions on the patient’s physical functioning. The WD-FAB computer-adaptive test version administered up to 10 items for each scale. The PWPE produces ratings from 0 to 5 indicating overall Level of Work ability: 0 (unable to work); 1 (sedentary); 2 (light); 3 (medium); 4 (heavy); 5 (very heavy). The PWPE also produces Level of Work ability ratings in the Dynamic Strength, Position Tolerance, and Mobility subsections.
Participating in the study were 50 patients with 1 or more conditions (shoulder, n=21; knee, n=16; low back, n=13; ankle/foot, n=10; neck, n=8; hip, n=7). The patient-based WD-FAB scores demonstrated moderate, statistically significant correlations with the provider proxy WD-FAB report (R=.49-.65). The WD-FAB Upper Body Function scale scores demonstrated moderate strength relationships with the PWPE overall ratings. The Whole Body Mobility and Changing and Maintaining Body Position scales did not demonstrate statistically significant relationships with the PWPE overall ratings.
We found moderate evidence for validity for the WD-FAB Upper Body Function, Whole Body Mobility, and Changing and Maintaining Body Position scales relative to clinician report and varied evidence relative to the PWPE in this clinical sample.
Work-related measures of physical and behavioral health function: Test-retest reliability
2015, Disability and Health Journal
The Work Disability Functional Assessment Battery (WD-FAB), developed for potential use by the US Social Security Administration to assess work-related function, currently consists of five multi-item scales assessing physical function and four multi-item scales assessing behavioral health function; the WD-FAB scales are administered as Computerized Adaptive Tests (CATs).
The goal of this study was to evaluate the test-retest reliability of the WD-FAB Physical Function and Behavioral Health CATs.
We administered the WD-FAB scales twice, 7–10 days apart, to a sample of 376 working age adults and 316 adults with work-disability. Intraclass correlation coefficients were calculated to measure the consistency of the scores between the two administrations. Standard error of measurement (SEM) and minimal detectable change (MDC₉₀) were also calculated to measure the scales precision and sensitivity.
For the Physical Function CAT scales, the ICCs ranged from 0.76 to 0.89 in the working age adult sample, and 0.77–0.86 in the sample of adults with work-disability. ICCs for the Behavioral Health CAT scales ranged from 0.66 to 0.70 in the working age adult sample, and 0.77–0.80 in the adults with work-disability. The SEM ranged from 3.25 to 4.55 for the Physical Function scales and 5.27–6.97 for the Behavioral Health function scales. For all scales in both samples, the MDC₉₀ ranged from 7.58 to 16.27.
Both the Physical Function and Behavioral Health CATs of the WD-FAB demonstrated good test-retest reliability in adults with work-disability and general adult samples, a critical requirement for assessing work related functioning in disability applicants and in other contexts.
Work disability functional assessment battery: Feasibility and psychometric properties
2015, Archives of Physical Medicine and Rehabilitation
Citation Excerpt :
All items are scored such that higher scores indicate higher functioning. Each raw BH scale score was standardized on a national normative sample and then converted to a T score (mean score ± SD, 50±10).7 Respondents reported their sex, marital status, race, ethnicity, age, when they became unable to work, and whether they were receiving disability support benefits.
To assess the feasibility and psychometric properties of 8 scales covering 2 domains of the newly developed Work Disability Functional Assessment Battery (WD-FAB): physical function (PF) and behavioral health (BH) function.
Cross-sectional study.
Community.
Adults (N=973) unable to work because of a physical (n=497) or a mental (n=476) disability.
Not applicable.
Each disability group responded to a survey consisting of the relevant WD-FAB scales and existing measures of established validity. The WD-FAB scales were evaluated with regard to data quality (score distribution, percentage of “I don't know” responses), efficiency of administration (number of items required to achieve reliability criterion, time required to complete the scale) by computerized adaptive testing (CAT), and measurement accuracy as tested by person fit. Construct validity was assessed by examining both convergent and discriminant correlations between the WD-FAB scales and scores on same-domain and cross-domain established measures.
Data quality was good, and CAT efficiency was high across both WD-FAB domains. Measurement accuracy was very good for PF scales; BH scales demonstrated more variability. Construct validity correlations, both convergent and divergent, between all WD-FAB scales and established measures were in the expected direction and range of magnitude.
The data quality, CAT efficiency, person fit, and construct validity of the WD-FAB scales were well supported and suggest that the WD-FAB could be used to assess PF and BH function related to work disability. Variation in scale performance suggests the need for future work on item replenishment and refinement, particularly with regard to the Self-Efficacy scale.

View all citing articles on Scopus

: Supported by SSA-NIH Interagency Agreements (NIH contract nos. HHSN269200900004C, HHSN269201000011C, HHSN269201100009I) and by the NIH intramural research program.

: No commercial party having a direct financial interest in the results of the research supporting this article has conferred or will confer a benefit on the authors or on any organization with which the authors are associated.

^†: Deceased.

View full text

Special communicationDevelopment of an Instrument to Measure Behavioral Health Function for Work Disability: Item Pool Construction and Factor Analysis

Abstract

Objectives

Design

Setting

Participants

Interventions

Main Outcome Measure

Results

Conclusions

Section snippets

Behavioral control item pool development

Results

Discussion

Conclusions

Suppliers

Arch Phys Med Rehabil

Arch Phys Med Rehabil

J Clin Epidemiol

Arch Phys Med Rehabil

The ICF as a conceptual platform to specify and discuss health and health-related concepts

Gesundheitswesen

A conceptual definition of vocational rehabilitation based on the ICF: building a shared global model

J Occup Rehabil

ICF: representing the patient beyond a medical classification of diagnoses

Perspect Health Inf Manag

Relationships between psychiatric symptomatology, work skills, and future vocational performance

Psychiatr Serv

Predictors of employment outcome for people with psychiatric disabilities: a review of the literature since the mid '80s

J Rehabil

Unique issues in assessing work function among individuals with psychiatric disabilities

J Occup Rehabil

Connecting contemporary paradigms to Social Security Administration's disability evaluation process

J Disabil Policy Stud

Administrative review process for adjudicating initial disability claims. Final rule

Fed Regist

Mental health in the workplace: using the ICF to model the prospective associations between symptoms, activities, participation and environmental factors

Disabil Rehabil

The International Classification of Functioning, Disability and Health (ICF) is emerging as the principal framework for the description of health and health related status

Can J Occup Ther

Special communication
Development of an Instrument to Measure Behavioral Health Function for Work Disability: Item Pool Construction and Factor Analysis