Validation of the Ankylosing Spondylitis Quality of Life assessment tool in patients with non-radiographic axial spondyloarthritis

Data from patients with active axSpA enrolled in a 24-week Phase III, multicenter, randomized, double-blind, placebo-controlled trial (RAPID-axSpA, NCT01087762) [21] were used in this analysis. Eligible patients were ≥18 years with a documented diagnosis of adult-onset axSpA of at least 3 months’ duration as defined by the ASAS axSpA classification criteria, a Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) [22] score ≥4, spinal pain ≥4 on a 0–10 numeric rating scale, C-reactive protein (CRP) greater than the upper limit of normal and/or evidence within three months of screening of sacroiliitis on magnetic resonance imaging (MRI) or X-ray as defined by Assessment of SpondyloArthritis international Society (ASAS)/Outcome Measures in Rheumatology (OMERACT) scoring [23]. All pelvic radiographs and MRI scans were assessed and confirmed by two central readers and, if necessary, an adjudicator. Patients were also required to be intolerant of non-steroidal anti-inflammatory drugs (NSAIDs) or have had an inadequate response to at least one NSAID after at least 30 days of treatment or to two NSAIDs after at least two weeks of treatment with each. The RAPID-axSpA study, from which the patient data used in this study were derived, had been approved by the independent ethics committee or institutional review board at participating sites, and written informed consent obtained from all patients.

Patients were classified as having AS [fulfilling ASAS axSpA classification criteria and modified New York (mNY) classification criteria [6]] or nr-axSpA (fulfilling ASAS axSpA classification criteria but not mNY classification criteria). Patients with nr-axSpA were further classified using the more stringent objective signs of inflammation criteria [24], defined as a Spondyloarthritis Research Consortium of Canada [SPARCC] [25] score ≥2 of MRI scans of the SI joint and/or serum C-reactive protein levels exceeding the upper limit of normal.

The following eight PROs were used in various stages of this evaluation of the ASQoL psychometric properties:

Data were described using standard descriptive statistics to characterize the overall patient population and subpopulations. Response pattern evaluations were also performed to assess inter-item tetrachoric correlations. Cross-sectional analyses, including the modern psychometric methods (MPMs), were conducted on baseline data. This approach enables evaluation of the psychometric properties of the ASQoL prior to any experimental and/or pharmacogenic interventions that could alter the underlying disease evaluated by the ASQoL. These analyses were complemented by sensitivity cross-sectional analyses at later time points.

The total axSpA intent-to-treat patient population was used for all MPMs, with further analyses performed on subpopulations (patients diagnosed with nr-axSpA overall as well as in the subgroups with and without objective signs of inflammation). Readers interested in considerations related to sample size and estimation of these models are directed to the discussion wherein a section is dedicated to the finite sample properties of estimability and bias.

MPMs employed a combination of full information item exploratory factor analysis (EFA) and item response theory (IRT) [32]. These methods generated evidence guiding domain specification, item performance evaluation, assessments of item bias, and scoring.

Following current best practice, the number of domains (factors) was determined from model fit indices [33]. These included the C2 χ² test of absolute fit [33] and the C2-based root mean squared error of approximation (RMSEA) goodness of fit test [34], and standard metrics were used for interpreting the estimates [35]. Interpretability of the final domain solutions was achieved through oblique Quartimax rotation.

Four alternative confirmatory IRT structures were assessed to evaluate item performance, bias, and empirically guide scoring. IRT models considered included a two-parameter logistic (2PL) model [36, 37], a Rasch analog of the 2PL model [38], a bifactor model [39], and a multidimensional item response theory (MIRT) model. In addition to the model fit assessment, item parameter quality and Chen’s local dependence statistic [40] were used to evaluate which IRT model best characterized the performance of the ASQoL items. Items with χ² values exceeding 3 indicated potentially serious local dependence. Items with IRT slopes exceeding 4 were considered to be potentially unstable [41].

Differential item functioning (DIF) was used to assess whether ASQoL items functioned identically between axSpA subpopulations within the final ASQoL domain solution. This was performed using the Wald-2 DIF χ² sweep procedure, with p-values adjusted for the false discovery rate using the Benjamini–Hochberg procedure [42]. Items identified as having significant DIF were further evaluated via a DIF severity assessment [43] to assess whether detected significant DIF would severely impact scores. For an item to be declared biased between axSpA subpopulations it had to demonstrate both significant and severe DIF.

The ASQoL score is presented as either the sum score (sum of score for each ASQoL item; scale of 0–18) or the mean score [sum score divided by 18 (total number of ASQoL items); scale 0–1]. For both, a lower score indicates better quality of life. As there was no item-level missing data, results were identical for any correlation-based analysis. The optimal scoring procedure was determined based on scoring statistics [44]. Four possible scores were considered and scoring statistics characterized the relative merits of each: unit-weighted (with each item given equal weighting) domain scores, unit-weighted total scores, and empirically weighted (each item weighted by its reliability) versions of domain and total scores.

Note that a Supplemental Web Appendix contains the complete tabular and graphical output of the modern psychometric results, and interested readers are directed there for additional evidence.

Internal consistency was assessed to characterize the performance of the ASQoL in addition to guiding scoring decisions. Four possible scores were evaluated via the ω-based statistics: unit-weighted domain scores, unit-weighted total scores, and empirically weighted versions of domain and total scores. Internal consistency was measured by McDonald’s ω statistic and the corresponding bifactor analog, ω_H [44]. These statistics are the least biased internal consistency estimators [42]. Subdomain scores would be supported if ω exceeded ω_H, and total scores would be supported if ω_H exceeded ω. Further, as the ω_H/ω ratio approaches 1, a total domain is favored. Low values (<0.7) on both ω and ω_H indicate a need for empirically weighted scores.

ASQoL score performance was evaluated in terms of the test characteristic curve (TCC) and the precision of score measurement via the test information function (TIF). Additional assessments included estimates of test–retest reliability, validity, ability to detect change (responsiveness) and meaningful within-patient change (MWPC).

Test–retest reliability of the ASQoL responses was estimated correlating Baseline with Week 12 and Week 24 follow-up data. Test–retest reliability was estimated in a group of stable patients, defined as those patients with no change in PGIC, PtGADA (defined as a change in scores between ±1 point), or PhGADA (defined as a change in score between ±15 points). The analysis was based on the two-way random intraclass correlation coefficient (ICC[2, 1]) [45] with estimates of at least 0.7 prespecified as indicating acceptable reproducibility of scores. Given the length of the retest interval and the fact that the retest interval spanned the interventional period of the randomized trial, the evidence presented for test–retest reliability could better be described as long-term stability. To remain consistent with regulatory review and interaction, in this manuscript we retain the description of this evidence as test–retest reliability.

Concurrent validity (both convergent and divergent) was estimated at baseline via Spearman correlations within the nr-axSpA population with objective signs of inflammation. Sensitivity analyses were conducted at Weeks 12 and 24. Convergent validity estimates were obtained by correlating ASQoL total scores and those of the BASDAI, BASFI, PtGADA, PhGADA, total and nocturnal spinal pain numeric rating scales, and the ASDAS composite score. Divergent validity estimates were obtained by correlating the ASQoL total scores with the physical functioning and physical component scores of the SF-36v2. The ability of the tests to detect change was also determined from the Spearman correlation coefficient for change in scores from Baseline to Week 12 and to Week 24 for ASQoL versus other PRO measures. In all cases correlations ≥|0.4| met the prespecified criterion for acceptable validity.

Known-groups validity evidence was generated at Baseline, Week 12 and Week 24. Scores from the PhGADA and ASDAS were dichotomized (median split and cut at 2.1, respectively) to define known groups. The mean differences in ASQoL score between the known groups for each measure were analyzed using analysis of variance (ANOVA).

MWPC was estimated by both distribution and anchor-based methods. Given regulatory emphasis on anchor-based methods, only anchor-based evidence is reported. The anchor-based method for MWPC estimation was based on patients whose change in ASQoL score between Baseline and Week 12 and Week 24 was equal to or greater than the estimated median change in ASQoL score in patients with a PGIC of 6 (moderate improvement) or 5 (minimal improvement). In addition, the MWPC point estimate was validated via empirical cumulative distribution functions (eCDFs) and 95% Clopper-Pearson confidence bands for change in ASQoL score from Baseline, stratified by PGIC strata (no change, minimal improvement, moderate improvement, marked improvement).

All analyses used observed case data only; no imputation of missing values was undertaken.

MPMs were conducted using FlexMIRT version 3.5 (Vector Psychometric Group). All other analyses were performed using a combination of Statistical Analysis Software version 9.4 (SAS^® Institute Inc., Cary, NC) and R statistical software version 3.4.3 (R Development Core Team).

These assessments were conducted in the total patient population (N = 325). Response pattern evaluation showed that all items in the ASQoL were well endorsed (>20%, a common boundary for insufficient endorsement [46]). The lowest endorsement rates were 22.2% for difficulties in washing hair (ASQoL item 16) and 37.0% for being unable to do chores (ASQoL item 11); all other items had endorsement values of >54%. No items crossed the common over-endorsement criterion of 90%; the strongest endorsements were 89.6% for morning delay, 84.9% for being easily tired, and 83.1% for being always in pain (ASQoL items 10, 12, and 14, respectively).

Results from the EFA showed that the ASQoL item response associations could be explained by four domains as this achieved an optimal fit with the fewest extracted factors. This solution satisfied the C2 χ² test of absolute fit (p = 0.27) as did the RMSEA and corresponding 90% CI (0.02 [0.00, 0.04]). Oblique quartimax rotation of this solution yielded a logical mixture of symptoms and impacts relating to the ASQoL items. Observed domains characterized: activities of daily living and pain; sleep disturbance and activity limitation; emotion; and fatigue. These domains were tested in the MIRT and bifactor IRT models described next.

The four confirmatory IRT structures could not be differentiated based on the χ² test of absolute fit (all p < 0.005). However, both of the unidimensional models were rejected as they showed evidence of strong local dependence. Although the bifactor model showed a large local dependence estimate between ASQoL items 13 (frustrated) and 7 (always fatigued), it was the only model that satisfied the goodness of fit test (RMSEA [90% CI]: 0.03 [0.000, 0.049]). The bifactor model demonstrated acceptable IRT slope stability, with only ASQoL items 4 (struggle to do chores) and 12 (easily fatigued) having slopes >3 (Table 2). Taking all of these findings into consideration, the bifactor model was selected as the final model from which test and score properties would be evaluated.

Scoring statistics supported the use of a unit-weighted total score. The total score accounted for 90% of the explainable internal consistency and the subdomains accounted for only the remaining 10% of explainable internal consistency. The explained common variance associated with the total domain was 0.73, and the factor determinacy statistic (H) for the total domain was 0.95. These findings meet the standards set by Rodriguez, Reise, and Haviland [44] for favoring a unit-weighted total score. This provides empirical support for the developer’s scoring algorithm. All evidence presented hereafter pertains to a unit-weighted total score.

ASQoL mean score performance evaluated by the TCC demonstrated a strong score discrimination function, indicating that patients 0.75 standard deviations above the mean on total AS severity endorsed 50% of the ASQoL items (Fig. 1). The TIF demonstrated that score precision was maximized for patients located between 0.5 and 1.5 standard deviations above the mean on total AS severity (Fig. 2). Therefore, the ASQoL detected and characterized AS of moderate to moderately high severity, defined as scores falling between 0.5 and 1.5 standard deviations above the mean.

DIF testing identified significant DIF between patients with AS or nr-axSpA (classified by mNY criteria) for ASQoL items 7 (always fatigued; p = 0.0034) and 12 (easily fatigued; p = 0.0036). However, the DIF severity effect size and corresponding weighted area between curve (wABC) plots demonstrated that this significant DIF was not meaningful. None of the ASQoL items demonstrated significant DIF (p ≥ 0.0685) between nr-axSpA patient subgroups with or without objective signs of inflammation. Therefore, no meaningful DIF was detected in the ASQoL and there was no difference in its function between patients with AS or nr-axSpA irrespective of the method of classification. Note that when DIF analyses are based upon either a focal or reference group as small as the focal group in this application (n = 67), DIF can yield either increased Type I or Type II error rates, thus motivating the need for graphical scrutiny of DIF via the wABC.

These assessments were performed in the nr-axSpA with objective signs of inflammation patient population (n = 67) and compared with the population having AS or nr-axSpA without objective signs of inflammation (n = 258). Unit-weighted ASQoL mean scores, expressed as the proportion of the 18 items with a positive response, were 0.75 at Baseline (n = 67), 0.54 at Week 12 (n = 59), and 0.35 at Week 24 (n = 56).

The ASQoL total score had a high internal consistency (ω_H = 0.82) within this population. Test–retest reliability, anchored on no change in PGIC, PtGADA, or PhGADA score between Baseline and Week 12 or Week 24, gave ICC(2,1) estimates that exceeded the prespecified criterion of 0.7.

Concurrent validity of the ASQoL at Baseline exceeded the prespecified criterion for acceptable validity for all validators (r ≥ |0.50|), excepting PhGADA and ASDAS composite score (r = 0.24 and r = 0.34, respectively). Sensitivity analyses at Week 12 and Week 24 exceeded the strength of baseline findings with no validators failing to meet the prespecified criterion of acceptable validity.

While baseline known-groups validity did not detect significant differences, results at both Week 12 and Week 24 demonstrated that significantly worse mean ASQoL scores were seen for patients with ASDAS >2.1 or with PhGADA at or above the median value compared to patients in the known reference groups (p ≤ 0.002 and p ≤ 0.003, respectively). On average, patients with ASDAS >2.1 endorsed 32% more ASQoL items at Week 12 and 27% more items at Week 24 compared with patients with ASDAS <2.1. Similarly, patients with PhGADA scores at or above the median value endorsed approximately 29% and 26% more items at Weeks 12 and 24, respectively, compared with patients with PhGADA values below the median value.

The sensitivity of the ASQoL to detect changes in the PRO assessments between Baseline and Week 12 or Week 24 met or exceeded the standard criterion of r ≥ |0.4| for all assessments except for PtGADA at Week 12 (r = 0.36). Based on the PGIC moderate improvement anchor group, the point estimate (using ASQoL mean score) for MWPC at Week 12 was –0.22 and at Week 24 was –0.19, representing an improvement in approximately four of the 18 ASQoL items.

The eCDF for change in ASQoL score stratified by PGIC anchor groups is presented in Fig. 3. The cumulative percentage of patients with PGIC-based moderate improvement versus those with no change meeting or exceeding the MWPC point estimate (4-point improvement, ASQoL sum score) were 54.5 and 8.3%, respectively, at Week 12 (a 46.2 percentage point advantage). At Week 24 these same cumulative percentages were 50.0 and 12.5%, respectively (a 37.5 percentage point advantage). This evidence supports the use of the 4-point improvement estimate for clinically meaningful change within this population.