Development of an ultra-short measure of eight domains of health-related quality of life for research and clinical care: the patient-reported outcomes measurement information system® PROMIS®-16 profile

The goal of the developmental approach, conducted in three phases, was to select the best item pair to represent each domain. In the first phase, we conducted an empirical evaluation of all candidate PROMIS items and item pairs using data from the MTurk development sample (N = 5775) to identify item pairs with relatively poor performance. This included basic descriptive information and performance of domain-specific T-scores for all item pairs relative to the gold standard (correlations with the gold standard and standardized mean differences from the gold standard with Cohen’s d) [13]. We also asked the stakeholder panel to select, for each HRQOL domain, the two items that ‘taken together, best reflect the domain’ based on item content. Ten of the thirteen stakeholders contributed initial ratings. We used the results from phase 1 to rule out several candidate pairs per domain.

In phase 2, we held our first stakeholder meeting in which we summarized the findings from phase 1, including the stakeholder preferences and discussed the remaining candidate pairs considering their content and psychometric information relative to the gold standard to agree on a reduced set of candidate pairs for further consideration. Item pair performance was presented graphically using item response theory (IRT)-based information curves [14‐16]. These curves display information (presented on the y-axis) as a continuous function that varies according to the underlying domain score (presented on the x-axis). Estimates of precision (standard error and reliability) can be derived from information, and the presentation of multiple item pairs on a single plot effectively display their relative performance. Higher information magnitude reflects increased reliability and lower standard error.

Phase 3 included a second survey of the stakeholder panel and the MTurk preference sample to solicit their preferences between remaining candidate pairs for each domain. A total of nine stakeholders and 124 adult MTurk respondents provided preference ratings at this phase. We arrived at a proposed final PROMIS-16-item set, selected based on the preference ratings, and held a second stakeholder meeting to review the set’s basic descriptive statistics and obtain stakeholder approval for the final PROMIS-16 items.

Across the eight HRQOL domains, empirical analyses revealed limited variability in the performance of the item pairs but did highlight some as performing better than others (see Table 3). In general, the T-score means and ranges showed values clustered around the population mean of 50, although anxiety and depression were slightly worse, whereas social role participation was slightly better. Correlations among items within each domain varied somewhat, with the largest ranges for the physical function and sleep disturbance domains. The average correlation among items was highest for pain interference and lowest for sleep disturbance. Item pair correlations with the gold standard were more consistent, although pairs composed of items from the PROMIS-29 + 2 were more highly correlated. A similar pattern was seen in the standardized mean differences of item pairs with the gold standard. Effect sizes for these mean differences tended to be small, although some exceeded 0.2 (small effect) within the physical function domain.

Stakeholder preferences were quite varied for physical function, social roles, anxiety, and depression and somewhat more consistent for sleep disturbance, pain interference, cognitive function, and fatigue (see rightmost column of Supplement Table S2).

We considered empirical IRT information functions and stakeholder ratings from phase 1, as well as IRT item parameters (thresholds and discrimination), to exclude some item pairs and prioritize others, reducing the number of pairs in each domain for further discussion during the stakeholder meeting. Fig. 1a–h displays IRT information curves for the remaining pairs plotted together with the gold standard for each domain and reveal variable degrees of precision across the domain score continua among the remaining set of item pairs for each domain.

During the discussion of each HRQOL domain, stakeholders considered the relative merits of item pairs that provided reasonable precision (reliability > 0.8) [15, 16] across a wide range of the T-score continuum, rejecting some item pairs based on content preferences and others based on format. For example, in the physical function domain, stakeholders noted that items in pair PF_26 use different formats and response options, mixing item stem introductions Are you able… with Does your health now limit you… and response options Without any difficulty—Unable to do with Not at all—Cannot do. In another example, stakeholders noted that the candidate items for social roles could be separated into two content groups, one representing more leisure or recreational roles (items 1, 4, and 5) and the other reflecting responsibility or work-related roles (items 2, 3, and 6) and recommended selecting an item pair representing these two aspects of social role participation. In this way, the stakeholders narrowed down the set of candidate pairs for all domains substantially during the meeting, reaching consensus on the final item pairs for sleep disturbance and fatigue. Following the meeting, the study team synthesized the stakeholder discussion points with the empirical evidence and item parameters and narrowed the set of remaining pairs to two pair options per domain.

Preference ratings from the stakeholders and MTurk preference sample respondents were remarkably consistent and the final item pair for each HRQOL domain was selected based on these ratings (see Table 4). Stakeholders had no objections to the selected item set when presented with the psychometric performance of the 16 items at the second stakeholder meeting.

The final version of the PROMIS-16 Profile contains 16 items measuring eight HRQOL domains with two items per domain. The measurement precision of the two-item domain scores is displayed in Fig. 1a–h, wherein a line with a marker depicts the selected item sets. The two-item sets provide acceptable precision across a moderate range of the score continuum for all domains. Although information curves for some domains fall below reliability of 0.7 at the low or high ends of the score continua, this tendency is also evident in the 4-item scales. The means of the domain T-scores ranged from 49.3 for fatigue to 54.8 for anxiety (see Table 5). Across the eight domains, there were moderate to strong correlations between items, ranging from 0.50 for cognitive function to 0.77 for anxiety and pain interference. All domains were highly correlated with the gold standard. This correlation exceeded 0.90 for all domains except sleep disturbance which correlated at 0.80. This result is expected given that the PROMIS-16 sleep disturbance domain does not share any items with the PROMIS-29, whereas other domains have some degree of item overlap. The standard mean differences of the final pairs with the gold standard were small, with five of the eight domains showing absolute effect sizes ≤ 0.060; absolute effect sizes for physical function (Cohen’s d = 0.11), ability to participate in social roles and activities (Cohen’s d = − 0.14), and cognitive function—abilities (Cohen’s d = − 0.16), while still considered trivial, exceeded 0.1. Table 6 shows the intercorrelations among domain scores for the PROMIS-16 (above diagonal) and the PROMIS-29 + 2 (below diagonal). The pattern and magnitude of relationships look similar across the two sources. Table 7 contains the item content, response options, and response frequencies for the PROMIS-16 by domain. In most cases, item response frequencies are distributed across the five response options, although the more extreme response options tend to have low endorsement rates. The table format and layout reflect the suggested format for administration. A version for administration is provided as Supplement Table S3. Because pairs of items with five response options produce a limited number of response patterns, the domain scoring of the PROMIS-16 is straightforward to document. Supplement Table S4 provides a scoring look-up table for the PROMIS-16, listing T-scores by domain for each item-pair response pattern.