Assessing the construct validity of the Quality-of-Life-Aged Care Consumers (QOL-ACC): an aged care-specific quality-of-life measure

Participants were older people receiving community-aged care services in their own homes via Commonwealth Home Support Programme (CHSP) or Home Care Packages (HCP) [2]. The CHSP provides entry level support services (e.g. meals, help with basic chores, home maintenance etc.) whereas HCP provides support services for more complex care (e.g. home modifications, personal care, nursing services, allied health etc.) at four levels (1 = basic to 4 = high) [13]. All study participants were recruited through an online panel as face-to-face interviewer-facilitated data collection in individual’s own homes was not possible due to COVID-19 restrictions at the time of the study. The online panel utilised for this study is an Australia-wide online panel network comprising of over 10,000 nationally representative Australians aged 65 years and older. Panel members who were aged 65 years and over, able to read and respond in English, living in Australia and currently receiving CHSP or HCP services were invited to take part in the survey. Quotas for recruitment were set by age, gender and types of care to ensure broad representation. All study participants provided online consent prior to completing the survey. The study was approved by the Human Research Ethics Committee at Flinders University (application numbers 8399 and 2201) and adheres to the tenets of the Declaration of Helsinki.

The content of the QOL-ACC was identified from in-depth qualitative interviews with older people receiving aged care services exploring what quality of life meant to them supplemented by a comprehensive literature review [12, 14]. A set of draft items were developed across QOL-ACC dimensions (mobility, pain management, independence, emotional well-being, social connections and activities). A mixed method approach using a traffic light system was used to integrate qualitative (face validity) and quantitative (psychometric assessments) evidence to develop the final descriptive system for the QOL-ACC [15]. The final QOL-ACC descriptive system has 6 dimensions with single item across all the dimensions rated on a 5 a five-point frequency scale (Table 1).

Survey respondents also completed four other instruments, two global items of quality of life and health status rated on a 5-point response scale (Poor to Excellent) and socio-demographic details. Using postcode data (geographical areas of their residents), two indices (Index of Relative Socio-economic Advantage and Disadvantage, IRSAD and Index of Education and Occupation, IEO) of socio-economic well-being were estimated using methodology described by the Australian Bureau of Statistics [16].

The EQ-5D-5L is a generic preference-based HRQOL utility instrument which provides descriptions and valuations of five dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression) of health status on a 5-level scale (no problems to extreme problems). It is one of the most widely used utility instruments and various sets of preference-based utility values have been developed across cross-national general population samples [17]. For this study, we used the Australian preference weights developed by Norman et al. [18]. The EQ-VAS is a vertical visual analogue scale of self-reported health which ranges from 0 (worst possible health one can imagine) to 100 (best possible health one can imagine). The EQ-VAS scores were used as a standalone measure of health reflecting the respondents’ own judgement of their health status.

The ASCOT (Adult Social Care Outcome Tool) is a preference based social-care-related QOL instrument [19, 20]. It has eight domains: personal cleanliness and comfort, food and drink, control over daily life, safety, accommodation cleanliness and comfort, social participation and involvement, occupation and dignity. Each domain is framed as “which of the following statements best describes...” and rated on a four response levels representing four different outcome status (Ideal, the preferred situation; No needs, where needs are made; Some needs, where there are needs but no immediate/long-term health implications; High needs, where needs have immediate and long-term health implications),[20] and the ASCOT total preference-weighted scores for English general population range from -0.17 to 1.0 with higher scores representing better QOL [19].

The QCE-ACC (Quality of Care-Aged Care Consumers) is a preference-based weighted measure of aged care-specific quality of care. The QCE-ACC was developed from a study commissioned by the Royal Commission into Aged Care Quality and Safety in Australia [21]. The QCE-ACC has 6 dimensions (respect and dignity, services and supports, decision making, staff skills and training, social relationships, complaints) rated across 5-response options (“all of the time” to “none of the time”). It has been validated in a separate sample of aged care recipients in the community and residential care settings [22].

The PWI (Personal Well-being Index) is a measure of subjective well-being [23]. It has seven dimensions of life satisfaction: standard of living, health, achievement in life, relationships, safety, community connectedness and future security. These seven domains are the core set of items forming a composite score and each dimension can also be scored separately. Respondents were asked to rate their level of satisfaction in each domain on an 11-point end-defined scale anchored by “not at all satisfied at all” to “completely satisfied”. There are two additional PWI items (Global life satisfaction and Spirituality or Religion) which are not included in the PWI composite scores, but can be scored separately and these items were also used to assess the construct validity of the QOL-ACC.

Validity assessments for the QOL-ACC were guided by the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) checklists [24, 25]. Two important validity assessments are content and construct validity of a descriptive system. Content validity is the extent to which the content of the instrument appears to include full scope of the constructs relevant to the target population. Our group has published two papers describing a multi-stage rigorous process of qualitative interviews to identify content of the QOL-ACC and face validity assessments of draft items with older people using aged care services to inform content validity of the QOL-ACC [12, 15].

Construct validity is the extent to which an instrument validly measures the construct it purports to measure. We assessed a range of indicators of construct validity in relation to convergent and known-group validity. Convergent validity is a type of construct validity which examines whether an instrument correlates with other instruments to the degree that is expected. Assessing convergent validity is an iterative process: the more hypotheses tested, the stronger the evidence towards the instrument being valid. In the absence of a gold standard, the convergent validity of the QOL-ACC and its dimensions was assessed against two existing validated preference-based instruments (EQ-5D-5L, ASCOT) [11, 26], a measure of quality of aged care (QCE-ACC) and a measure of life satisfaction (PWI). A significant but low to medium (correlation range of > 0.30 to 0.70) correlation is indicative of good convergent validity, with related constructs expected to have a stronger correlation than unrelated constructs [27]. A series of 21 key hypotheses were developed to appraise convergent validity of the QOL-ACC against other constructs (Table 2). For example, we hypothesised that the QOL-ACC would be more closely associated with PWI (a psychosocial component of QOL) and ASCOT (social-care-related QOL) than EQ-5D-5L (health-related QOL). Convergent validity was perceived as adequate if more than 75% of the hypothesised relationships, in terms of the directions and strengths of correlations, were supported by the analysis results [27, 28]. Known-group validity is the extent to which an instrument discriminates between groups known to be different. Known-group validity was assessed by testing the QOL-ACC’s ability to discriminate the varying levels of aged care needs, aged care quality, self-reported health and quality of life.

Descriptive data were analysed using STATA Version 15.1, Stata Corp LLC, Texas, USA. The Winsteps software (Ver 5.0.0) was used to conduct Rasch analysis on the final descriptive QOL-ACC data and to produce interval level overall and dimensional level scorings [29]. Rasch analysis is an item response theory-based scaling psychometric technique that unlike traditional psychometric methods (e.g. Classical Test Theory) can provide insights into important psychometric properties both at the scale and item level such as adequacy of category functioning, monotonicity of category option use, item fit statistics, dimensionality, item bias etc. We used quantitative evidence from Rasch analysis to identify items possessing the best psychometric properties to develop the final descriptive system of the QOL-ACC from an initial item pool set [15]. Further, as the QOL-ACC descriptive system is a new instrument under construction, Rasch-based psychometric properties guided item selection provided important evidence to take it to the next stages of its development and validation. Further, Rasch analysis converts each categorical dimension onto a unique unidimensional latent scale to estimate interval level scoring for both the items and respondents on the same continuum scale. Rasch analysis has been widely applied in the development and validation of instruments to measure health-specific quality-of-life constructs [30‐33]. Given that the QOL-ACC has 6 dimensions, we used the Rasch Partial Credit Model with conditional maximum likelihood to estimate respondents’ parameters. Similarly, a Rasch model based methodological approach was applied by the EuroQoL group for the EQ-5D-5L initial validations [33].

The QOL-ACC Rasch scores in logits were rescaled from 0 (worse score) to 100 (best score) to facilitate ease of interpretation. We used Pearson’s Chi-squared tests for categorical variables and Kruskal Wallis Test to assess the difference in the EQ-5D-5L, EQ-VAS, ASCOT, PWI and QCE-ACC scores by the QOL-ACC dimension levels (test of monotonicity: to indicate that the scores on other instruments increased by response levels across the QOL-ACC dimensions). Person’s Chi-squared test was used to assess whether the distribution of the QOL-ACC response categories across its six dimensions was different between respondents using CHSP and HCP. For convergent validity (to assess the extent to which the QOL-ACC and other instruments measure related constructs), Spearman’s rank absolute correlation co-efficient (P values) were produced based on the instruments scores distribution assessments (Appendix S1). The size of correlation coefficients is interpreted as negligible (0.00 to 0.30), low (> 0.30 to 0.50), moderate (> 0.50 to 0.70) and high (> 0.70 to 0.90). These analyses were complemented by locally weighted scatterplot smoothing (LOWESS) techniques. The LOWESS is a form of non-parametric regression which plots a line of central tendency between two measures on a scatterplot (visually) to demonstrate relationship across all the possible score ranges without making assumption about the actual relationships [34]. For known-group validity, Kruskal–Wallis test was used to test differences between the multiple groups. Dunn’s test was carried out following Kruskal–Wallis test for multiple pairwise comparison between the groups [35]. A total of 21 hypotheses were tested to assess the construct validity of the QOL-ACC. To adjust for multiple testing, we used the Bonferroni technique to set the significance threshold at P ≤ 0.05/21 = 0.002.

In total, 1878 older people (≥ aged 65 years) living in the community were initially approached; however, 1479 did not meet the inclusion criteria (i.e. not currently receiving aged care services) and were screened out leaving 399 individuals who attempted the survey. Out of 399, 313 (78.4%) older people fully completed the survey. Of those who completed 54.6% were female and 76.4% were Australian born. Participants ranged in age from 65 to 91 years (mean 74.5 years, ± 5.9) and slightly more than half of the study sample (50.5%) were between 65 and 74 years old (Table 3). Approximately half of the study respondents (50.5%) lived with their spouse or partner and 41.9% lived alone at the time of the survey. Only 14.7% of survey respondents described their overall health as either “excellent” or “very good” whilst 32.9% described their overall QOL as either “excellent” or “very good” (Table 3). More participants were receiving a lower level of care package such as the CHSP (38.3%), or HCP level 1 (18.5%) and level 2 (21.7%) than the higher-level HCPs (level 3 and level 4). Almost half of the respondents were in the lower quintiles of SEIFA index for IRSEAS (46.4%) and more than one third were in the lower quintiles of SEIFA index for IEOA (35.1%) (Table 3). In comparison with national data describing the socio-demographic characteristics of older Australians using CHSP and HCP nationally in 2021, our study cohort also had more representation from females (study cohort vs national cohort; 54.6% vs 65.4%), aged < 85 years older (study cohort vs national cohort; 93.6% vs 63.8%) and using lower-level support services i.e. CHSP to HCP2 (study cohort vs national cohort; 78.5% vs 92.2%) (Table 3) [5].

The responses to the final 6-dimension QOL-ACC are summarised in Table 4. With the notable exception of the pain management dimension, the majority of respondents (> 70%) indicated good QOL across QOL-ACC dimensions. Three dimensions (mobility, emotional well-being and independence) had a significant difference in the distribution of responses between those using CHSP and HCP reflective of the increasing dependency levels associated with HCPs relative to CHSP which provides an entry level of care and support. This was not observed in the remaining three dimensions (pain management, social connections and activities) between CHSP and HCP groups.

The independence dimension had a stronger correlation with the ASCOT (ρ = 0.51, P < 0.001) and its domains of control (ρ = 0.57, P < 0.001, hypothesis 7) and occupation (ρ = 0.52, P < 0.001, hypothesis 8). It had similar correlations with the PWI global item of life satisfaction (ρ = 0.47, P < 0.001), PWI domains of achievement (ρ = 0.44, P < 0.001) and health (ρ = 0.41, P < 0.001) (Table 6).

The mobility dimension had the highest correlation with the EQ-5D-5L (ρ = 0.53, P < 0.001, hypothesis 9) and the ASCOT domain of control (ρ = 0.41, P < 0.001, hypothesis 10).

The social connections dimension demonstrated a strong correlation with the ASCOT domain of social (ρ = 0.50, P < 0.001, hypothesis 11), PWI domains of relationships (ρ = 0.55, P < 0.001, hypothesis 12) and PWI domain of community (ρ = 0.50, P < 0.001, hypothesis 13) (Table 6).

The emotional well-being dimension had a strong correlation with the PWI global item of life satisfaction (ρ = 0.61, P < 0.001), PWI (ρ = 0.58, P < 0.001), PWI domains of achievement (ρ = 0.53, P < 0.001, hypothesis 15), standard of living (ρ = 0.49, P < 0.001, hypothesis 16) and future security (ρ = 0.48, P < 0.001, hypothesis 17) (Table 6).

The dimension of pain management demonstrated a moderate and significant correlation with the EQ-5D-5L (ρ = 0.41, P < 0.001) and the PWI domain of health (Table 6). However, it did not show high correlation with the PWI domain of personal health (ρ = 0.35, P < 0.001, hypothesis 18) and EQ-5D VAS (ρ = 0.37, P < 0.001, hypothesis 19) as expected.

The activity dimension demonstrated a stronger correlation with the PWI (ρ = 0.54, P < 0.001) and the PWI domain of achievements (ρ = 0.53, P < 0.001, hypothesis 20). It demonstrated moderate correlation with the ASCOT domains for occupation (ρ = 0.45, P < 0.001, hypothesis 21) (Table 6).

Of the 21 priori hypotheses constructed (Table 2), 18 hypotheses (85.7%) were met suggesting that the QOL-ACC descriptive system and its dimensions have demonstrated adequate evidence of convergent validity.

The QOL-ACC discriminated across respondents with different self-reported QOL ratings (Fig. 1a), health ratings (Fig. 1b), aged care quality experience categories (Fig. 2a) and those receiving different levels of community-aged care services (Fig. 2b).

Self-reported QOL ratings were associated with poor QOL-ACC scores (Chi-squared = 121.4, df = 4. P < 0.001) with significant differences between groups. Self-reported health ratings were also associated with poorer QOL-ACC scores (Chi-squared = 69.1, df = 3, P < 0.001) with significant differences within and between all groups except between very good and good ratings (P = 0.10).

Higher aged care quality experience was associated with higher QOL-ACC scores (Chi-squared = 33.1, df = 2, P < 0.001). There was statistically significant difference in QOL-ACC scores between those who had high versus satisfactory (P < 0.001) and high versus unacceptable (P < 0.001) aged care quality experience but not between individuals who had unacceptable versus satisfactory (P = 0.46) aged care quality experience. Individuals who were receiving higher levels of care and support (HCP3&4) had lower QOL-ACC scores on average than those receiving lower levels of care and support (CHSP, HCP1&2) (Chi-squared = 30.3, df = 4, P < 0.001).