Introduction
The global population is ageing rapidly. The proportion of older adults (people aged 65 years and over) is expected to increase from 9.7% in 2019 to 16.4% by 2050 worldwide [
1]. Cognitive impairment and dementia are most prevalent in older age groups and commensurate with population ageing, the number of older adults diagnosed with dementia in Australia is predicted to more than double by 2058, [
2]. Given population ageing and increasing resource scarcity, health and social care provision for older adults into the future will necessitate careful and proactive policy planning [
3]. Economic evaluation of interventions and services for the care of older adults is necessary to ensure that funding is efficiently allocated to maximise older adults’ quality of life (QOL) [
4]. Monitoring the health and wellbeing of older adults accessing care is also essential to ensure high quality service provision, as reflected by the recommendations of the recent Australian Royal Commission into Aged Care Quality and Safety [
5]. Aged care in Australia is provided either through community home-care services, for people with low care needs or, for people requiring higher level of care and support, in residential care facilities. Whilst care needs vary, most residents (89%) have medium (26.8%) or high (62.6%) care needs in at least one domain of care [
6].
Preference-based measures of QOL such as the EuroQol-5 Dimension-5 Level (EQ-5D-5L) instrument can be used for quality assessment and as a key outcome measure for the economic evaluation of health and social care interventions. For both quality assessment and economic evaluation, it is important to ascertain the person’s own assessment of their QOL wherever possible, hence self-reported QOL is preferable over proxy report [
7,
8]. The EQ-5D-5L has been widely applied with older people in a variety of care settings including aged care [
8,
9]. However, currently little evidence or practical guidance is available to the research community, policy makers and practitioners to indicate the validity of self-reported QOL using the EQ-5D-5L in populations of older people with varying levels of cognitive impairment and dementia.
Given that the EQ-5D-5L was developed with general population samples and not specifically with older adults, it is important to explore how older adults interpret and understand the EQ-5D-5L, and what information is being considered when responding to ensure that meaningful self-report data is being elicited [
10]. This is particularly important for population of older people in aged care settings with a high prevalence of cognitive impairment and dementia.
Therefore, this study aimed to assess how older adults in residential aged care with different levels of cognitive impairment understand, interpret and respond to the EQ-5D-using a qualitative think-aloud approach. (background information on the think aloud approach is provided in Online Resource 1).
Method
Ethical approval for this research was provided by the Flinders University Human Research Ethics Committee (Project Number: 6732).
Participants
Participants were recruited through the South Australian Innovation Hub, a not-for-profit group of eight residential aged care providers. Target sample size guideline for cognitive interviewing studies indicate that 20 participants is sufficient to ensure data saturation [
11]. Consenting older adults were stratified into one of two cognition sub-groups indicating the presence or absence of cognitive impairment with the aim of recruiting 20 participants in each sub-group for a total sample of 40 residents. Residents were eligible to participate if they could understand and communicate in the English language and were aged 65 or older. Residents with severe cognitive impairment or dementia who were unable to provide informed assent or consent to participate in the research were excluded [
12].
Materials
Interview Guide: A detailed interview guide (Online Resource 2) was developed based on published guidelines for cognitive interviewing [
13].
Demographic questionnaire: Demographic questions included age, education level, country of birth and length of time residing at their residential facility.
Standardised Mini-Mental State Examination (SMMSE): The SMMSE is a standardised version of the Mini-Mental State Examination (MMSE), an internationally recognised screening tool for the assessment of cognitive impairment in older adults [
14]. It consists of 12 items and scores range from 0 to 30. The UK’s National Institute for Health and Care Excellence (NICE) cut-off scores for cognition sub-group were utilised where a score of 10–20 indicates moderate cognitive impairment, 21–26 indicates mild cognitive impairment, and 27–30 indicates no cognitive impairment [
15].
EuroQol-5 dimensions-5 level (EQ-5D-5L): The EQ-5D-5L was developed in 2009 from the original three-response level version (EQ-5D-3L) to improve the sensitivity and reliability of the instrument and to increase the number of possible health states [
16]. The instrument consists of five dimensions: mobility, personal care, usual activities, pain/discomfort and anxiety/depression. Participants are asked to rate their health as of ‘today’, with five response options (no problems, slight problems, moderate problems, severe problems, extreme problems/unable). The EuroQoL visual analogue scale (EQ VAS) component of the EQ-5D-5L consists of a vertical scale numbered from 0 to 100, with 0 representing ‘the worst health you can imagine’ and 100 representing ‘the best health you can imagine’. Respondents are asked to choose a point on the scale that best represents their own health today.
Procedure
Data were collected over a 6-month period from June to December 2021. Residential aged care facilities that expressed interest in the research were attended to by two of the authors who gave a short presentation about the research at a residents’ meeting and distributed participant information sheets and a project flier. Where the researchers were not able to attend a residents’ meeting (due to timing or COVID19 restrictions) fliers and participant information sheets were distributed by facility staff. Staff were informed that the research team sought participants with varying levels of cognitive decline and were explicitly asked to distribute information to all residents regardless of perceived capacity to participate or consent.
Contact details of interested residents were forwarded to the research team by the residential care facility’s manager following initial verbal consent. Days and times for interviews were co-ordinated with facility staff. Where residents had a power-of-attorney in place, assent was sought from the resident and consent from the person with power-of-attorney [
17]. Participants were assessed and allocated into one of two cognition sub-groups, based on their score on the SMMSE.
Participants chose the interview location (e.g., their bedroom, a common space) according to where they felt most comfortable. Interviews were completed by the first author and a research assistant, trained in cognitive interviewing and administration of the SMMSE with older adults. A family member or friend could be present for the interview duration if the participant preferred.
Once formal, written consent had been received, residents were asked to provide some basic socio-demographic information. The interviewer then completed the SMMSE with the participant. Participants were handed a “practice survey” with three questions with Likert-type scale response options and asked to think-aloud while responding. Following this, participants were asked to complete the paper-based version of the EQ-5D-5L and the EQ-VAS and were encouraged to think-aloud while doing so (concurrent think aloud). If participants were silent for a period of over three seconds, they were prompted to keep thinking aloud (either with scripted or unscripted prompts). If a participant completed a question on the EQ-5D-5L without thinking aloud, they were asked to stop and go back over their responses while thinking aloud (retrospective think aloud). Scripted verbal probes such as “can you tell me what you were thinking about when you were answering that question” and “why did you choose that particular response” were utilised to engage the participant in retrospective think-aloud. The think-aloud portion of the interviews were audio recorded and transcribed verbatim.
Analysis
Using NVivo qualitative analysis software (release 1.3) sections of text from transcripts were coded to each of the EQ-5D-5L dimensions. The text segments were then exported into an excel spreadsheet and coded to a framework developed from the Tourangeau four-stage response model. The Tourangeau response model is based on a cognitive theory of survey response and identifies four stages at which response issues can occur, comprehension, recall, judgement, and response mapping [
18] (Table
1).
Table 1
Response model framework
Reflects the encoding process: participants had problems understanding, or a misunderstanding of, a word, phrase or response option OR participants understood the question in a different way than the measure intends the question to be understood (often closely linked to judgement) | Refers to the retrieval process—including using the wrong time frame for retrieval and problems remembering relevant information | Problems with judging the information retrieved, including retrieving irrelevant information or assessing the information inadequately—leading to under or over-reporting of health state (often closely linked to comprehension) | Problems with the response categories including not liking the options or inappropriately applying them e.g., choosing two or not wanting to choose any OR stated answer (verbal protocol data) misaligned with chosen answer (survey response data) |
Text segments were independently coded by three authors (KL, MC and LE), with each author keeping notes on their interpretation of the response issue. Coders were blinded to participants’ cognition subgroup and all other demographic details. Inter-coder reliability was estimated using Krippendorff’s alpha (α) [
19] and disagreement resolved through a group discussion comprising all three coders. Following coding for response issues, a chi square test was performed to test for association between cognition and response stage issue and frequency, both overall and by dimension. Summary statistics were calculated for socio-demographic data and Pearson’s chi-squared test with Yates’ continuity correction and Kruskal–Wallis rank sum test used to test for association between cognitive impairment subgroup and socio-demographic factors. Pearson’s chi-squared test and Mann–Whitney U were used to test for differences in frequency of response issues by socio-demographic factors (gender, age and level of education). EQ-5D-5L scores were converted to utilities using the Australian pilot value set [
20].
Discussion
The findings from this study indicate that contextual factors, specific to older adults, may affect the reliable and consistent comprehension of the EQ-5D-5L. A high frequency of response issues were evident across all dimensions and across both cognition groups. Previous think-aloud research applying the Tourangeau model to assess responses to the EQ-5D-3L in the adult general population found a far smaller proportion of response issues, with just 15 response issues across 34 participants [
21], indicating that this general population sample largely correctly interpreted the EQ-5D-5L descriptive system as expected by the instrument developers. However, a study that used the Tourangeau model to analyse think-aloud responses to the EQ-5D-3L in a population of community dwelling older adults, found a response issue to participant ratio comparable to the current study, 33 issues across 10 participants [
22]. This suggests that interpretation of the EQ-5D for both the three and five levels versions may be problematic for older adults.
It was evident that aged care residents may interpret and respond to the EQ-5D-5L in a way that general population samples may not. The ubiquitous use of treatments or aids in this population and confusion about how to incorporate these factors when formulating a response was responsible for several response issues across three dimensions. In the current study, the examples provided in the ‘usual activities‘ dimension were largely unrelated to life in a residential aged care facility, leading to confusion about what ‘usual activities‘ to consider. The same response issue was also reported in van Leeuwen’s [
22] sample of community dwelling older adults.
Participants reported having no problems or slight problems with some dimensions, particularly personal care, despite receiving daily assistance with personal care tasks or, in the case of the mobility dimension, almost exclusively using mobility aids. It is common for mobility-impaired individuals to reframe mobility items to better reflect their situation, including the use of aids [
23,
24]. This finding also aligns with early research on using cognitive interviewing to develop surveys for older adults [
25]. The authors found that older adults reported an ability to complete tasks despite not having attempted them for weeks or months, and despite reportedly struggling with the tasks previously.
The reluctance of participants to incorporate two distinct concepts into responses to the anxiety/ depression dimension supports the findings of previous research in a similar population [
26]. Whilst this question structure has been found to be particularly problematic for people with cognitive impairment, in our study, issues with the combining of two concepts was also observed in the NCI sub-group. Difficulties or discomfort with these questions may also be exacerbated in an aged care population where there may be generational stigma regarding acknowledging mental health conditions, particularly depression [
25‐
28].
Participants attempted to formulate meaningful responses where possible as demonstrated by the adoption of strategies to overcome the problem of fluctuations and combined concepts. These strategies included assessing their health over a longer time-period (‘averaging’), choosing the ‘middle option’, or crossing out part of the question e.g., indicating that their response only related to discomfort or anxiety, not both. The problem of fluctuating health states, particularly on the pain/discomfort dimension has been observed in a diverse range of populations, including patients with cancer [
29], patients with advanced HIV [
30], and patients nearing death [
31]. The same pattern was also identified in a study examining the feasibility of using the EQ-5D-3L with nursing home residents [
32]. Whilst some commentators have suggested that recall periods of a week or more may mitigate this [
33,
34], it is acknowledged that longer recall periods can be problematic, especially for people with cognitive impairment [
35]. Furthermore, respondents with widely fluctuating health and/or QOL may equally struggle with longer time frames [
36].
Older adults, particularly those with functional disability or long-term health conditions are likely to show what is termed ‘Beta’ response shift or ‘scale recalibration’, where a person adapts to declining health over time by shifting their reference point to reflect their lowered expectations of health and/or QOL [
21,
37,
38]. Questions that require interpretations of especially subjective concepts such as “health” and “best” are particularly at risk of obfuscation from Beta response shift effects [
39]. This was particularly evident in our study by participants’ responses to the VAS, which revealed that some participants compared themselves to others of the same age or to what they might expect their health to be given their age. Unsurprisingly, individuals with lowered expectations under-report health-related problems [
40], as demonstrated by the tendency of participants in this study to adjust their responses to reflect what they deemed to be a realistic expectation of health for their age or circumstance.
Although the EQ-5D has been psychometrically tested in older people, including dementia populations, and found to be feasible [
41,
42] and reliable [
43,
44], our findings raise questions as to the content validity of the EQ-5D-5L with an older aged care resident population. To ensure the EQ-5D-5L is meaningful for this cohort, some alterations to the phrasing may be necessary. Modifications, including providing more relevant examples of ‘usual activities‘, explicit clarity around what constitutes ‘best health’, wording that allows for, and explains further how residents should factor fluctuating health and specific instructions on how to factor mobility aids and other treatments, may increase the utility of the EQ-5D-5L with an aged care and older person specific population. Consideration should also be given to the appropriateness of the ‘today’ timeframe for this cohort or to rephrasing questions to place more emphasis on instructions for the recall period.
Regardless of the measure adopted, assessing QOL in older people in residential care settings is challenging due to the relatively high prevalence of cognitive impairment and dementia in this population. To drive inclusivity in self-reporting of QOL wherever possible, some instrument developers have developed easy-read and pictorial versions that have been tested with older adult service care recipients and found to have increased comprehensibility [
44,
45]. It has also been suggested that support from an interviewer may be necessary to ensure meaningful responses to QOL measures in older people with cognitive impairment [
46]. Towers and colleagues [
47] have developed a mixed methods version of the Adult Social Care Outcomes Toolkit (ASCOT) and tested its acceptability and feasibility to inform practice in UK care homes. It was found that the mixed methods approach directly facilitated inclusion of many cognitively impaired residents who otherwise would have been excluded from the research or had their own views on their social-care related QOL represented only by a proxy.
This study has several limitations. Whilst the aim was to recruit even numbers in each sub-group, COVID19 pandemic related recruitment barriers at the time of data collection and aged care facilities’ tendency to recommend participants with no cognitive impairment contributed to uneven sub-group numbers. Interviewers were vigilant to reduce or change prompts when participants became fatigued or frustrated, nevertheless some participants clearly struggled with the think-aloud process and this may have caused them to lose focus and attend less carefully to their responses. The administration of the SMMSE immediately prior to the think-aloud interview may have further exacerbated this issue by adding to the cognitive load and thereby fatigue. There is also some evidence that the added cognitive burden of the think-aloud protocol can contribute to this effect [
34,
48]. Data was not collected on the use of mobility aids, personal care assistance and treatments or medications for this sample. Controlling for these factors may have provided more clarity around the patterns of response issue across cognition sub-groups. Finally, whilst the completion rate for this study was high, this cannot be relied upon to reflect the self-report feasibility of this measure within this population due to the involvement of interviewers in the completion process. Despite these limitations this study provides valuable insights and to our knowledge represents the first in-depth qualitative study in Australia and internationally to apply a think-aloud protocol to improve our understanding of how older residents with different levels of cognitive impairment interpret and respond to the EQ-5D-5L descriptive system. The findings have potential wider relevance and application beyond the EQ-5D-5L for the development and application of other QOL tools administered with populations of older people.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.