How to measure fluctuating impairments in people with MS: development of an ambulatory assessment version of the EQ-5D-5L in an exploratory study

People with the neurological disease multiple sclerosis (MS) have different symptoms and impairments that can reduce their quality of life. These impairments are often not constantly present but change within a day or from one day to another. Measuring these changes might help clinicians treat people with MS better, and it might also be useful in studies, for example those investigating the effectiveness of MS medications. Therefore, we developed a way to measure the fluctuations in these patients’ everyday lives, using mobile phones. First, we discussed with a group of patients how the instrument should look. Second, we developed a first version of the instrument, which was tested by patients and then refined. In the new instrument, patients answer questions about their health three times a day over nine days on their mobile phones. The questions were taken from the EQ-5D-5L questionnaire, which is a well-established instrument measuring health-related quality of life. The questions covered mobility, self-care, usual activities, pain/discomfort and anxiety/depression, as well as a 0–100 scale where patients rate their subjective health. Our study participants found the new instrument feasible and useful.

Multiple sclerosis (MS) is a chronic, currently uncurable, inflammatory disease of the central nervous system characterised by clinically significant fluctuations in symptoms and functioning. MS frequently affects vision, mobility, cognition, bladder control and other functions [1]. The most frequent MS phenotype is relapsing–remitting, followed by secondary and primary progressive disease course [2]. In relapsing–remitting MS, symptoms worsen during the clinical episodes (relapses) and last for a period of weeks to months [3]. However, symptoms also fluctuate at shorter intervals within a single day [4, 5] and from one day to the next [6]. For example, in fatigue, a frequent MS symptom, 35.5% of variability could be attributed to moment-to-moment fluctuations, 8.2% to day-to-day changes and 56.6% to individual differences [7].

The vast majority of patient-reported outcomes measures (PROMs) do not assess fluctuation but ask for the extent of impairment within a specific period like “the last seven days” or “today”. To choose a response option, respondents must summarise their experience within the reference period to form some kind of average or typical value. For example, a person may rate pain that is mild in the morning but gets more severe over the day as “moderate” as this represents the average intensity; another person in the same situation may choose “severe” as this represents the maximum.

However, information on short-term fluctuation is crucial for the understanding of impairments in MS. In addition to considerable diurnal variability within persons, temporal patterns differ between persons. Furthermore, fluctuation data within a single day can help uncover the interrelation between different impairments, as associations between symptoms were found predominantly within a day with little carry-over effect from one day to the next [8]. In clinical practice, information on these fluctuations is highly relevant for rehabilitation, medication adjustment and life planning. For example, spasticity substantially fluctuates depending on time of the day, activity level, temperature, but also psychological factors. A sensitive assessment of impairments related to this symptom can help to adjust dosing of antispastic agents which have also substantial side effects.

Fluctuations can be captured by a method called ambulatory assessment (AA) where respondents provide information on mobile devices multiple times per day over several days [9, 10]. In addition to capturing within-person dynamics, AA reduces the need for respondents to average their health problems over longer periods of time, reduces recall bias, and can be assessed in everyday life, thereby providing high external validity [11]. As a drawback, AA increases response burden. Moreover, when repeatedly answering the same questions and thereby gaining experience with the surveyed construct, respondents may adjust their responses to the rating scale. Their answers will thus not be fully comparable anymore, a phenomenon known as recalibration response shift [12]. AA is increasingly being used in PROMs [13, 14] where it has been found to be feasible and valid [13, 15].

One of the most widely used PROMs is the EQ-5D-5L, a generic instrument of health status [16, 17]. Its first part, the EQ-5D descriptive system, includes five items (one per dimension) assessing mobility, self-care, usual activities, pain/discomfort and anxiety/depression, each with five response options representing different levels of severity [18]. The second part, EQ VAS, measures self-rated health with a horizontal visual analogue scale (VAS), the anchors labelled “The best health you can imagine” (100) and “The worst health you can imagine” (0). Both parts refer to health “today” without differentiating by time of day. The EQ-5D-5L has replaced the previous version EQ-5D-3L that had only three response options, hoping to decrease the considerable ceiling effects. These are still found for the 5L version in the general population, but less so in people with increased morbidity [19, 20].

Psychometric properties of the EQ-5D-5L have been investigated in people with MS (PwMS), finding good test–retest reliability and convergent validity, but limited content validity and discriminative ability [21]. In other chronic diseases, it has also been found that the EQ-5D misses some relevant aspects of health-related quality of life; for example, fatigue [22]. We nonetheless decided to use the EQ-5D-5L in this study because of its combination of widespread use and brevity, the latter being crucial for feasibility in an AA.

The EQ-5D-5L also captures dimensions of health that are highly relevant in MS: Persons with relapsing-remitting MS reported “some” or “extreme” problems in mobility (68.9%), self-care (38.2%), usual activities (77.9%), pain/discomfort (63.9%) and anxiety/depression (58.5%) (using the former, three-level EQ-5D version) [23]. When currently in a relapse, the number of PwMS who experience problems was found to be even higher with 55 to 94% by dimension [24].

To our knowledge only two other studies have measured within-day fluctuations with adapted versions of the EQ-5D-5L, both in non-MS patient groups. In Kerr et al. 2016, persons with Parkinson’s disease completed the EQ-5D-5L both for “on-time” (where medication is working well) and “off-time” (where it does not), reporting also the duration of both states [25]. Considerable within-day fluctuations were found. With MS, however, it is not as clear cut as good/bad, calling for a different approach to capturing health dynamics. In the second study, a momentary version of the EQ VAS with 10 assessments per day has successfully been tested in three patient groups and healthy people [26]. They found that average AA ratings correlated with, but also significantly differed from, the standard EQ VAS as assessed after the AA period and may therefore provide important additional information. The EQ-5D descriptive system was not included in that study.

To enable the measurement of health fluctuations in PwMS, we therefore aimed to develop an AA version of the complete EQ-5D-5L (called EQ-5D-AA, for ambulatory assessment of the EQ-5D) for use in this patient group. As an AA implies higher response burden than a one-time questionnaire, we also aimed to assess the EQ-5D-AA’s feasibility and its face validity from the patient perspective as compared to the standard EQ-5D-5L. This study focuses only on the EQ-5D as a measure of health in research and clinical practice, not on its role as a tool for economic evaluation for which it is frequently used.

This was a qualitative descriptive study [27] involving focus groups and one-on-one, in-person or telephone interviews with additional exploratory quantitative analyses. It included two phases: (a) the use of patient focus groups, resulting in a first version of the EQ-5D-AA and (b) completion of the EQ-5D-AA by subsequent waves of PwMS, each followed by cognitive debriefing and refinement of the instrument (Fig. 1).

Participants were recruited at the MS outpatient clinic at the University Medical Center Hamburg-Eppendorf and through MS self-help groups (newsletter and posting). Inclusion criteria were: age ≥ 18 years, confirmed MS diagnosis, fluent in German, and sufficient cognitive and physical ability. The study sample should be heterogeneous with regard to disease severity, cognitive impairment and age, and should include both men and women. Participants received financial reimbursement.

In this study, we developed an AA version of the EQ-5D-5L for use in people with MS. After two focus groups and four waves of iterative testing and refining, the EQ-5D-AA was finalised. The AA was extended from seven to nine days due to participants reporting recalibration response shift within the first two days. Participants judged the AA as not too burdensome to complete for this duration and also considered it feasible. Most of them found it more informative than the standard one-time assessment of EQ-5D-5L.

There was high agreement between one-time assessment and average AA index scores in spite of intra-individual variability in AA responses. This shows that times in better and worse health evened out over the 7–10-day AA period. Descriptively, similar values were also found on single-item basis, but ratings were slightly more negative in the standard EQ-5D-5L than in the AA. This may indicate that the AA does not provide much added information and therefore does not warrant the additional effort. However, this finding seems in contrast to most participants clearly favouring AA over one-time assessment because they believed it captured important information about their health. An explanation may be that they regard the variability and pattern of health fluctuations as relevant over and above the average level of impairment. Indeed, all six EQ-5D-AA items showed variation in most participants, and the patterns were also quite different: some participants had highly stable values, while others showed considerable fluctuation. Detecting these patterns may be of additional value in understanding a person’s health status, comparable to findings that emotion variability has added value next to average emotion intensity when predicting well-being [33, 34]. However, these quantitative findings were exploratory only and need confirmation in a larger sample.

The EQ-5D-AA items ask retrospectively to the previous assessment; thereby, covering the complete assessment period (except for the night where two items were not applicable, usual activities and self-care). This approach is called proximal intensive assessment or complete coverage [15, 31]. In contrast, a sampling strategy would assess a—usually random—sample of moments only, which are taken to be representative for all moments within the sampling period. Such a strictly momentary approach would be applicable to the EQ-5D-5L dimensions of pain/discomfort and anxiety/depression: Both are states of mind that have some intensity at any given (waking) moment, including a possible intensity of zero. However, for some dimensions a momentary approach is not appropriate as they do not apply to most moments. This is especially true for the self-care dimension (because most of the day, no washing or dressing is needed), and to a lesser extent probably also for usual activities and mobility. However, the coverage approach used here also comes with disadvantages: recall bias is possible, and respondents still have to build an average value for the respective—albeit short—time period.

For the exploratory analyses, we calculated an index score for the seven-day AA period by first determining the score for each day, using the respective highest score of each item, and then averaging over days. It was not possible to determine a score for each time point because only three out of five items were assessed three times a day. However, with this calculation, scores will be the same regardless of whether an impairment was present during the complete day or only parts of it. As an alternative, the median or mode score of all item values of the week could be used for index score calculation. In addition, one could determine seven-day fluctuation scores, using variability or instability parameters [33‐35]. Which of all these scores carry most information on patient-relevant aspects of health and/or are predictive of future health outcomes, needs to be evaluated with larger samples. Score calculation is further complicated by the missing data, which are very common in AA due to the high number of assessments and have also been found in the majority of our participants. Imputing missing values using statistical techniques, such as multiple imputation is recommended [36].

We would not recommend the EQ-5D-AA for use as a utility measure in health-economic evaluations for several reasons. One, existing valuation sets for the EQ-5D-5L cannot be used for an AA version; instead, specific preference elicitation studies would be needed which require large representative samples. Two, AA comes with higher respondent burden and also logistic effort than the standard EQ-5D-5L assessment. As health-economic evaluation often draws on the data from clinical trials, it is probably unrealistic that the additional effort of an AA will be taken in these studies.

Larger, subsequent studies also need to evaluate psychometric properties of the EQ-5D-AA as compared to the EQ-5D-5L and confirm its feasibility. They should use a stand-alone AA application that is compatible with both Android and iOs so that most participants can use their own mobile phone, probably reducing missing values. Finally, it should be evaluated whether and under which circumstances (e.g., one’s job and family situation) people would also be willing to complete the AA for a longer period of time than tested here—for example for monitoring purposes in clinical practice. This is of particular importance as our sample was small and probably also subject to selection bias in that only people who were willing to complete an AA took part.

If the EQ-5D-AA will prove valid and reliable, it can be used in future research, but also by individual PwMS self-tracking their health; some of our participants mentioned this to be an interesting option. Such data might also support patient-clinician communication on symptom dynamics and management, for example for activity planning and symptomatic treatment applications: Whether such use in clinical care is feasible and useful would need to be addressed in additional research, also investigating feasibility and usefulness from the health providers’ perspective.

A strength of this study was its iterative approach to AA development with subsequent waves of real-life testing, debriefing and adaptation. This approach may also be suited for AA development in other health conditions. Furthermore, our multidisciplinary research team included experts on PROMs and electronic PRO assessment, members of the EuroQol group, and a clinician specialised in MS care, each contributing their unique perspective on the AA development.

While our study sample was heterogeneous with regard to gender, age, disease duration, and both cognitive and subjective health impairment, it should be considered a limitation that most participants were from Hamburg, Germany, and people with lower education were underrepresented. It will therefore be important to include PwMS from this subgroup as well as people from other regions, including rural areas, in subsequent validation studies. Owing to the small sample, which also represents a limitation, the exploratory quantitative analyses can only give a hint on possible associations and patterns in EQ-5D-AA data that may warrant investigation in follow-up studies.

This study suggests that an one-week AA of the EQ-5D-5L can capture within-day and day-to-day fluctuations in subjective health and was feasible in people with MS. Patients stated that the EQ-5D-AA can provide important information on their health beyond what is captured by the EQ-5D-5L standard version.