Surgical prioritization based on decision model outcomes is not sensitive to differences between the health-related quality of life values estimates of physicians and citizens

COVID-19 has provided unprecedented shocks to healthcare systems worldwide with consequences that will continue long after the pandemic has subsided. One of the consequences for surgical disciplines was a growing surgical backlog due to the allocation of surgical capacity to intensive care units [1‐3]. Future crises may cause similar or even greater difficulties. Therefore, thought needs to be given to the allocation of scarce surgical resources.

The allocation of surgical resources (e.g. operating room capacity) is still largely based on historical patterns or decision-making driven by intuition [4]. In recent years, increasing attention has been given to systematic and evidence-based approaches such as decision analysis to inform resource allocation [5]. In decision analyses, relevant trade-offs are captured in a decision model. These models estimate the outcome of various strategies in the presence of uncertainty.

Health-related quality of life (HRQoL) values are frequently used as input parameters for decision models. HRQoL refers to the health aspects of QoL [6]. Here, the HRQoL gain is seen as a measure of the benefit of a treatment. Nevertheless, there is a continuing debate about methodological design choices used in HRQoL assessment studies [7‐10]. Concerns include the methods used in evaluation and whose values should be used. Leading national and international guidelines for cost-effectiveness analysis recommend methods where patients report their own health state, using (EuroQol-5 dimension) EQ-5D, and then normative values, that are based on the views of the general public, are applied [11‐13].

Amidst the COVID-19 crisis, we developed a decision model to support the prioritization of surgical care in times of scarcity. The impact of surgical delays on health for elective and semi-elective surgeries was estimated [14, 15]. Notably, in our model, we deviated from the international recommendations for HRQoL estimates, and instead obtained estimates through a physician panel using the visual analogue scale (VAS) method. This was for two reasons. First, the rapidly evolving COVID-19 crisis demanded the rapid development of decision models. Consequently, HRQoL valuations were needed immediately, which was only feasible in the existing situation by using physicians. Second, physicians can be seen as a desirable panel for the determination of HRQoL values as they are skilled and, collectively offer, a broad view and depth of expertise that should enable them to make comparative judgments across a range of diseases.

One could offer the counter-argument that the general public and physicians value health states differently. This could impact decision model outcomes and, subsequently, surgical prioritizations. Therefore, the aims of the current study are two-fold. First, HRQoL values estimated by physicians will be compared with the HRQoL estimates from citizens. Second, the impact of the choice of respondent panel on the decision model outcomes and subsequent ranking of the surgical procedures will be assessed.

In our previous studies, physicians had estimated HRQoL values for 43 semi-elective surgeries frequently performed in academic hospitals [14, 16]. For each surgery, the health loss, expressed as disability-adjusted life years (DALY) per month of surgical delay, was calculated. This outcome was subsequently used to compose a ranking of these surgeries. We argue that the surgery with the greatest health loss should be prioritized. A detailed description of this preliminary work can be found in an earlier publication[14]. For the current study, a sample consisting of ten surgical interventions deliberately selected from the 43 surgeries was used. Three inclusion criteria were applied: (1) the sample should be representative of different surgical specialties, (2) oncological and non-oncological diseases should be included, and (3) they should be spread across the earlier ranking of 43 surgeries based on health loss per month of surgical delay (including the extreme ends). An overview of the surgeries included is provided in Table 1.

For each surgery, we established a description of the pre- and post-operative health states, the so-called “vignettes”, for use with our citizen panel. These were derived from vignettes used in earlier studies [14, 16]. In live panel sessions, experienced physicians described health states of typical patients, while others asked clarifying questions [14]. These vignettes were documented and reused in the follow-up study after a collective review with our research group [16]. In the current study, these vignettes were again employed. There is no standard regarding the content of such vignettes [17], and so we adhered to the following principles: vignettes should be as brief as possible, written in simple language, and describe the typical patient. They included descriptions of symptoms and effect on activities associated with daily living. These descriptions were similar to those we had used in our previous studies with physicians but without any medical jargon. An overview of the vignettes and VAS is shown in Online Resource 1.

Citizens were recruited through a panel company. To compose a balanced panel that was representative of the Dutch population, the participants were sampled on age, sex, and socioeconomic status. Two independent citizen panels were established (CP1 and CP2) to (1) validate the results obtained using the physician panel, and (2) to compare the results of each citizen panel to each other to establish their reliability. We aimed for 15–20 citizens per panel as this is within the typical range of other Delphi studies [18, 19]. To obtain HRQoL values from physicians, we used data from our previously published studies [16]. This physician panel was purposefully chosen because they had previously valued the same health states using the same method, which will be discussed below.

We compared the citizens’ HRQoL estimates of health states with those estimated by physicians in a previously published study. It was found that physicians valued health states systematically higher than citizens. Furthermore, the two citizens’ panels (CP1 and CP2) showed a very high degree of consensus, giving us confidence in the reliability of our results. Moreover, the difference in HRQoL values did not substantially alter the decision model outcomes, and the ranking of the surgical interventions remained consistent, with only adjacent entries in the ranking list changing position.

Previous studies that have evaluated the differences between physicians’ and citizens’ HRQoL valuations have produced inconsistent results. In line with our results, some previous studies have found that physicians in general assign higher HRQoL values to health states than do citizens [28‐31]. The impact of demographic or contextual differences on HRQoL valuations has been extensively studied [32; 33]. On the whole, these studies found that the impact of these aspects was minor compared to differences that could arise from methodological choices [32].

Since previous studies have highlighted the impact of the method used for establishing HRQoL valuations, it is imperative that we critically appraise our approach. Two alternatives spring to mind: (1) obtain values from patients, rather than physicians and citizens as used in the present study; or (2) using values based on ‘utility measures’ such as the EQ-5D of patients actually in the health states being considered. The consequences of using such alternative approaches is discussed below.

First, one could expect that patient panels whose members have not experienced the specific health state being considered will give the same results as citizens since both of these panels will provide non-experience-based values [34]. This is in contrast with patients evaluating their own health state, i.e. a state that they are currently in or have previously experienced. There is considerable literature describing adaptation phenomena in that patients attach substantially higher values to a health state they are experiencing than do people who are not in the compromised state [34‐36]. At the same time, the values given to health states other than their own seem not to be affected by coping, and are most likely similar to values from other respondent panels who are not experiencing the specific health state (e.g. citizens) [37]. As such, any differences between the HRQoL values of patient and of citizen panels are expected to be small.

Another alternative approach would be to obtain HRQoL values from empirical research involving patients experiencing the relevant health states (e.g. EQ-5D questionnaires). Although this would be in line with the current international recommendations for cost-effectiveness analysis [11‐13], this approach would have some notable shortfalls in regard of our decision model. Foremost, it is questionable whether one could collect values for all the health states included in the model in one coherent study. The current mode already incorporates 84 surgical interventions [15], and will be continuously updated and extended, so collecting sufficient EQ-5D data would require a tremendous effort. Another option would be to collect data from observational or routine datasets, or to use existing estimates identified in the literature. However, this approach would raise several concerns. First, it is unlikely that health states evaluated in other studies would be representative of our population of interest (i.e. patients in the Netherlands who are awaiting the surgical interventions that are simulated in our model). Given that most published HRQoL values represent patients in randomized clinical trials that have strict inclusion and exclusion criteria, these values are unlikely to be fully representative of the wider population. Such studies might also be compromised by imprecise evidence, poor response rates, and incomplete follow-up data. For the same reasons, the Global Burden of Disease study also avoided using patient-reported HRQoL data [23]. Second, using data from other studies has the risk that one combines different utility measures. This is a concern since it has been demonstrated that different utility measures give quite different results [38‐41], and therefore using multiple measures has been strongly discouraged [42].

Although alternative utility measures methods have been proposed, such as time trade-off and standard gamble, which might, at least theoretically, be preferable for HRQoL evaluation, we deliberately opted for evaluating HRQoLs using the calibrated visual analogue scale (VAS) with reference points as introduced in the Global Burden of Disease study. Given the necessarily practical approach of the current study, we considered this to be a reliable and easy-to-interpret measure that allowed us to contemporaneously estimate HRQoL values for multiple diseases. An additional advantage is that we could then use one single measurement instrument and include consistent overall health states, thereby following existing recommendations [42].

Since we only evaluated ten surgical interventions in the current study, this could impact the generalizability of our results. This small number was deliberately chosen for practical reasons. We intended to provide the participating citizens with a short, manageable task to maximize engagement. We foresaw that an online task which took longer than 30 min would be off-putting for potential participants [43, 44], and potentially reduce the response rate. We therefore chose ten surgical interventions with care to cover the spread of likely impacts of surgical delays, believing that evaluating the associated 20 health states in the Delphi study was realistic within this time limit, and would increase the representativeness of the sample.

The fact that there are no standardized descriptions for the vignettes evaluated is another potential concern since the HRQoL values given by the citizens will be highly dependent on these descriptions. Although our vignettes are not all identical in structure, we addressed this limitation by applying general principles [17] (e.g. they should be brief, avoid medical jargon, reflect typical patient experiences) when formulating them. Further, the vignettes provided to the citizen panels were based on those used earlier with the physician panel albeit with any medical jargon omitted or rephrased.

In general, there seems to be a consensus that the source of HRQoL values should match the perspective and the level of decision-making [45‐48]. As such, in the case of individual, patient-level decision-making, it is preferable to use patients’ health states. For macro-level decision-making, where HRQoL values are used to compare population health, panels of medical experts can be used. Indeed, there are many studies that have used valuations provided by medical experts for the same reasons as we did [10]. We would argue, when setting surgical priorities, that using values provided by physicians who have a generic perspective is the most appropriate approach. Furthermore, national guidelines [13] recommend using panels of medical experts when HRQoL values representative of the population of interest are lacking. Here, the Delphi method is broadly seen as an acceptable approach to reach a consensus within such panels.

Our findings affirm our model’s reliability, consistent with previous studies [14‐16]. In these studies, we assessed the reliability of each physician panel by comparing the HRQoL values obtained from one physician panel to those obtained from another physician panel. Minor, yet consistent, HRQoL variations among physician panels did not affect surgical rankings. Our approach, using physicians to represent societal values, handles complex health state descriptions effectively. Extending our model to include more surgeries using this methodology seems appropriate in crises.

We would like to stress that other factors which are currently not represented as input parameters in our model may well impact decision model outcomes. Therefore, instead of re-evaluating our HRQoL valuation method, it would be more worthwhile to instead consider incorporating patient characteristics such as comorbidities and age in our model since these characteristics are known to influence pre- and post-operative surgical outcomes. As such, such additions would contribute to a much more refined model outcome. Further, we acknowledge that prioritization is, in practice, a complex decision-making process that involves normative criteria, contextual factors, capacity constraints, and stakeholder interests. Ultimately, the usability of our model’s outcomes in daily practice will be greatly influenced by all of these aspects.

Minimal differences in HRQoL valuations between citizens and physicians had little impact on decision model outcomes in terms of prioritizing surgeries. Regarding our decision model, values from sources beyond the general public can be used, facilitating quicker model extensions which enhance the usability of our model. It is crucial to emphasize that our method should not replace patient-reported outcome measures in routine care.