Skip to main content
Log in

One Method, Many Methodological Choices: A Structured Review of Discrete-Choice Experiments for Health State Valuation

  • Systematic Review
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Background

Discrete-choice experiments (DCEs) are used in the development of preference-based measure (PBM) value sets. There is considerable variation in the methodological approaches used to elicit preferences.

Objective

Our objective was to carry out a structured review of DCE methods used for health state valuation.

Methods

PubMed was searched until 31 May 2018 for published literature using DCEs for health state valuation. Search terms to describe DCEs, the process of valuation and preference-based instruments were developed. English language papers with any study population were included if they used DCEs to develop or directly inform the production of value sets for generic or condition-specific PBMs. Assessment of paper quality was guided by the recently developed Checklist for Reporting Valuation Studies. Data were extracted under six categories: general study information, choice task and study design, type of designed experiment, modelling and analysis methods, results and discussion.

Results

The literature search identified 1132 published papers, and 63 papers were included in the review. Paper quality was generally high. The study design and choice task formats varied considerably, and a wide range of modelling methods were employed to estimate value sets.

Conclusions

This review of DCE methods used for developing value sets suggests some recurring limitations, areas of consensus and areas where further research is required. Methodological diversity means that the values should be seen as experimental, and users should understand the features of the value sets produced before applying them in decision making.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  1. Brooks R. EuroQol: the current state of play. Health Policy. 1996;37:53–72.

    Article  CAS  PubMed  Google Scholar 

  2. Herdman M, Gudex C, Lloyd A, Janssen MF, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20(10):1727–36.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002;21:271–92.

    Article  PubMed  Google Scholar 

  4. de Bekker-Grob E, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2010;21(2):145–72.

    Article  PubMed  Google Scholar 

  5. Clark M, Determann D, Petrou S, et al. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32:883–902.

    Article  PubMed  Google Scholar 

  6. Bansback N, Brazier J, Tsuchiya A, Anis A. Using a discrete choice experiment to estimate societal health state utility values. J Health Econ. 2012;31(1):306–18.

    Article  PubMed  Google Scholar 

  7. Norman R, Cronin P, Viney R. A pilot discrete choice experiment to explore preferences for EQ-5D-5L health states. Appl Health Econ Health Policy. 2013;11(3):287–98.

    Article  PubMed  Google Scholar 

  8. Norman R, Viney R, Brazier J, et al. Valuing SF-6D health states using a discrete choice experiment. Med Decis Mak. 2014;34(6):773–86.

    Article  Google Scholar 

  9. Mulhern B, Bansback N, Hole AR, Tsuchiya A. Using Discrete Choice Experiments with duration to model EQ-5D-5L health state preferences: testing experimental design strategies. Med Decis Mak. 2017;37(3):285–97.

    Article  Google Scholar 

  10. Viney R, Norman R, Brazier J, et al. An Australian discrete choice experiment to value EQ-5D health states. Health Econ. 2013;23:729–42.

    Article  PubMed  Google Scholar 

  11. Jonker M, Attema A, Donkers B, Stolk E, Versteegh M. Are health state valuations from the general public biased? A test of health state reference dependency using self-assessed health and an efficient discrete choice experiment. Health Econ. 2017;26(12):1534–47.

    Article  PubMed  Google Scholar 

  12. Rowen D, Brazier J, van Hout B. A comparison of methods for converting DCE values onto the full health- dead QALY scale. Med Decis Making. 2015;35(3):328–40.

    Article  PubMed  Google Scholar 

  13. Ratcliffe J, Brazier J, Tsuchiya A, Symonds T, Brown M. Using DCE and ranking data to estimate cardinal values for health states for deriving a preference based single index from the sexual quality of life questionnaire. Health Econ. 2009;18:1261–76.

    Article  PubMed  Google Scholar 

  14. Street DJ, Burgess L. The construction of optimal stated choice experiments. Hoboken: Wiley; 2007.

    Book  Google Scholar 

  15. Bansback N, Hole AR, Mulhern B, Tsuchiya A. Testing a discrete choice experiment including duration to value health states for large descriptive systems: addressing design and sampling issues. Soc Sci Med. 2014;114:38–48.

    Article  PubMed  PubMed Central  Google Scholar 

  16. StataCorp. Stata statistical software: release 15. College Station: StataCorp LP; 2017.

    Google Scholar 

  17. Metrics Choice. Ngene [software for experimental design]. Sydney: Choice Metrics; 2012.

    Google Scholar 

  18. Craig BM, Busschbach JV. The episodic random utility model unifies time trade off and discrete choice approaches in health state valuation. Pop Health Metr. 2009;7:3.

    Article  Google Scholar 

  19. Craig BM, Rand K, Bailey H, Stalmeier PFM. Quality adjusted life years without constant proportionality. Value Health. 2018. https://doi.org/10.1016/j.jval.2018.02.004.

    Article  PubMed  Google Scholar 

  20. Norman R, Mulhern B, Viney R. The impact of different DCE-based approaches when anchoring utility scores. Pharmacoeconomics. 2016;34(8):805–14.

    Article  PubMed  Google Scholar 

  21. Hensher DA, Rose JM, Greene WH. Applied choice analysis. 2nd ed. Cambridge: University Press; 2015.

    Book  Google Scholar 

  22. Ramos Goni JM, Pinto Prades JL, Oppe M, Cabases JM, Serrano Aguiar P, Rivero Arias O. Valuation and modelling of EQ-5D-5L health states using a hybrid approach. Med Care. 2017;55(7):e51–8.

    Article  PubMed  Google Scholar 

  23. Cheung KL, Wijnen BFM, Hollin IL, Janssen EM, Bridges JF, Evers SMAA, Hiligsmann M. Using best-worst scaling to investigate preferences in health care. Pharmacoeconomics. 2016;34(12):1195–209.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Karimi M, Brazier J, Paisley S. How do individuals value health states? A qualitative investigation. Soc Sci Med. 2017;172:80–8.

    Article  CAS  PubMed  Google Scholar 

  25. Mulhern B, Bansback N, Brazier J, Buckingham K, Cairns J, Devlin N, Dolan P, Hole AR, Kavetsos G, Longworth L, Rowen D, Tsuchiya A. Preparatory study for the revaluation of the EQ-5D tariff: methodology report. Health Technol Assess. 2014;18:12.

    Article  Google Scholar 

  26. Oppe M, Devlin NJ, van Hout B, Krabbe PFM, de Charro F. A program of methodological research to arrive at the new international EQ-5D-5L valuation protocol. Value Health. 2014;17:445–53.

    Article  PubMed  Google Scholar 

  27. Kim SH, Ahn J, Ock M, Shin S, Park J, Luo N, Jo MW. The EQ-5D-5L valuation study in Korea. Qual Life Res. 2016;25(7):1845–52.

    Article  PubMed  Google Scholar 

  28. Xie F, Pickard AS, Krabbe PF, Revicki D, Viney R, Devlin N, Feeny D. A checklist for reporting valuation studies of multi-attribute utility-based instruments (CREATE). Pharmacoeconomics. 2015;33(8):867–77.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Brazier J, Connell J, Papaioannou D, Mukuria C, Mulhern B, Peasgood T, Lloyd Jones M, Paisley S, O’Cathain A, Barkham M, Knapp M, Byford S, Gilbody S, Parry G. A systematic review, psychometric analysis and qualitative assessment of Generic Preference-Based Measures of Health in Mental Health Populations and the estimation of mapping functions from widely used specific measures. Health Technol Assess. 2014;18:34.

    Article  Google Scholar 

  30. Longworth L, Yang Y, Young T, Mulhern B, Hernandez-Alava M, Mukuria C, Rowen D, Tosh J, Tsuchiya A, Evans P. Use of generic and condition specific measures of health related quality of life in NICE decision making: systematic review, statistical modelling and survey. Health Technol Assess. 2014;18:9.

    Article  Google Scholar 

  31. Hakim Z, Pathak D. Modelling the EuroQol data: a comparison of discrete choice, conjoint and conditional preference modelling. Health Econ. 1999;8:103–16.

    Article  CAS  PubMed  Google Scholar 

  32. Ryan M, Netten A, Skatun D, Smith P. Using discrete choice experiments to estimate a preference based measure of outcome—an application to social care for older people. J Health Econ. 2006;25:927–44.

    Article  PubMed  Google Scholar 

  33. Burr J, Kilonzo M, Vale L, Ryan M. Developing a preference based glaucoma utility index using a discrete choice experiment. Optom Vis Sci. 2007;84(8):797–809.

    Article  PubMed  Google Scholar 

  34. Stolk E, Oppe M, Scalone L, Krabbe P. Discrete choice modelling for the quantification of health states: the case of the EQ-5D. Value Health. 2010;13(8):1005–13.

    Article  PubMed  Google Scholar 

  35. Hauber AB, Mohamed A, Johnson FR, Oyelowo O, Curtis B, Coon C. Estimating importance weights for the IWQOL-Lite using conjoint analysis. Qual Life Res. 2011;19:701–9.

    Article  Google Scholar 

  36. Potoglu D, Burge P, Flynn T, Netten A, Malley J, Forder J, Brazier J. Best worst scaling vs. discrete choice experiments: an empirical comparison using social care data. Soc Sci Med. 2011;72:1717–27.

    Article  Google Scholar 

  37. Bailey H. Results from a preliminary study to develop the quality adjustments for quality adjusted life year values for Trinidad and Tobago. West Indian Med J. 2013;62(6):543–7.

    Article  CAS  PubMed  Google Scholar 

  38. Pullenayegum E, Xie F. Scoring the 5-level EQ-5D: can latent utilities derived from a discrete choice model be transformed to health utilities derived from a time trade off task. Med Decis Mak. 2013;33:567–78.

    Article  Google Scholar 

  39. Ramos Goni JM, Rivero-Arias O, Errea M, Stolk E, Herdman M, Cabases JM. Dealing with the health state ‘dead’ when using discrete choice experiments to obtain values for EQ-5D-5L health states. Eur J Health Econ. 2013;14(S1):S33–42.

    Article  PubMed  Google Scholar 

  40. Craig B, Pickard AS, Stolk E, Brazier J. US valuation of the SF-6D. Med Decis Mak. 2013;33(6):793–803.

    Article  Google Scholar 

  41. Krabbe P, Devlin N, Stolk E, Shah K, Oppe M, van Hout B, Quik E, Pickard AS, Xie F. Multinational evidence of the applicability and robustness of discrete choice modelling for deriving EQ-5D-5L health state values. Med Care. 2014;52(11):935–43.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Xie F, Pullenayegum E, Gaebel K, Oppe M, Krabbe P. Eliciting preferences to the EQ-5D-5L health states: discrete choice experiment or multiprofile case of best worst scaling. Eur J Health Econ. 2014;15:281–8.

    Article  PubMed  Google Scholar 

  43. Gu Y, Norman R, Viney R. Estimating health state utility values from discrete choice experiments—a QALY space model approach. Health Econ. 2014;23(9):1098–114.

    Article  PubMed  Google Scholar 

  44. van Hoorn R, Donders A, Oppe M, Stalmeier P. The better than dead method: feasibility and interpretation of a valuation study. Pharmacoeconomics. 2014;32:789–99.

    Article  PubMed  Google Scholar 

  45. Robinson A, Spencer A, Moffatt P. A framework for estimating health state utility values within a discrete choice experiment: modelling risky choices. Med Decis Mak. 2015;35(3):341–50.

    Article  Google Scholar 

  46. Hoefman R, van Exel J, Rose J, van de Wetering E, Brouwer W. A discrete choice experiment to obtain a tariff for valuing informal care situations measured with the CarerQol instrument. Med Decis Mak. 2014;34(1):84–96.

    Article  Google Scholar 

  47. Craig B, Reeve B, Brown P, Cella D, Hays R, Lipscomb J, Pickard AS, Revicki D. US valuation of health outcomes measured using the PROMIS-29. Value Health. 2014;2014(7):846–53.

    Article  Google Scholar 

  48. Scalone L, Stalmeier P, Milani S, Krabbe P. Values for health states with different life durations. Eur J Health Econ. 2015;16(9):917–25.

    Article  PubMed  Google Scholar 

  49. Gartner F, de Bekker-Grob E, Stiggelbout A, Rijnders M, Freeman L, Middeldorp J, Bloemenkamp K, de Miranda E, van den Akker-van Marle M. Calculating preference weights for the labor and delivery index: a discrete choice experiment on women’s birth experiences. Value Health. 2015;18:856–64.

    Article  PubMed  Google Scholar 

  50. Hole AR, Norman R, Viney R. Response patterns in health state valuation using endogenous attribute attendance and latent class analysis. Health Econ. 2016;25(2):212–24.

    Article  PubMed  Google Scholar 

  51. Mulhern B, Shah K, Janssen MF, Longworth L. Valuing health using Time Trade Off and Discrete Choice methods: does dimension order impact on health state values? Value Health. 2016;19(2):210–7.

    Article  PubMed  Google Scholar 

  52. Shiroiwa T, Ikeda S, Noto S, Igarashi A, Fukuda T, Saito S, Shimizuma K. Comparison of value set based on DCE and/or TTO data: scoring for EQ-5D-5L health states in Japan. Value Health. 2016;19:648–54.

    Article  PubMed  Google Scholar 

  53. Norman R, Viney R, Aaronson N, Brazier J, Cella D, Costa D, Fayers P, Kemmler G, Peacock S, Pickard AS, Rowen D, Street D, Velikova G, Young T, King M. Using a discrete choice experiment to value the QLU-C10D: feasibility and sensitivity to presentation format. Qual Life Res. 2016;25:637–49.

    Article  CAS  PubMed  Google Scholar 

  54. Craig B, Greiner W, Brown D, Reeve B. Valuation of child health related quality of life in the United States. Health Econ. 2016;25:768–77.

    Article  PubMed  Google Scholar 

  55. Craig B, Brown D, Reeve B. Valuation of child behavioural problems from the perspective of US adults. Med Decis Mak. 2016;36(2):199–209.

    Article  Google Scholar 

  56. Versteegh M, Vermeulen K, Evers S, de Wit G, Prenger R, Stolk E. Dutch tariff for the five level version of EQ-5D. Value Health. 2016;19:343–52.

    Article  Google Scholar 

  57. Bailey H, Stolk E, Kind P. Toward explicit prioritisation for the Caribbean: an EQ-5D value set for Trinidad and Tobago. Value Health Regional Issues. 2016;11C:60–7.

    Article  Google Scholar 

  58. Norman R, Kemmler G, Viney R, Pickard AS, Gamper E, Holzner B, Nerich V, King M. Order of presentation of dimensions does not systematically bias utility weights from a discrete choice experiment. Value Health. 2016;19:1033–8.

    Article  PubMed  Google Scholar 

  59. Robinson A, Spencer A, Pinto-Prades J, Covey J. Exploring differences between TTO and DCE in the valuation of health states. Med Decis Mak. 2017;37(3):273–84.

    Article  Google Scholar 

  60. Xie F, Pullenayegum E, Pickard AS, Ramos Goni JM, Jo MW, Igarashi A. Transforming latent utilities to health utilities: east does not meet west. Health Econ. 2017;26(12):1524–33.

    Article  PubMed  Google Scholar 

  61. Krucien N, Watson V, Ryan M. Is best worst scaling suitable for health state valuation? A comparison with discrete choice experiments. Health Econ. 2017;26(12):e1–16.

    Article  PubMed  Google Scholar 

  62. Mulhern B, Norman R, Lorgelly P, Lancsar E, Ratcliffe J, Brazier J, Viney R. Is dimension order important when valuing health states using Discrete Choice Experiments including duration? Pharmacoeconomics. 2017;35(4):439–51.

    Article  PubMed  Google Scholar 

  63. Goossens LMA, Rutten-van Mölken MPMH, Boland MRS, Donkers B, Jonker MF, Slok A, Salomé P,5 van Schayck OCP, in ‘t Veen JCCM, Stolk EA. ABC Index: quantifying experienced burden of COPD in a discrete choice experiment and predicting costs. BMJ Open. 2017;7:e017831.

  64. Huynh E, Coast J, Rose J, Kinghorn P, Flynn T. Values for the ICECAP-supportive care measure (ICECAP-SCM) for use in economic evaluation at end of life. Soc Sci Med. 2017;189:114–28.

    Article  PubMed  Google Scholar 

  65. Purba FD, Hunfeld JAM, Iskandarsyah A, Fitriana TS, Sadarjoen SS, Ramos-Goni JM, Passchier J, van Busschbach JJ. The Indonesian EQ-5D-5L value set. Pharmacoeconomics. 2017;35(11):1153–65.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Hoefman RJ, van Exel J, Brouwer WB. Measuring care-related quality of life of caregivers for use in economic evaluations: CarerQol tariffs for Australia, Germany, Sweden, UK, and US. Pharmacoeconomics. 2017;35(4):469–78.

    Article  PubMed  Google Scholar 

  67. Jonker MF, Donkers B, de Bekker-Grob E, Stolk E. The effect of level overlap and color coding on attribute non-attendance in discrete choice experiments. Value Health. 2018;21(7):767–71.

    Article  PubMed  Google Scholar 

  68. Devlin N, Shah K, Feng Y, Mulhern BJ, van Hout B. Valuing health-related quality of life: an EQ-5D-5L value set for England. Health Econ. 2018;27(1):7–22.

    Article  PubMed  Google Scholar 

  69. Feng Y, Devlin N, Shah K, Mulhern BJ, van Hout B. New methods for modelling EQ-5D-5L value sets: an application to English data. Health Econ. 2018;27(1):23–38.

    Article  PubMed  Google Scholar 

  70. Rowen D, Mulhern B, Stevens K, Vermaire E. Estimating a Dutch value set for the paediatric preference-based CHU-9D using a discrete choice experiment with duration. Value Health. 2018. https://doi.org/10.1016/j.jval.2018.03.016.

    Article  PubMed  Google Scholar 

  71. King MT, Viney R, Pickard AS, Rowen D, Aaronson NK, Brazier JE, Cella D, Costa D, Fayers P, Kemmler G, McTaggart-Cowen H, Mercieca-Bebber R, Peacock S, Street DJ, Young TA, Norman R. Australian utility weights for the EORTC QLU-C10D, a multi-attribute utility instrument derived from the cancer-specific quality of life questionnaire, EORTC QLQ-C30. PharmacoEconomics. 2018;36(2):225–38.

    Article  PubMed  Google Scholar 

  72. Cole A, Shah K, Mulhern B, Feng Y, Devlin N. Valuing EQ-5D-5L health states ‘in context’ using a discrete choice experiment. Eur J Health Econ. 2018;19(4):595–605.

    Article  PubMed  Google Scholar 

  73. Mulhern B, Norman R, Shah K, Bansback N, Longworth L, Viney R. How should DCE with duration choice sets be presented for the valuation of health states? Med Decis Mak. 2018;38(3):306–18.

    Article  Google Scholar 

  74. Purba F, Hunfeld JAM, Timman R, Iskandarsyah A, Fitriana T, Sadarjoen S, Passchier J, Busschbach JJV. Test-retest reliability of EQ-5D-5L valuation techniques: the composite time trade off and discrete choice experiments. Value Health. 2018. https://doi.org/10.1016/j.jval.2018.02.003.

    Article  PubMed  Google Scholar 

  75. Gamper EM, Holzner B, King MT, Norman R, Viney R, Nerich V, Kemmler G. Test-retest reliability of discrete choice experiment for valuations of QLU-C10D health states. Value Health. 2018;21(8):958–66.

    Article  PubMed  Google Scholar 

  76. Ramos-Goñi JM, Craig B, Oppe M, Ramallo-Fariña Y, Pinto-Prades JL, Luo N, Rivero-Arias O. Handling data quality issues to estimate the Spanish EQ-5D-5L value set using a hybrid interval regression approach. Value Health. 2018;21(5):596–604.

    Article  PubMed  Google Scholar 

  77. Craig B, Rand K. Choice defines QALYs: a US valuation of the EQ-5D-5L. Med Care. 2018;56(6):529–36.

    Article  PubMed  Google Scholar 

  78. Jakubczyk M, Craig B, Barra M, Groothuis-Oudshoorn C, Hartman J, Huynh E, Ramos-Goñi JM, Stolk E, Rand K. Choice defines value: a predictive modeling competition in health preference research. Value Health. 2018;21(2):229–38.

    Article  PubMed  Google Scholar 

  79. Craig BM, Rand K, Bailey H, Stalmeier P. Quality-adjusted life-years without constant proportionality. Value Health. 2018. https://doi.org/10.1016/j.jval.2018.02.004.

    Article  PubMed  Google Scholar 

  80. Jonker M, Donkers B, de Bekker-Grob E, Stolk E. Advocating a paradigm shift in health-state valuations: the estimation of time-preference corrected QALY tariffs. Value Health. 2018;21(8):993–1001.

    Article  PubMed  Google Scholar 

  81. Feng Y, Hole AR, Karimi M, Tsuchiya A, van Hout B. An exploration of the non-iterative time trade-off method to value health states. Health Econ. 2018;27(8):1247–63.

    Article  PubMed  PubMed Central  Google Scholar 

  82. Hole AR. CLOGITHET: Stata module to estimate heteroscedastic conditional logit models. Statistical Software Components S456737; 2006.

  83. Hole AR. Small-sample properties of tests for heteroscedasticity in the conditional logit model. Econ Bull. 2006;3(18):1–14.

    Google Scholar 

  84. Swait J, Louviere J. The role of the scale parameter in the estimation and comparison of multinomial logit models. J Mark Res. 1993;30(3):305–14.

    Article  Google Scholar 

  85. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Mulhern B, Longworth L, Brazier JE, Rowen D, Bansback N, Devlin N, Tsuchiya A. Health state valuation questions: head to head comparison of online and CAPI. Value Health. 2013;16(1):104–13.

    Article  PubMed  PubMed Central  Google Scholar 

  87. Street DJ, Mulhern B, Norman R, Viney R. Using simulations to compare DCE designs that could be used to value EQ-5D. Barcelona: EuroQol Plenary; 2017.

    Google Scholar 

Download references

Acknowledgements

The authors thank Liz Chinchen for her support with the literature searching, and Elly Stolk for comments on an earlier version of the review. This work was presented at the EuroQol Group Plenary (Krakow 2015) and the International Society of Pharmacoeconomics and Outcomes Research (Washington 2016) meetings.

Author information

Authors and Affiliations

Authors

Contributions

BM carried out the literature search, extracted the data, led the data synthesis and interpretation and developed the first draft of the manuscript. RN, RV and DJS supported the data-extraction process and interpretation of the results and were involved in the development of the manuscript.

Corresponding author

Correspondence to Brendan Mulhern.

Ethics declarations

Funding

This review was partly funded by the Australian National Health and Medical Research Council. Mr Mulhern was funded by a University of Technology Sydney President’s Scholarship.

Conflicts of interest

BM, RN, DJS and RV have no conflicts of interest.

Data availability statement

No datasets were generated or analysed during this study.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mulhern, B., Norman, R., Street, D.J. et al. One Method, Many Methodological Choices: A Structured Review of Discrete-Choice Experiments for Health State Valuation. PharmacoEconomics 37, 29–43 (2019). https://doi.org/10.1007/s40273-018-0714-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-018-0714-6

Navigation