Patient Perception, Preference and Participation
Adaptive Conjoint Analysis as individual preference assessment tool: Feasibility through the internet and reliability of preferences

https://doi.org/10.1016/j.pec.2009.05.020Get rights and content

Abstract

Objective

Patient values are not routinely assessed in clinical practice. Adaptive Conjoint Analysis (ACA) is increasingly applied in studies assessing treatment preferences, and could provide a means to routinely assess individual patients’ treatment preferences.

Methods

An ACA-questionnaire was administered three times (7–10 days apart) to 98 long-term rectal cancer survivors either on a portable computer or through internet, to assess whether (a) responses differ according to administration mode, (b) relative importances of rectal cancer treatment outcomes (survival, local control, incontinence, sexual problems) consolidate over time, (c) ACA-outcomes are sufficiently reliable (ICC) for use in individual decision-making. We also evaluated patients’ acceptance of ACA.

Results

Mode did not affect ACA-completion or evaluation. Importance scores did not consolidate over time. ICCs were poor for sexual problems and fair for the other outcomes, and were at least equal or higher from first to second retest. Most participants valued completing the ACA-questionnaire and learning their results.

Conclusion

Values did not show consolidation over time. ACA-derived preferences should not determine which treatment patients should choose.

Practice implications

Findings extend ACA-validation studies to the health care setting and suggest that ACA-questionnaires might be appreciated as adjuncts to treatment decision-making in newly diagnosed patients.

Introduction

Clinicians often do not adequately judge the importance patients place on treatment aspects or outcomes [1], [2], [3], [4]. Especially preference-sensitive decisions should incorporate patient values [5]. However, the latter are not routinely assessed in practice. Several methodologies have been developed to clarify personal values patients associate with treatment aspects. Often-used techniques such as decision boards [6], [7], videotapes [8], [9], and rating scales ask for treatment preference. They do not reveal how treatment aspects contribute to overall judgments [10], so results are limited to particular clinical contexts. Techniques such as Standard Gambles and Time TradeOffs, measure utilities for treatment aspects; their applicability at the individual level is questionable [11]. Balance scales ask individuals to nominate and value treatment aspects they consider relevant. These exercises are straightforward but a-theoretical.

Adaptive Conjoint Analysis (ACA)-based exercises seem good candidates to implement in practice as values clarification tasks to aid individually tailored treatment decision-making. ACA originates from traditional conjoint analysis and has a strong theoretical basis [12], [13], [14]. Using this decomposed methodology, the relative importance of treatment aspects may be investigated. The ACA-questionnaire is computer-administered and presents paired combinations of treatment aspects. The pairs are increasingly tailored to what participants consider relevant tradeoffs, thereby fostering involvement in the task [15] and possibly assisting patients’ thoughtful evaluation. ACA is increasingly applied in studies assessing preferences at a group-level, including studies among HIV [16], [17], rheumatology [18], [19], [20], [21], and oncology patients [22], patients with growth hormone deficiencies [23], and patients about to undergo major surgery [24]. In this study we wished to assess the usefulness of ACA at the individual level, to support treatment-related decision-making.

The choice for a preference elicitation technique in the clinic should be directed by its feasibility and reliability. Validity criteria are difficult to set. People may not be familiar with thinking about the value of health outcomes [25], [26]. Consequently, it has been long acknowledged that preferences are being constructed while being elicited rather than revealed [27], hampering the assessment of criterion validity.

We have assessed the internal and convergent validity, and reliability of ACA-derived preferences in disease-free rectal cancer patients [28]. Primary rectal cancer treatment can be considered a preference-sensitive decision. Six-year follow-up trial data have shown that adding preoperative radiotherapy (PRT) to surgery results in a reduction in the local recurrence rate from 11% to 6% with no significant survival benefit [29], [30]. Both surgery and PRT followed by surgery induce risks of faecal incontinence and sexual dysfunction, yet probabilities are higher with PRT [31], [32]. Our results showed participants’ responses to be consistent and valid overall. Also, the ACA-questionnaire captured preferences independently from treatment experience. Test–retest data suggested that results were sufficiently reliable for use at the group but not individual level. However, people's thinking about the task may have changed their values over time, invalidating the reliability measure. Possible changes in valuations over time are most expected when people start considering the issues, rather than when they have already spent time thinking them over. We might therefore rather detect value changes between a first and second test, than between a second and third test. A second retest may shed light on whether values appear to consolidate over time.

In our previous study, participants responded to the ACA-questionnaire in a Windows (Computer-aided Personal Interview, CAPI) version on a portable computer in the presence of an interviewer. The possibility to complete the questionnaire through the internet may be worthwhile for implementation in practice, in terms of limiting time investment for clinic personnel and of flexibility in use for patients. Differences in utilities based on CAPI- versus Internet-versions were found minimal in a study valuing notebooks [33]. It is uncertain whether valuations of health issues differ according to modality, and if users’ evaluation of ACA differs according to administration mode. Additionally, it may be useful in practice to obtain immediate feedback on how patients value treatment aspects, based on their responses to the task. Such feedback may facilitate the inclusion of values information in doctor–patient discussions on treatment choice. ACA was originally not designed with a feedback module. It is therefore unclear how participants value such feedback.

This study was set up to assess (a) the effect of administration mode on ACA-scores, (b) whether values consolidate over time, (c) whether the reliability of treatment outcome importance scores and treatment preferences show these to unequivocally indicate preferences, and (d) individuals’ evaluation of the ACA-questionnaire and immediate feedback, including the value of ACA-questionnaires as adjunct to treatment decision-making.

Section snippets

Participants

Participants were disease-free rectal cancer patients who had participated in a multicentre trial [29]. We exhausted the sample of trial participants who had been treated in the Netherlands; had agreed to being approached for further research at five-year follow-up [32]; were alive and free of recurrent disease or other tumours at last moment of yearly follow-up; had not been approached in an earlier study on preferences for PRT [28]; and were at most 75 years old when the inclusion started.

Participants

Two-hundred and twenty-nine eligible patients were approached. Fifteen could not be reached, and 15 were deceased. Of the remaining 199 patients, 115 (58%) agreed to participate. Twenty-one did not respond in spite of a reminder and 63 declined participation because they did not want to be reminded of the past (23), did not have a computer or lacked computer skills (15), refused the time commitment (4), were in grief (3), felt they could not evaluate the pros and cons of PRT (2), were visually

Discussion

Conceptually, the value of ACA-questionnaires as adjuncts to treatment-related decision-making would lie in their ability to help patients think about how they value treatment aspects, particularly in decision situations that are new to patients. Practically, ACA-exercises need to be doable for and acceptable to patients and to provide results that are helpful in decision-making processes.

ACA-questionnaires need to be administered by computer. In our sample of older adults, it turned out that

Acknowledgements

Financial support for this study was provided by a grant from the Dutch Cancer Society (UL 2005-3213) and a grant from the Netherlands Organization for Scientific Research NWO Innovational Research Incentives (Grant number 917.56.356).

References (41)

  • K. Sepucha et al.

    Doing the right thing: systems support for decision quality in cancer care

    Ann Behav Med

    (2006)
  • J.J. Tol-Geerdink et al.

    Do patients with localized prostate cancer treatment really want more aggressive treatment?

    J Clin Oncol

    (2006)
  • T. Whelan et al.

    Effect of a decision aid on knowledge and treatment decision making for breast cancer surgery: a randomized trial

    J Amer Med Assoc

    (2004)
  • A.B. Flood et al.

    The importance of patient preference in the decision to screen for prostate cancer. Prostate patient outcomes research team

    J Gen Intern Med

    (1996)
  • H.A. Llewellyn-Thomas

    Investigating patients’ preferences for different treatment options

    Can J Nurs Res

    (1997)
  • M. Ryan et al.

    Eliciting public preferences for healthcare: a systematic review of techniques

    Health Technol Assess

    (2001)
  • P.E. Green et al.

    Conjoint measurement for quantifying judgmental data

    J Mark Res

    (1971)
  • M. Ryan et al.

    Using conjoint analysis to elicit preferences for health care

    Brit Med J

    (2000)
  • Sawtooth Software. ACA 6.0 Technical Paper 2007, <www.sawtoothsoftware.com> [accessed...
  • K.M. Beusterien et al.

    Patient preferences among third agent HIV medications: a US and German perspective

    AIDS Care

    (2007)
  • Cited by (0)

    View full text