Several methods to investigate relative attribute impact in stated preference experiments

Abstract

There is growing use of discrete choice experiments (DCEs) to investigate preferences for products and programs and for the attributes that make up such products and programs. However, a fundamental issue overlooked in the interpretation of many choice experiments is that attribute parameters estimated from DCE response data are confounded with the underlying subjective scale of the utilities, and strictly speaking cannot be interpreted as the relative “weight” or “impact” of the attributes, as is frequently done in the health economics literature. As such, relative attribute impact cannot be compared using attribute parameter size and significance. Instead, to investigate the relative impact of each attribute requires commensurable measurement units; that is, a common, comparable scale. We present and demonstrate empirically a menu of five methods that allow such comparisons: (1) partial log-likelihood analysis; (2) the marginal rate of substitution for non-linear models; (3) Hicksian welfare measures; (4) probability analysis; and (5) best–worst attribute scaling. We discuss the advantages and disadvantages of each method and suggest circumstances in which each is appropriate.

Introduction

A common objective of discrete choice experiments (DCEs) is to compare the relative impact of attributes of the product or program under investigation. For example, is test accuracy relatively more important to patients than time spent waiting for results when choosing diagnostic tests? Most studies compare relative impacts of attributes by comparing the size and significance of estimated parameters for attributes of interest. Unfortunately, these parameters are not directly comparable because the attribute parameter estimates in discrete choice models (DCMs) are confounded with the underlying subjective utility scale. That is, parameter estimates combine the relative impact or importance of an attribute and the utility scale values associated with its levels. Thus, utility estimates for attribute levels cannot be interpreted as indicating relative importance of an attribute.

In particular, the estimated utility of each attribute level is measured on an interval scale, but the origins and units of each attribute's utility scale differ. Apart from obvious differences in underlying physical attribute units like price in dollars, time in minutes/hours etc., qualitative attributes have no physical referents. For example, attribute levels for “provider of care” might be nurse, doctor, etc. Thus, distances between the levels of different attributes need not have the same meaning. So, utility scale locations, or utility differences between levels of different attributes, generally do not have equal scale units. One can equate the origins of each scale, but not the scale units; hence, direct comparisons of ranges of utility estimates are meaningless without transforming them in a theoretically acceptable way, or modifying a choice experiment. Put simply, one cannot determine whether the magnitudes of the parameter estimates for an attribute's levels, and hence the resulting range of parameter estimates for these levels, are due to the “impact” of that attribute or the position of each attribute level on the underlying utility scale. To assess relative attribute impacts one needs to measure each on a common, comparable scale.

The purpose of this paper is to focus attention on the confound between attribute impact and attribute level scale utilities in DCEs, and to outline and discuss five ways to compare relative attribute impacts: (1) partial log-likelihood analysis; (2) marginal rates of substitution (MRS); (3) Hicksian welfare measures; (4) probability analysis; and (5) best worst attribute scaling (BWAS). The first four methods deal with the issue of relative attribute impact within a traditional DCE. We demonstrate these in an empirical application, which to our knowledge is the first health-related DCE to include two-way attribute interactions in a non-linear indirect utility function (IUF). The BWAS method is a modified DCE.

The rest of the paper is organised as follows. The next section discusses the theoretical background for the confound between attribute impact and level scale. The third section outlines a menu of five methods to investigate the relative impact of attributes that are illustrated in two empirical applications in the fourth section. The fifth section discusses advantages and disadvantages of each method and circumstances in which each may be appropriate. The final section concludes.