A Samuelsonian validation test for happiness data☆
Introduction
There are two possible meanings of the term utility. On the one hand, Samuelson’s decision utility refers to preferences as revealed by observed choices (Samuelson, 1948). On the other hand, Bentham’s experienced utility refers to the actual feelings of pleasure and pain that an individual experiences in response to certain stimuli (Kahneman, Wakker, & Sarin, 1997). Samuelson’s revealed preference principle is arguably the most important and influential concept in economics. However, this decision utility may sometimes be difficult to estimate or less reliable. For example, economic analysis often relies on the assumption that decision utility reflects experienced utility, but this assumption can be violated by bounded rationality, endogenous preferences, or problems of self-control, among others (Chetty, 2015, Kahneman et al., 1997).1
An alternative measure of experienced utility would be valuable for situations where decision utility is less reliable, too costly, or impossible to estimate. Economists and other social scientists have long sought a direct measurement of experienced utility. For example, Edgeworth (1881) fantasized about a hedonimeter, an instrument capable of measuring pleasure in real time. Neuroscience may someday provide this tool, but for now we rely on other sources of data. Among social scientists, there has been exponential growth in the use of subjective well-being data. For example, some studies use data on responses to questions like, “Are you very happy, pretty happy, or not too happy?” This literature, sometimes referred to as happiness economics, has grown rapidly over the last few decades (for a review, see Di Tella & MacCulloch (2006)). This growth was facilitated by the availability of subjective well-being data in countless surveys around the world and spanning several decades. These studies often compare the well-being of individuals or nations (e.g., Easterlin, 1974, Kahneman and Deaton, 2010, Ludwig et al., 2012, Stevenson and Wolfers, 2008, Stevenson and Wolfers, 2009). However, a significant number of these studies use subjective well-being data to test theories about human behavior (e.g., Easterlin, 1995, Gruber and Mullainathan, 2005, Luttmer, 2005, Oswald and Powdthavee, 2008, Perez-Truglia, 2013, Senik, 2004). In spite of the increasing use of subjective well-being data, a majority of economists remain skeptical, although this skepticism may be about subjective data in general and not just well-being data (Bertrand & Mullainathan, 2001).
Several studies provide validation tests for subjective well-being measures. These tests typically consist of gauging whether a subjective measure of well-being is positively correlated to objective measures of well-being, such as emotional expressions (e.g., Sandvik, Diener, & Seidlitz, 1993), neurological measures (e.g., Urry et al., 2004), third-party evaluations (e.g., Sandvik et al., 1993), aggregate suicide rates (e.g., Di Tella, MacCulloch, & Oswald, 2003), among others (for a recent review, see Diener, Inglehart, & Tay (2013)). Most economists seem unpersuaded by this evidence, because it does not discard the possibility that these subjective scores measure superficial aspects of life, like mood. For instance, suppose that reporting to like the Rolling Stones is positively correlated to the frequency of smiling. This correlation is not sufficient evidence to support a Rolling Stones Index to guide economic policy or test economic theories. Nevertheless, psychologists have devoted a large literature to investigating these subjective scores measures. They have identified several measurement challenges, such as sensitivity to contextual factors and selective memory, and developed tools to address them (e.g., Stone, Shiman, & DeVries, 1999).2 However, methodological differences make such insights less compelling for the average economist. Thus, this paper proposes a framework for testing the validity of a subjective well-being measure that should be more compelling for economists.
Consider a context in which decision utility is believed to provide an accurate approximation of experienced utility. If, in that context, reported utility is similar to decision utility, then that would provide a compelling argument for using reported utility in cases where decision utility is absent or less reliable. Based on this insight, we propose a test consisting of three steps: using regression analysis to infer preferences from subjective well-being data; using those estimated parameters to predict how a rational utility-maximizer individual should behave; and comparing the predicted behavior with the actual behavior.
In addition to formulating a general version of the test, we provide a simple application based on a model of household consumption. To illustrate the intuition behind this application, assume that we have an hedonimeter that is capable of measuring well-being, . After experimenting with different combinations of consumptions of two goods, and , we find the following relationship in the data: . If this individual rationally maximized , then she should choose an expenditure share equal to . To test the validity of the hedonimeter, we should measure the expenditure share that this individual chooses in reality, and compare that share to the prediction of the hedonimeter.
We estimate a model of food consumption using panel data on subjective well-being, expenditures, and prices from a Russian household panel. The life satisfaction data is used to estimate consumption preferences in two categories of food: animal source and non-animal source. We assume that preferences are heterogeneous across individuals but stable over time. Thus, we can exploit the panel structure of the data to estimate individual-specific preference parameters. Ideally, we would test whether the actual and predicted choices are equal for every individual. However, because of data limitations, our application lacks enough power to perform an ideal version of the test. Instead, we perform a weaker version of the test by measuring the cross-individual association between actual and predicted expenditure choices.
We find that, across individuals, a one percentage point increase in the actual expenditure share is associated with an increase of 0.76 (SE 0.196) percentage points in the expenditure share predicted by life satisfaction. Indeed, we cannot reject the hypothesis of a regression coefficient of 1, which corresponds to the ideal value in the context of this test. However, given the imprecise estimation of this parameter, we cannot reject the hypothesis that the coefficient equals 0.5 either. Additionally, we show that life satisfaction performs significantly better in this test relative to other objective and subjective measures of well-being (e.g., income, satisfaction with income, perception of economic ranking, and health self-evaluation). The evidence suggests that life satisfaction offers useful information about experienced utility. However, it is unknown whether life satisfaction would perform well in the strong version of the test.
To the best of our knowledge, only a few papers provide validation tests that are similar to our test. First, Oswald and Wu (2010) use objective data to compare state-level averages of happiness and a quality-of-life measure inferred from residential choices. They report a statistically and economically significant correlation coefficient of 0.6 between the subjective and objective measures of well-being. Although they do not compare predicted behavior to actual behavior, as we propose, their test is implicitly based on the comparison between reported and decision utility.
Second, Benjamin, Heffetz, Kimball, and Rees-Jones (2012) provide a test based on a cleverly designed survey. Subjects are shown pairs of hypothetical tradeoff scenarios (e.g., higher income but longer workdays). Subjects are then asked to report which scenario they would choose and which scenario would make them happier. The evidence suggests that, despite significant discrepancies, predicted choice is largely consistent with predicted well-being: 83% of the time, respondents chose the scenario with the highest anticipated well-being. In a follow-up paper, Benjamin, Heffetz, Kimball, and Rees-Jones (2014) present further evidence from a survey of students from U.S. medical schools shortly after submitting their choice rankings for the National Resident Matching Program. They analyze the students’ choice rankings, anticipated subjective well-being rankings, and expectations about the quality of residencies (e.g., prestige). They estimate the marginal rates of substitution inferred from choice rankings and the anticipated subjective well-being rankings. They find that, even though life satisfaction does better than other subjective measures of well-being, the trade-offs inferred from life satisfaction are substantially different from the trade-offs inferred from choice rankings.3
Benjamin et al., 2012, Benjamin, Heffetz, Kimball, and Rees-Jones, 2014 study the validity of hypothetical well-being data (e.g., they ask whether the respondent anticipates more life satisfaction for particular hypothetical situations). However, the vast majority of the subjective well-being literature uses stated well-being data (e.g., whether the respondent feels a little or a lot satisfied with her life at the moment). Although the evidence in Benjamin et al., 2012, Benjamin, Heffetz, Kimball, and Rees-Jones, 2014 is suggestive, it does not directly address the validity of stated well-being data.4 Our paper tries to fill this gap.
Section snippets
A general validation test for subjective well-being data
Let denote a vector with the choice variables, where the subscript i indexes individuals and subscript t indexes time. For instance, the elements in could denote consumption of different goods or work hours. Let denote the utility function, where is a vector of preference parameters. Let denote a measure of subjective well-being (i.e., reported utility). Consider the following regression equation:
Given a set of instrumental variables , i.e.,
An application to consumption choices
We illustrate the approach with a very simple application: the choice of allocate a fixed budget across J goods.9 Let subscripts i and t denote
Data
We use panel data from rounds 5 to 20 (corresponding to years 1994–2011) of the Russian Longitudinal Monitoring Survey13 (RLMS).
Results
Table 3 presents the final results. All coefficients are shown with bootstrapped standard errors, clustered at the individual level. Column (1) reports the baseline specification, which uses life satisfaction as the dependent variable. Columns (2) though (7) report specifications with additional control variables or with alternative dependent variables.
Table 3 reports the averages of the predicted and actual shares of expenditures in animal source food. Both of these averages are computed for
Conclusions
We proposed a general validity test for subjective well-being data. The test consists of three steps: inferring preferences from subjective well-being data through regression analysis; using those estimates to predict how a rational utility-maximizer individual should act; and comparing predicted behavior with actual behavior. We provided an application of this test based on a model of food consumption. In this application, the estimates of the predicted choices are not nearly precise enough to
References (32)
Does economic growth improve the human lot? Some empirical evidence
Nations and Households in Economic Growth
(1974)Will raising the incomes of all increase the happiness of all?
Journal of Economic Behavior and Organization
(1995)- et al.
Does happiness adapt? A longitudinal study of disability with implications for economists and judges
Journal of Public Economics
(2008) A test of the conspicuous-consumption model using subjective well-being data
Journal of Socio-Economics
(2013)When information dominates comparison: Learning from Russian subjective panel data
Journal of Public Economics
(2004)- Becker, G. S., & Rayo, L. (2008). Comment on ’Economic growth and subjective well-being: Reassessing the Easterlin...
- et al.
What do you think would make you happier? What do you think you would choose?
American Economic Review
(2012) - et al.
Can marginal rates of substitution be inferred from happiness data? Evidence from residency choices
American Economic Review
(2014) - et al.
Beyond happiness and satisfaction: Toward well-being indices based on stated preference
American Economic Review
(2014) - et al.
Do people mean what they say? Implications for subjective survey data
American Economic Review Papers and Proceedings
(2001)
Behavioral economics and public policy: A pragmatic perspective
American Economic Review
Theory and validity of life satisfaction scales
Social Indicators Research
Happiness is the frequency, not the intensity, of positive versus negative affect
Some uses of happiness data in economics
The Journal of Economic Perspectives
The macroeconomics of happiness
Review of Economics and Statistics
Cited by (25)
Using memories to assess the intrapersonal comparability of wellbeing reports
2022, Journal of Economic Behavior and OrganizationIncome-comparison attitudes in the United States and the United Kingdom: Evidence from discrete-choice experiments
2019, Journal of Economic Behavior and OrganizationSuboptimal behavior in strategy-proof mechanisms: Evidence from the residency match
2018, Games and Economic BehaviorWhen experienced and decision utility concur: The case of income comparisons
2017, Journal of Behavioral and Experimental EconomicsCitation Excerpt :The authors conclude that we should then be wary of using the results from the empirical analysis of happiness or life-satisfaction questions to inform us about the trade-offs that appear in actual choices. Perez-Truglia (2015) also sounds a warning note. He considers the concordance of SWB data and behaviour by estimating consumption functions from SWB data and then looking at the difference between predicted consumption and actual consumption.
Would you choose to be happy? Tradeoffs between happiness and the other dimensions of life in a large population survey
2017, Journal of Economic Behavior and OrganizationCitation Excerpt :It is critical to note at this point that the studies mentioned above focus on respondents’ anticipated change of happiness. In a recent study, Perez-Truglia (2015) directly addresses the validity of ‘currently’ stated happiness reports. Using food consumption and happiness data, the study infers preferences from happiness data via regression analysis, predicts how a utility-maximizing individual should act based on these estimates, and draws comparisons to actual behavior to find overall that, overall, actual happiness does predict choice.
Experienced versus decision utility: large-scale comparison for income–leisure preferences<sup>*</sup>
2023, Scandinavian Journal of Economics
- ☆
Thanks to Rafael Di Tella and Sendhil Mullainathan for very valuable discussions. Thanks to two anonymous reviewers for their great feedback that substantially improved the paper. Thanks to Daniel Benjamin, Nicolas Bottan, Jan-Emmanuel De Neve, Martin Tetaz and seminar participants at the London School of Economics for useful comments. The author would like to acknowledge support from the Simon Kuznets Research Grant from the Department of Economics at Harvard University. The usual disclaimer applies.