Skip to main content
Top
Gepubliceerd in:

25-05-2023 | Review

The effect of incentivization on the conjunction fallacy in judgments: a meta-analysis

Auteurs: Eldad Yechiam, Dana Zeif

Gepubliceerd in: Psychological Research | Uitgave 8/2023

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

The conjunction fallacy is a classical judgment bias that was argued to be a robust cognitive illusion insensitive to the positive effect of incentivization. We conducted a meta-analysis of the literature (n = 3276) and found that although most studies did not report a significant effect of incentivization, the results across studies show a significant positive effect for incentivization, d = 0.19, with an odds ratio of 1.40 for answering correctly when incentivized. There was no moderating effect of payoff size despite the differences in incentive value between studies. Additionally, the effect was relatively smaller when examining absolute differences in the probability of correct judgment instead of odds ratios, suggesting that it may be partly driven by studies with low baseline performance. These findings join those of other judgment-bias studies to suggest a small but nevertheless robust debiasing effect of incentivization.
Bijlagen
Alleen toegankelijk voor geautoriseerde gebruikers
Voetnoten
1
Likewise, the liberal paternalism (or “nudge”) approach was originally based on the assumption that people cannot in practice increase their cognitive effort (e.g., through incentives) so as to overcome judgment and decision biases (Thaler & Sunstein, 2008).
 
2
Reviews of this literature concluded either that incentives had a positive effect on judgment performance (Hertwig & Ortmann, 2001) or that it had a very limited effect (Camerer & Hogarth, 1999).
 
3
On average the rate of correct choices in the non-incentivized condition was 45% compared to 55% in the incentivized condition.
 
4
Rather, the two metrics were used as robustness tests. Formally comparing the two statistics likely requires a larger number of studies.
 
Literatuur
go back to reference Arkes, H. R. (1991). Costs and benefits of judgment errors: Implications for debiasing. Psychological Bulletin, 110, 486–498.CrossRef Arkes, H. R. (1991). Costs and benefits of judgment errors: Implications for debiasing. Psychological Bulletin, 110, 486–498.CrossRef
go back to reference Awasthi, V., & Pratt, J. (1990). The effects of monetary incentives on effort and decision performance: The role of cognitive characteristics. Accounting Review, 65, 797–811. Awasthi, V., & Pratt, J. (1990). The effects of monetary incentives on effort and decision performance: The role of cognitive characteristics. Accounting Review, 65, 797–811.
go back to reference Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101.PubMedCrossRef Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101.PubMedCrossRef
go back to reference Brañas-Garza, P., Kujal, P., & Lenkei, B. (2019). Cognitive reflection test: Whom, how, when. Journal of Behavioral and Experimental Economics, 82, 101455.CrossRef Brañas-Garza, P., Kujal, P., & Lenkei, B. (2019). Cognitive reflection test: Whom, how, when. Journal of Behavioral and Experimental Economics, 82, 101455.CrossRef
go back to reference Camerer, C. F., & Hogarth, R. M. (1999). The effects of financial incentives in experiments: A review and capital-labor-production framework. Journal of Risk and Uncertainty, 19, 7–42.CrossRef Camerer, C. F., & Hogarth, R. M. (1999). The effects of financial incentives in experiments: A review and capital-labor-production framework. Journal of Risk and Uncertainty, 19, 7–42.CrossRef
go back to reference Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014). Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis. Psychological Bulletin, 140, 980–1008.PubMedCrossRef Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014). Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis. Psychological Bulletin, 140, 980–1008.PubMedCrossRef
go back to reference Chandrashekar, S.P., Cheng, Y.H., Fong, C.L., Leung, Y.C., Wong, Y.T., Cheng, B.L., & Feldman, G. (2021). Frequency estimation and semantic ambiguity do not eliminate conjunction bias, when it occurs: Replication and extension of Mellers, Hertwig, and Kahneman (2001). Meta-Psychology, 5, MP.2020.2474. Chandrashekar, S.P., Cheng, Y.H., Fong, C.L., Leung, Y.C., Wong, Y.T., Cheng, B.L., & Feldman, G. (2021). Frequency estimation and semantic ambiguity do not eliminate conjunction bias, when it occurs: Replication and extension of Mellers, Hertwig, and Kahneman (2001). Meta-Psychology, 5, MP.2020.2474.
go back to reference Charness, G., Karni, E., & Levin, D. (2010). On the conjunction fallacy in probability judgment: New experimental evidence regarding Linda. Games and Economic Behavior, 68, 551–556.CrossRef Charness, G., Karni, E., & Levin, D. (2010). On the conjunction fallacy in probability judgment: New experimental evidence regarding Linda. Games and Economic Behavior, 68, 551–556.CrossRef
go back to reference Cortina, J. M., & Landis, R. S. (2009). When small effect sizes tell a big story, and when large effect sizes don’t. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences (pp. 287–308). Routledge. Cortina, J. M., & Landis, R. S. (2009). When small effect sizes tell a big story, and when large effect sizes don’t. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences (pp. 287–308). Routledge.
go back to reference Dale, D., Rudski, J., Schwarz, A., & Smith, E. (2007). Innumeracy and incentives: A ratio bias experiment. Judgment and Decision Making, 2, 243–250.CrossRef Dale, D., Rudski, J., Schwarz, A., & Smith, E. (2007). Innumeracy and incentives: A ratio bias experiment. Judgment and Decision Making, 2, 243–250.CrossRef
go back to reference Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455–463.PubMedCrossRef Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455–463.PubMedCrossRef
go back to reference Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634.PubMedPubMedCentralCrossRef Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634.PubMedPubMedCentralCrossRef
go back to reference Enke, B., Gneezy, U., Hall, B., Martin, D., Nelidov, V., Offerman, T., & van de Ven, J. (2022). Cognitive biases: Mistakes or missing stakes? In press. Enke, B., Gneezy, U., Hall, B., Martin, D., Nelidov, V., Offerman, T., & van de Ven, J. (2022). Cognitive biases: Mistakes or missing stakes? In press.
go back to reference Epley, N., & Gilovich, T. (2005). When effortful thinking influences judgmental anchoring: Differential effects of forewarning and incentives on self-generated and externally provided anchors. Journal of Behavioral Decision Making, 18, 199–212.CrossRef Epley, N., & Gilovich, T. (2005). When effortful thinking influences judgmental anchoring: Differential effects of forewarning and incentives on self-generated and externally provided anchors. Journal of Behavioral Decision Making, 18, 199–212.CrossRef
go back to reference Erev, I., & Cohen, B. L. (1990). Verbal versus numerical probabilities: Efficiency, biases, and the preference paradox. Organizational Behavior and Human Decision Processes, 45, 1–18.CrossRef Erev, I., & Cohen, B. L. (1990). Verbal versus numerical probabilities: Efficiency, biases, and the preference paradox. Organizational Behavior and Human Decision Processes, 45, 1–18.CrossRef
go back to reference Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.CrossRef Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.CrossRef
go back to reference Gao, G. (2009). Probability error in diagnosis: The conjunction fallacy among beginning medical students. Family Medicine, 41, 262–265. Gao, G. (2009). Probability error in diagnosis: The conjunction fallacy among beginning medical students. Family Medicine, 41, 262–265.
go back to reference Harrison, G. W. (1994). Expected utility theory and the experimentalists. In J. D. Hey (Ed.), Experimental economics (pp. 43–76). Springer-Verlag.CrossRef Harrison, G. W. (1994). Expected utility theory and the experimentalists. In J. D. Hey (Ed.), Experimental economics (pp. 43–76). Springer-Verlag.CrossRef
go back to reference Hedges, L. V., & Olkin, I. (2014). Statistical methods for meta-analysis. Academic press. Hedges, L. V., & Olkin, I. (2014). Statistical methods for meta-analysis. Academic press.
go back to reference Hertwig, R., & Ortmann, A. (2001). Experimental practices in economics: A methodological challenge for psychologists? Behavioral and Brain Sciences, 24, 383–403.PubMedCrossRef Hertwig, R., & Ortmann, A. (2001). Experimental practices in economics: A methodological challenge for psychologists? Behavioral and Brain Sciences, 24, 383–403.PubMedCrossRef
go back to reference Kahneman, D. (1973). Attention and effort. Prentice-Hall. Kahneman, D. (1973). Attention and effort. Prentice-Hall.
go back to reference Kahneman, D. (1991). Judgment and decision making: A personal view. Psychological Science, 2, 142–145.CrossRef Kahneman, D. (1991). Judgment and decision making: A personal view. Psychological Science, 2, 142–145.CrossRef
go back to reference Kim, A. J., Lee, D. S., & Anderson, B. A. (2021). The influence of threat on the efficiency of goal-directed attentional control. Psychological Research Psychologische Forschung, 85, 980–986.PubMedCrossRef Kim, A. J., Lee, D. S., & Anderson, B. A. (2021). The influence of threat on the efficiency of goal-directed attentional control. Psychological Research Psychologische Forschung, 85, 980–986.PubMedCrossRef
go back to reference Kontopantelis, E., & Reeves, D. (2012). Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: A comparison between DerSimonian–Laird and restricted maximum likelihood. Statistical Methods in Medical Research, 21, 657–659.PubMedCrossRef Kontopantelis, E., & Reeves, D. (2012). Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: A comparison between DerSimonian–Laird and restricted maximum likelihood. Statistical Methods in Medical Research, 21, 657–659.PubMedCrossRef
go back to reference Larrick, R. P. (2004). Debiasing. In D. Koehler & N. Harvey (Eds.), Blackwell handbook of judgment and decision making (pp. 316–337). Blackwell Publishing.CrossRef Larrick, R. P. (2004). Debiasing. In D. Koehler & N. Harvey (Eds.), Blackwell handbook of judgment and decision making (pp. 316–337). Blackwell Publishing.CrossRef
go back to reference Lawson, M. A., Larrick, R. P., & Soll, J. B. (2020). Comparing fast thinking and slow thinking: The relative benefits of interventions, individual differences, and inferential rules. Judgment and Decision Making, 15, 660–684.CrossRef Lawson, M. A., Larrick, R. P., & Soll, J. B. (2020). Comparing fast thinking and slow thinking: The relative benefits of interventions, individual differences, and inferential rules. Judgment and Decision Making, 15, 660–684.CrossRef
go back to reference Lefebvre, M., Vieider, F. M., & Villeval, M. C. (2011). The ratio bias phenomenon: Fact or artifact? Theory and Decision, 71, 615–641.CrossRef Lefebvre, M., Vieider, F. M., & Villeval, M. C. (2011). The ratio bias phenomenon: Fact or artifact? Theory and Decision, 71, 615–641.CrossRef
go back to reference Levin, I. P., Chapman, D. P., & Johnson, R. D. (1988). Confidence in judgments based on incomplete information: An investigation using both hypothetical and real gambles. Journal of Behavioral Decision Making, 1, 29–41.CrossRef Levin, I. P., Chapman, D. P., & Johnson, R. D. (1988). Confidence in judgments based on incomplete information: An investigation using both hypothetical and real gambles. Journal of Behavioral Decision Making, 1, 29–41.CrossRef
go back to reference McInnes, M. D. F., Moher, D., Thombs, B. D., McGrath, T. A., Bossuyt, P. M., the PRISMA-DTA Group, & Willis, B. H. (2018). Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: The PRISMA-DTA statement. Journal of the American Medical Association, 319, 388–396.PubMedCrossRef McInnes, M. D. F., Moher, D., Thombs, B. D., McGrath, T. A., Bossuyt, P. M., the PRISMA-DTA Group, & Willis, B. H. (2018). Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: The PRISMA-DTA statement. Journal of the American Medical Association, 319, 388–396.PubMedCrossRef
go back to reference Mellers, B., Hertwig, R., & Kahneman, D. (2001). Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychological Science, 12, 269–275.PubMedCrossRef Mellers, B., Hertwig, R., & Kahneman, D. (2001). Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychological Science, 12, 269–275.PubMedCrossRef
go back to reference Navon, D., & Gopher, D. (1979). On the economy of the human processing system. Psychological Review, 86, 214–253.CrossRef Navon, D., & Gopher, D. (1979). On the economy of the human processing system. Psychological Review, 86, 214–253.CrossRef
go back to reference Richter, M., & Gendolla, G. H. E. (2009). The heart contracts to reward: Monetary incentives and preejection period. Psychophysiology, 46, 451–457.PubMedCrossRef Richter, M., & Gendolla, G. H. E. (2009). The heart contracts to reward: Monetary incentives and preejection period. Psychophysiology, 46, 451–457.PubMedCrossRef
go back to reference Sánchez-Meca, J., Marín-Martínez, F., & Chacón-Moscoso, S. (2003). Effect-size indices for dichotomized outcomes in meta-analysis. Psychological Methods, 8, 448–467.PubMedCrossRef Sánchez-Meca, J., Marín-Martínez, F., & Chacón-Moscoso, S. (2003). Effect-size indices for dichotomized outcomes in meta-analysis. Psychological Methods, 8, 448–467.PubMedCrossRef
go back to reference Shafir, E., & LeBoeuf, R. A. (2002). Rationality. Annual Review of Psychology, 53, 419–517.CrossRef Shafir, E., & LeBoeuf, R. A. (2002). Rationality. Annual Review of Psychology, 53, 419–517.CrossRef
go back to reference Shaw, J. D., & Gupta, N. (2015). Let the evidence speak again! Financial incentives are more effective than we thought. Human Resource Management Journal, 25, 281–293.CrossRef Shaw, J. D., & Gupta, N. (2015). Let the evidence speak again! Financial incentives are more effective than we thought. Human Resource Management Journal, 25, 281–293.CrossRef
go back to reference Stolarz-Fantino, S., Fantino, E., Zizzo, D. J., & Wen, J. (2003). The conjunction effect: New evidence for robustness. American Journal of Psychology, 116, 15–34.PubMedCrossRef Stolarz-Fantino, S., Fantino, E., Zizzo, D. J., & Wen, J. (2003). The conjunction effect: New evidence for robustness. American Journal of Psychology, 116, 15–34.PubMedCrossRef
go back to reference Szollosi, A., Bago, B., Szaszi, B., & Aczel, B. (2017). Exploring the determinants of confidence in the bat-and-ball problem. Acta Psychologica, 180, 1–7.PubMedCrossRef Szollosi, A., Bago, B., Szaszi, B., & Aczel, B. (2017). Exploring the determinants of confidence in the bat-and-ball problem. Acta Psychologica, 180, 1–7.PubMedCrossRef
go back to reference Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Yale University Press. Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Yale University Press.
go back to reference Travers, E., Rolison, J. J., & Feeney, A. (2016). The time course of conflict on the cognitive reflection test. Cognition, 150, 109–118.PubMedCrossRef Travers, E., Rolison, J. J., & Feeney, A. (2016). The time course of conflict on the cognitive reflection test. Cognition, 150, 109–118.PubMedCrossRef
go back to reference Tversky, A., & Kahneman, D. (1982). Judgments of and by representativeness. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 84–100). Cambridge University Press.CrossRef Tversky, A., & Kahneman, D. (1982). Judgments of and by representativeness. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 84–100). Cambridge University Press.CrossRef
go back to reference Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90, 293–315.CrossRef Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90, 293–315.CrossRef
go back to reference Viechtbauer, W. (2010). Conducting meta-analyses in R with the metaphor package. Journal of Statistical Software, 36, 1–48.CrossRef Viechtbauer, W. (2010). Conducting meta-analyses in R with the metaphor package. Journal of Statistical Software, 36, 1–48.CrossRef
go back to reference von Hippel, P. T. (2015). The heterogeneity statistic I2 can be biased in small meta-analyses. BMC Medical Research Methodology, 15, 35.CrossRef von Hippel, P. T. (2015). The heterogeneity statistic I2 can be biased in small meta-analyses. BMC Medical Research Methodology, 15, 35.CrossRef
go back to reference Wright, R. A. (1998). Ability perception and cardiovascular response to behavioral challenge. In M. Kofta, G. Weary, & G. Sedek (Eds.), Control in action: Cognitive and motivational mechanisms (pp. 197–232). Plenum Press. Wright, R. A. (1998). Ability perception and cardiovascular response to behavioral challenge. In M. Kofta, G. Weary, & G. Sedek (Eds.), Control in action: Cognitive and motivational mechanisms (pp. 197–232). Plenum Press.
go back to reference Wright, W. F., & Anderson, U. (1989). Effects of situation familiarity and financial incentives on use of the anchoring and adjustment heuristic for probability assessment. Organizational Behavior and Human Decision Processes, 44, 68–82.CrossRef Wright, W. F., & Anderson, U. (1989). Effects of situation familiarity and financial incentives on use of the anchoring and adjustment heuristic for probability assessment. Organizational Behavior and Human Decision Processes, 44, 68–82.CrossRef
go back to reference Xue, G., Lu, Z., Levin, I. P., Weller, J. A., Li, X., & Bechara, A. (2009). Functional dissociations of risk and reward processing in the medial prefrontal cortex. Cerebral Cortex, 19, 1019–1027.PubMedCrossRef Xue, G., Lu, Z., Levin, I. P., Weller, J. A., Li, X., & Bechara, A. (2009). Functional dissociations of risk and reward processing in the medial prefrontal cortex. Cerebral Cortex, 19, 1019–1027.PubMedCrossRef
go back to reference Yechiam, E., & Hochman, G. (2013). Losses as modulators of attention: Review and analysis of the unique effects of losses over gains. Psychological Bulletin, 139, 497–518.PubMedCrossRef Yechiam, E., & Hochman, G. (2013). Losses as modulators of attention: Review and analysis of the unique effects of losses over gains. Psychological Bulletin, 139, 497–518.PubMedCrossRef
go back to reference Yechiam, E., & Hochman, G. (2014). Loss attention in a dual task setting. Psychological Science, 25, 494–502.PubMedCrossRef Yechiam, E., & Hochman, G. (2014). Loss attention in a dual task setting. Psychological Science, 25, 494–502.PubMedCrossRef
go back to reference Yechiam, E., & Zeif, D. (2022). Revisiting the effect of incentivization on cognitive reflection: A meta-analysis. Journal of Behavioral Decision Making., 36, e2286.CrossRef Yechiam, E., & Zeif, D. (2022). Revisiting the effect of incentivization on cognitive reflection: A meta-analysis. Journal of Behavioral Decision Making., 36, e2286.CrossRef
go back to reference Young, M. S., & Stanton, N. A. (2002). Malleable attentional resources theory: A new explanation for the effects of mental underload on performance. Human Factors, 44, 365–375.PubMedCrossRef Young, M. S., & Stanton, N. A. (2002). Malleable attentional resources theory: A new explanation for the effects of mental underload on performance. Human Factors, 44, 365–375.PubMedCrossRef
go back to reference Zeif, D. (2022). The effect of strategies of attention on judgment and decision-making performance. Unpublished doctoral dissertation. Technion – Israel Institute of Technology. Zeif, D. (2022). The effect of strategies of attention on judgment and decision-making performance. Unpublished doctoral dissertation. Technion – Israel Institute of Technology.
go back to reference Zizzo, J. Z., Stolarz-Fantino, S., Wen, J., & Fantino, E. (2000). A violation of the monotonicity axiom: Experimental evidence on the conjunction fallacy. Journal of Economic Behavior & Organization, 41, 263–276.CrossRef Zizzo, J. Z., Stolarz-Fantino, S., Wen, J., & Fantino, E. (2000). A violation of the monotonicity axiom: Experimental evidence on the conjunction fallacy. Journal of Economic Behavior & Organization, 41, 263–276.CrossRef
Metagegevens
Titel
The effect of incentivization on the conjunction fallacy in judgments: a meta-analysis
Auteurs
Eldad Yechiam
Dana Zeif
Publicatiedatum
25-05-2023
Uitgeverij
Springer Berlin Heidelberg
Gepubliceerd in
Psychological Research / Uitgave 8/2023
Print ISSN: 0340-0727
Elektronisch ISSN: 1430-2772
DOI
https://doi.org/10.1007/s00426-023-01837-5