Elsevier

Neuropsychologia

Volume 48, Issue 11, September 2010, Pages 3343-3350
Neuropsychologia

It is less than you expected: The feedback-related negativity reflects violations of reward magnitude expectations

https://doi.org/10.1016/j.neuropsychologia.2010.07.023Get rights and content

Abstract

The anterior cingulate cortex (ACC) is involved in performance monitoring and in learning from performance feedback. Recent research suggests that the feedback-related negativity (FRN), an event-related potentials (ERP) component reflecting neural activity in the ACC, codes the size of a negative prediction error when reward probabilities are varied. There is as yet no clear evidence that the FRN is also sensitive to violations of reward magnitude expectations. In the present study, 20 healthy young subjects engaged in a learning task in which a coin had to be found on each trial. The value of the coin (the potential reward magnitude) was varied from trial to trial and amounted to 5 cent, 20 cent or 50 cent. Analysis of ERPs revealed that FRN amplitude differences between reward and non-reward were significantly modulated by (potential) reward magnitude. This effect was driven by the neural response to non-reward: the larger the potential reward, the larger was the FRN amplitude in response to non-reward. In contrast, the P300 was larger for positive outcomes and showed an effect of (potential) reward magnitude independent of valence. Together with evidence from previous studies, these results show that the FRN codes negative prediction errors in the context of varying reward probabilities and magnitudes. The findings are in line with recent results based on functional neuroimaging and lend further support to the idea of a key role of the ACC in the integration of information on different aspects of performance outcomes.

Research highlights

ā–¶ The FRN codes deviations of reward magnitude expectations. ā–¶ The FRN in response to no reward is larger for higher potential rewards. ā–¶ This effect does not depend on insight into reward probability. ā–¶ In learners the effect correlates with performance accuracy/reward expectation.

Introduction

Learning the association between behavioural responses and their consequences is a prerequisite for successful adaptation to varying environmental demands. In monkeys, single dopaminergic neurons in the midbrain have been shown to code positive and negative errors in reward prediction (Schultz et al., 1997, Schultz and Dickinson, 2000, Tobler et al., 2005). In estimating the overall reward associated with specific responses or events, the expected value is relevant which takes into account valence, magnitude and probability of an outcome. In humans, it has been shown that these different components of expected value are coded in a distributed set of brain regions along the mesolimbic pathway (Knutson, Taylor, Kaufman, Peterson, & Glover, 2005). Apart from the basal ganglia, which are critically involved in learning from monetary feedback (Bellebaum, Koch, Schwarz, & Daum, 2008), a key role in this context is assigned to the anterior cingulate cortex (ACC) which integrates information about outcome valence, probability and magnitude (Knutson & Cooper, 2005).

Evidence obtained from event-related potentials in human subjects strongly suggests that the ACC is implicated in performance monitoring and action selection. Performance errors in human subjects are accompanied by a negative deflection in the event-related potential (ERP), the error related negativity (ERN) or Ne, which peaks at about 100Ā ms after the motor response and is generated in the ACC (Dehaene et al., 1994, Falkenstein et al., 1990, Gehring et al., 1993). A similar component is elicited by performance feedback in situations when stimulus response associations are unknown to the subject. This component is termed feedback-related negativity (FRN) or feedback ERN which is also generated in the ACC, as revealed by source localization techniques (Bellebaum and Daum, 2008, Gehring and Willoughby, 2002). The FRN is typically more pronounced for negative compared to positive feedback (Miltner et al., 1997, Nieuwenhuis et al., 2004) and for active learning compared to learning by observation (Bellebaum, Kobza, Thiele, & Daum, 2010). Evidence from functional neuroimaging supports the results of ERP source localisation. The dorsal ACC is activated when subjects commit performance errors and when external signals give error feedback (Holroyd, Nieuwenhuis, et al., 2004).

Holroyd and Coles (2002) proposed a reinforcement-learning (RL) theory of the FRN suggesting that ACC activity as reflected by the FRN mirrors the activity of the midbrain dopamine (DA) system. According to this account, reward prediction errors should be coded in the amplitude of the FRN. Despite the finding that a significant FRN is only observed in response to unexpected negative outcomes (Holroyd, Nieuwenhuis, Yeung, & Cohen, 2003), early studies did not find evidence for prediction error coding in the ACC. Similarly sized FRN amplitudes for different outcome probabilities were reported, suggesting that the FRN reflects a binary system of performance monitoring distinguishing between outcomes which are better or worse than expected, without coding the magnitude of the prediction error (Hajcak, Holroyd, Moser, & Simons, 2005). One shortcoming of these early studies, however, relates to the fact that reward expectations were not taken into account. As the prediction error is conceptualized as the deviation of the actual from the expected outcome, assessing reward expectations of the subjects is of central importance. When subjectsā€™ expectations were accounted for in a follow-up study, an increase in FRN amplitude with increasing prediction error magnitude was indeed observed (Hajcak, Moser, Holroyd, & Simons, 2007). Again, prediction errors were manipulated by varying reward probability. Several other studies corroborated this finding (Bellebaum and Daum, 2008, Eppinger et al., 2008, Holroyd et al., 2009), supporting the RL theory of the FRN.

As outlined above, the expected reward value is a function of reward probability and magnitude. If the ACC coded the deviation of actually obtained from expected outcomes, the FRN would be expected to be sensitive to deviations from expected reward magnitude. There is as yet no clear evidence for such a modulation. When subjects could choose between risky and non-risky, i.e. high or low magnitude, outcomes without knowing whether their choices resulted in a gain or loss, the pattern of outcome-related FRN amplitudes reflected binary processing. Losses were associated with larger amplitude FRNs, but no differences between large and small losses (or gains) were observed (Yeung & Sanfey, 2004). The authors concluded that the monitoring system located in the ACC is involved in a fast and coarse evaluation of ongoing events leading to a simple distinction between good and bad outcomes. Similar results emerged in other studies, corroborating the notion that the ACC evaluates outcomes in a binary manner (Hajcak et al., 2006, Holroyd et al., 2004a, Holroyd et al., 2006, Sato et al., 2005). More recent studies provided first hints that reward magnitude information might be processed in the FRN time window (Goyer et al., 2008, Wu and Zhou, 2009), but these findings could not clearly be assigned to the processing of reward magnitude per se or to prediction errors. Moreover, FRN has been reported to show sensitivity to magnitude in correspondence to risk-avoidant behavior (Polezzi, Sartori, Rumiati, Vidotto, & Daum, 2009).

The present study was motivated by the assumption that any violation of reward expectations is reflected in ACC activity and thus in FRN amplitude, especially for negative outcomes. We hypothesized that higher expected monetary rewards should yield larger FRN amplitudes in response to non-rewarding feedback. A task previously applied to assess the effect of reward probabilities on feedback processing was modified to induce expectations on reward magnitude (Bellebaum and Daum, 2008, Bellebaum et al., 2010). In contrast to previous studies examining the effect of reward magnitude, subjects could maximize the amount of money they earned by learning an explicit rule determining reward probability. Recent evidence suggests that FRN amplitude modulations are stronger, if action outcome contingencies can be learned (Holroyd et al., 2009). A further aim of the study was to add new evidence on the coding of outcome magnitude and valence in the P300 ERP component. Previous work on this issue has provided inconsistent results. Some studies suggested that the P300 codes reward magnitude information without being sensitive to outcome valence (Sato et al., 2005, Yeung and Sanfey, 2004). Other studies reported larger P300 amplitudes for gains than for losses (Hajcak et al., 2007) and very recent work suggests that the P300 codes both outcome valence and magnitude (Wu & Zhou, 2009).

Section snippets

Subjects

Twenty healthy subjects with a mean age of 23.6 years (SDĀ =Ā 4.2) participated in this study (12 women). All subjects were students at the Ruhr University of Bochum, Germany. They were all right-handed and had normal or corrected-to-normal vision. The study was approved by the Ethics Committee of the Faculty of Medicine of the Ruhr-University of Bochum and all participants gave written informed consent.

The learning task

A variant of a previously administered learning task was used (Bellebaum and Daum, 2008,

Accuracy

Fig. 2A shows the average number of correct responses for all 20 subjects in six blocks of trials for the three reward magnitudes. Repeated-measures ANOVA with the factors BLOCK (1ā€“6) and RM (5 cent, 20 cent and 50 cent) yielded a significant linear increase in the number of correct responses over the six blocks (linear trend: F(1,19)Ā =Ā 5.740; pĀ =Ā .027). Neither the main effect of RM nor the RM x BLOCK interaction reached significance (both pĀ >Ā .709).

Reaction times

Mean reaction times for the three potential reward

Discussion

According to the RL theory, the FRN reflects the activity of a reinforcement-learning system and codes negative errors in reward prediction (Holroyd & Coles, 2002), mirroring the activity of single dopaminergic neurons in the midbrain (Schultz et al., 1997). The results of earlier studies suggested that the FRN codes outcomes in a binary manner, i.e. a distinction between good and bad or better and worse than expected outcomes, without reflecting the degree of deviation from expectancy (Hajcak

Acknowledgement

We thank the Ministry of Innovation, Science, Research and Technology of the federal state of Nordrhein-Westfalen, Germany, for supporting this research within the young researcher programme (Ministerium fĆ¼r Innovation, Wissenschaft, Forschung und Technologie (MIWFT) des Landes Nordrhein-Westfalen).

References (45)

  • C. Bellebaum et al.

    Learning-related changes in reward expectancy are reflected in the feedback-related negativity

    European Journal of Neuroscience

    (2008)
  • C. Bellebaum et al.

    Focal basal ganglia lesions are associated with impairments in reward-based reversal learning

    Brain

    (2008)
  • C. Bellebaum et al.

    It was not MY fault: Event related brain potentials in active and observational learning from feedback

    Cerebral Cortex

    (2010)
  • S. Dehaene et al.

    Localization of a neural system for error detection and compensation

    Psychological Science

    (1994)
  • N.U. Dosenbach et al.

    Distinct brain networks for adaptive and stable task control in humans

    Proceedings of the National Academy of Sciences U.S.A.

    (2007)
  • M. Falkenstein et al.

    Effects of errors in choice reaction tasks on the ERP under focused and divided attention

  • W.J. Gehring et al.

    A neural system for error detection and compensation

    Psychological Science

    (1993)
  • W.J. Gehring et al.

    The medial frontal cortex and the rapid processing of monetary gains and losses

    Science

    (2002)
  • J.P. Goyer et al.

    Rapid electrophysiological brain responses are influenced by both valence and magnitude of monetary rewards

    Journal of Cognitive Neuroscience

    (2008)
  • G. Hajcak et al.

    Brain potentials associated with expected and unexpected good and bad outcomes

    Psychophysiology

    (2005)
  • G. Hajcak et al.

    To err is autonomic: Error-related brain potentials, ANS activity, and post-error compensatory behavior

    Psychophysiology

    (2003)
  • G. Hajcak et al.

    It's worse than you thought: The feedback negativity and violations of reward prediction in gambling tasks

    Psychophysiology

    (2007)
  • Cited by (157)

    • Age-related differences in ERP correlates of value-based decision making

      2023, Neurobiology of Aging
      Citation Excerpt :

      Findings on the FB-P3 have been less consistent than for the FRN (San Martin, 2012). While some studies focused on enlarged FB-P3 in response to larger sized outcomes regardless of valence (Bellebaum, Polezzi, & Daum, 2010; Yeung & Sanfey, 2004), some others found that the FB-P3 is enlarged more in response to unexpected outcomes rather than outcome size per se and thus reflects expectancy violation (Donaldson et al., 2016; Stewardson and Sambrook, 2020; Wu & Zhou, 2009). Distinct from the FB-P3, the following LPP is a more sustained response that shows sensitivity to both outcome valence (e.g., positive vs. negative feedback) as well as expectancy (e.g., positive RPE and negative RPE), but not to their interaction (Donaldson et al., 2016).

    View all citing articles on Scopus
    View full text