Elsevier

Behavioural Brain Research

Volume 225, Issue 2, 1 December 2011, Pages 396-404
Behavioural Brain Research

Research report
The feedback-related negativity is modulated by feedback probability in observational learning

https://doi.org/10.1016/j.bbr.2011.07.059Get rights and content

Abstract

The feedback-related negativity (FRN), an event-related potentials (ERPs) component reflecting activity of the anterior cingulate cortex (ACC), has been shown to be modulated by feedback expectancy following active choices in feedback-based learning tasks. A general reduction of FRN amplitude has been described in observational feedback learning, raising the question whether FRN amplitude is modulated in a similar way in this type of learning. The present study investigated whether the FRN and the P300 – a second ERP component related to feedback processing – are modulated by feedback probability in observational learning. Thirty-two subjects participated in the experiment. They observed a virtual person choosing between two symbols and receiving positive or negative feedback. Learning about stimulus-specific feedback probabilities was assessed in active test trials without feedback. In addition, the bias to learn from positive or negative feedback and – in a subsample of 17 subjects – empathy scores were obtained. General FRN and P300 modulations by feedback probability were found across all subjects. Only for the FRN in learners, an interaction between probability and valence was observed. Larger FRN amplitudes for negative relative to positive feedback only emerged for the lowest outcome probability. The results show that feedback expectancy modulates FRN amplitude also in observational learning, suggesting a similar ACC function as in active learning. On the other hand, the modulation is only seen for very low feedback expectancy, which suggests that brain regions other than those of the reward system contribute to feedback processing in an observation setting.

Highlights

► FRN and P300 modulation by feedback probability in observational learning. ► Interaction between valence and probability on FRN in learners but not non-learners. ► Valence effect on FRN in learners only for most unexpected feedback.

Introduction

Learning the association between actions and their consequences enables us to adapt to the environment by increasing the frequency of rewarded and reducing the frequency of non-rewarded behaviour. In their reinforcement learning theory, Holroyd and Coles [1] suggest that this form of learning is driven by an error-signal generated in the dopaminergic midbrain (DA) and projected to the anterior cingulate cortex (ACC): unexpected reward (positive prediction error) leads to an increase of DA activity, whereas it is attenuated in case of expected but undelivered reward (negative prediction error). This pattern was shown in primates [2] and – by means of functional magnetic resonance imaging (fMRI) – in humans [3]. Via inhibitory pathways from the DA through the basal ganglia [see 1], the activity of the ACC is reduced for positive but enhanced for negative prediction errors. In line with the reinforcement-learning theory, the feedback-related negativity (FRN), an event-related potentials (ERP) component resembling the error-related negativity (ERN) observed following performance errors [4], [5] and reflecting activity of the ACC [6], [7], was shown to be increased both when subjects were informed about upcoming negative feedback [8] and when they received negative feedback for their performance [9], [10], [11], [12], [13], especially when subjects expected [7], [14], [15] or predicted [16] positive feedback.

The mismatch between predicted and received feedback is only one of multiple factors influencing ACC activity. Another important factor is control over the events leading to feedback. In this respect, the basal ganglia appear to play a modulating role: while neural activity in both the ventral and dorsal striatum codes prediction error magnitude when subjects learn from feedback following their own actions, only the ventral part is involved if subjects perceive the feedback to be independent from their behaviour [17]. In line with this view, ACC activity – as indicated by the FRN – is reduced when subjects observe random choices made by a computer [11].

Flexible adaptation to changing feedback contingencies cannot only be accomplished in active learning, i.e. by choosing between different behavioural options oneself. An alternative is to observe the behaviour and accompanying feedback in other persons. This strategy can be considered as advantageous in evolutionary terms, because the risk of receiving negative feedback oneself is minimized. Reduced FRN amplitudes in response to negative relative to positive feedback stimuli were found in subjects who learned by observing the choices of another person when compared to subjects who learned actively from their own choices [18]. These differences in activation raise the question whether the functional contributions of the ACC are similar for both active and observational feedback-based learning. For active learning situations, research in recent years addressed the question whether FRN amplitude is modulated by reward expectancy. As Holroyd and Coles [1] hypothesized that the FRN reflects a DA-driven error signal which is used as a teaching signal in feedback-based learning, the amplitude of the FRN should mirror the relative size of the (negative) prediction error. In early studies with different reward probabilities, FRN amplitude did not appear to be modulated by the degree of deviation of obtained and expected outcome, suggesting a binary feedback coding in terms of good vs. bad [19]. However, more recent work took reward expectations more directly into account, and FRN amplitude was indeed found to be more pronounced the more unexpected negative feedback was [7], [16], [20], [21]. In addition, Bellebaum et al. [22] demonstrated that the FRN was modulated by reward expectations referring to different reward magnitudes. If the DA system also provides a teaching signal in observational learning, FRN amplitude should be modulated by reward expectancy. To date, ACC activity as indicated by the FRN has only rarely been studied in observational learning. As outlined above, the FRN was found to be generally reduced relative to situations involving active choices, but the question, whether the FRN in observational learning also reflects different degrees of reward expectation, was either not addressed [23] or could not be resolved unequivocally [18].

In the present study we hypothesized that ACC activity – despite the general reduction in observational learning – is modulated by feedback expectancy in a purely observational learning paradigm, i.e. when subjects have no control over choices leading to positive and negative feedback but merely observe choices made by another person. To test this hypothesis, we modified a probabilistic feedback-based learning task introduced by Frank et al. [24] to investigate the influence of feedback probability on the FRN in an observational learning setting. As in previous studies which administered variants of this task, we used performance feedback instead of monetary reward, which has been shown to reliably elicit an FRN [25].

The task at hand also allowed the assessment of whether individual subjects showed a bias to learn better from positive or negative feedback and therefore the analysis of FRN differences between positive and negative observational learners, as they were found for active learning [25]. Furthermore, exploratory analyses in a subset of our sample aimed at exploring the relationship between empathy scores on the one hand and performance measures and FRN amplitude on the other hand. In previous studies correlations between error processing and empathy have been described, both for active error commitment and observed errors [26], [27]. However, this association has not as yet been investigated in contexts where the participants could actually benefit from observing another person's behaviour to improve their own performance on a feedback-based learning task. It is plausible to assume that it is in situations like these that empathic perspective taking abilities are of particular relevance.

Another ERP component which has been related to feedback processing is the P300. However, results from previous studies make it difficult to draw firm conclusions on P300 involvement in feedback processing: effects of both positive [16], [19], [28] and negative [25] feedback on the P300 as well as the absence of feedback valence effects [29], [30], [31] have been reported. More consistently, larger P300 amplitudes following unexpected than expected outcomes could be shown [7], [31]; data from one study by Bellebaum et al. [28] suggest that this effect is more pronounced for positive feedback. As, however, it is as yet unclear whether these P300 modulations are restricted to active rather than observational learning, the P300 was additionally analyzed in the present study.

Section snippets

Subjects

32 healthy adult volunteers (19 female) participated in the study. All participants were students of the Ruhr University Bochum and had normal or corrected-to-normal vision. Subjects gave written informed consent prior to participation. The study conforms to the Declaration of Helsinki and received ethical clearance by the Ethics Board of the Faculty of Psychology of the Ruhr University Bochum, Germany.

Stimuli and task

Participants were comfortably seated approximately 70 cm in front of a computer monitor and

Behavioural data

Depending on task performance (see Sections 2.2 Stimuli and task, 2.6.1 Behavioural data for criteria), subjects were assigned to one of two groups—either “learners” or “non-learners”. 19 subjects were classified as learners (10 women and 9 men; mean age [M] = 23.5 years, standard deviation [SD] = 4.2 years; correct choices on active A/B trials M = 88.23%; SD = 10.75%); the group of non-learners consisted of the remaining 13 subjects (9 women and 4 men; M = 23.5 years, SD = 4.1 years; correct choices on

Discussion

In the present study, we investigated whether the ACC response to positive and negative feedback – as reflected by the FRN – is modulated by outcome expectancy in an observational learning task. Subjects were required to observe choices of a virtual person and the accompanying outcomes (correct vs. incorrect) while brain activity was assessed using EEG. Learning was measured in blocks of test trials in which subjects had to actively choose between different options without receiving feedback.

We

Acknowledgements

We thank the Ministry of Innovation, Science and Research of the Federal State of Nordrhein-Westfalen, Germany, for supporting this research (Ministerium für Innovation, Wissenschaft und Forschung des Landes Nordrhein-Westfalen; MIWF; grant number 334-4). We also thank Christof Leibfacher and Daniel Schemberg for their assistance in data acquisition.

References (53)

  • U. Sailer et al.

    Effects of learning on feedback-related brain potentials in a decision-making task

    Brain Res

    (2010)
  • S.J. Luck et al.

    Event-related potential studies of attention

    Trends Cogn Sci

    (2000)
  • K.C. Dickerson et al.

    Parallel contributions of distinct human memory systems during probabilistic learning

    Neuroimage

    (2011)
  • C.B. Holroyd et al.

    The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity

    Psychol Rev

    (2002)
  • W. Schultz

    Multiple reward signals in the brain

    Nat Rev Neurosci

    (2000)
  • B. Knutson et al.

    Functional magnetic resonance imaging of reward prediction

    Curr Opin Neurol

    (2005)
  • W.J. Gehring et al.

    A neural system for error-detection and compensation

    Psychol Sci

    (1993)
  • M. Falkenstein et al.

    Effects of errors in choice reaction tasks on the ERP under focused and divided attention

  • W.J. Gehring et al.

    The medial frontal cortex and the rapid processing of monetary gains and losses

    Science

    (2002)
  • C. Bellebaum et al.

    Learning-related changes in reward expectancy are reflected in the feedback-related negativity

    Eur J Neurosci

    (2008)
  • J.P. Dunning et al.

    Error-related negativities elicited by monetary loss and cues that predict loss

    Neuroreport

    (2007)
  • W.H. Miltner et al.

    Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a generic neural system for error detection

    J Cogn Neurosci

    (1997)
  • S. Nieuwenhuis et al.

    Sensitivity of electrophysiological activity from medial frontal cortex to utilitarian and performance feedback

    Cereb Cortex

    (2004)
  • N. Yeung et al.

    ERP correlates of feedback and reward processing in the presence and absence of response choice

    Cereb Cortex

    (2005)
  • A. Yasuda et al.

    Error-related negativity reflects detection of negative reward prediction error

    Neuroreport

    (2004)
  • C.B. Holroyd et al.

    Errors in reward prediction are reflected in the event-related brain potential

    Neuroreport

    (2003)
  • Cited by (31)

    • Beauty premium: Event-related potentials evidence of how physical attractiveness matters in online peer-to-peer lending

      2017, Neuroscience Letters
      Citation Excerpt :

      FRN is a frontal-central negative deflection that peaks at approximately 200–350 ms after presentation of feedback, and it shows maximal amplitude over medial frontal scalp locations [13,24,31]. The FRN amplitude is larger after negative feedback, such as incorrect response, game failure, or monetary loss [14–17]. Evidence from ERP source localization and fMRI studies showed that FRN is generated in the anterior cingulate cortex (ACC) [5,13,28].

    • The better, the bigger: The effect of graded positive performance feedback on the reward positivity

      2016, Biological Psychology
      Citation Excerpt :

      This is why we oversampled in the present experiment to reach a sufficiently large sample size. As summarized above, reward prediction errors, as reflected by reward positivity, are influenced by at least three factors: feedback probability (frequency), magnitude, and valence (Holroyd, 2004; Kobza, Thoma, Daum, & Bellebaum, 2011). Thus, frequency and valence have to be controlled when examining the effects of magnitude.

    • Performance monitoring and empathy during active and observational learning in patients with major depression

      2015, Biological Psychology
      Citation Excerpt :

      This effect should be more pronounced for observational learning, as this condition ought to be particularly affected by empathy impairment. Based on previous findings (Kobza et al., 2011; Rak et al., 2013), we expected negative associations between reward learning performance/ERPs and trait empathy. Twenty-three patients with MDD and 18 healthy control (HC) participants were recruited.

    • Probabilistic reward learning in adults with Attention Deficit Hyperactivity Disorder-An electrophysiological study

      2015, Psychiatry Research
      Citation Excerpt :

      It was defined as the amplitude difference between the most negative peak in the time window between 200 and 350 ms after feedback presentation and the preceding most positive peak with a minimum latency of 130 ms. The positive peak preceding the FRN was analyzed separately as P200, at electrode site FCz. The P300 was analyzed at electrode Pz, defined as the peak amplitude in the time window between 300 and 500 ms. Finally, the N100, scored as the maximum negative peak amplitude between 50 and 150 ms after feedback onset at electrode site FCz, was analyzed as a measure of early bottom-up stimulus processing and visual attention (Mangun et al., 1993; Vogel and Luck, 2000), which has been found to be modulated in observational feedback learning (Kobza et al., 2011; Rak et al., 2013). Due to their probabilistic nature, the learning tasks applied in the present study appear to be more adequate for the assessment of feedback- than response-locked ERPs, as there are no clear response-feedback associations.

    View all citing articles on Scopus
    View full text