Review
Reward expectation alters learning and memory: The impact of the amygdala on appetitive-driven behaviors

https://doi.org/10.1016/j.bbr.2008.10.028Get rights and content

Abstract

The capacity to seek and obtain rewards is essential for survival. Pavlovian conditioning is one mechanism by which organisms develop predictions about rewards and such anticipatory or expectancy states enable successful behavioral adaptations to environmental demands. Reward expectancies have both affective/motivational and discriminative properties that allow for the modulation of instrumental goal-directed behavior. Recent data provide evidence that different cognitive strategies (cue-outcome associations) and neural systems (amygdala) are used when subjects are trained under conditions that allow Pavlovian-induced reward expectancies to guide instrumental behavioral choices. Furthermore, it has been demonstrated that impairments typically observed in a number of brain-damaged models are alleviated or eliminated by embedding unique reward expectancies into learning/memory tasks. These results suggest that Pavlovian-induced reward expectancies can change both behavioral and brain processes.

Section snippets

Pavlovian associations produce specific reward expectancies that can modulate instrumental behavior

Expectations of reward can have powerful effects on learning and performance. Learning to anticipate a reward can occur by associating a discrete stimulus (S–O; Pavlovian conditioning) and/or an action (R–O; instrumental conditioning) with an outcome (see Fig. 1A). A predictive cue such as a conditioned stimulus (CS) or discriminative stimulus (SD) can enter into associations with either the general motivational properties of the reward or the sensory specific properties of reward (see [23],

Reward expectancies produced by the DOP can change behavioral success after brain damage

Our laboratory [98], [95], [94], [84], [72] proposed that the DOP allows subjects to use reward expectancies to solve the task and this cognitive process activates different brain regions compared to when a subject must rely on retrospective recall of information (as must be done under NOP conditions) to solve a task. The reliance on reward-related processes activates a memory system that is different, structurally and associatively, from the system commonly used to solve conditional

Functional evidence for the role of the amygdala in Pavlovian associative processes

There is an accumulation of evidence from a number of studies that the amygdala is critical for expectancy of both aversive and appetitive outcomes. In all species examined, the amygdala plays an important role in Pavlovian conditioning and significant progress has been made in understanding the neural mechanisms of Pavlovian-conditioned fear. For example, functional neuroimaging studies indicate an increased hemodynamic response in the human amygdala during presentation of CSs such as faces,

Specific conditions evoke the BLA in the development and maintenance of reward expectancies

There is evidence from a number of procedures that the amydala is critical for the development of reward expectancies. As describe in detail below, lesions of the BLA produce deficits in Pavlovian second-order conditioning, the reward specific effects of reinforcer devaluation, the differential outcomes effect, and reward-specific Pavlovian-to-instrumental transfer (PIT; [11], [36], [84]). However, lesions to the BLA do not affect acquisition of approach or orientation behaviors elicited by a

Summary

Reward expectancies can be evoked to direct behavior under a range of circumstances. Environmental conditions set the stage for the type of associative relationships that an organism will form between stimuli, responses and outcomes. Data suggest that several interconnected brain structures – the basolateral amygdala (BLA), the orbitofrontal cortex (OFC), and the nucleus accumbens (NA) are part of the neural circuitry that is integral for cues that predict features of reward and goal-directed

References (119)

  • J. Iwata et al.

    Cardiovascular responses elicited by stimulation of neurons in the central amygdaloid nucleus in awake but not anesthetized rats resemble conditioned emotional responses

    Brain Res

    (1987)
  • B.S. Kapp et al.

    Cardiovascular responses elicited by electrical stimulation of the amygdala central nucleus in the rabbit

    Brain Res

    (1982)
  • J.M. Kruse et al.

    Pavlovian CS effects upon instrumental choice behavior are reinforcer specific

    Learn Motiv

    (1983)
  • P.J. Langlais et al.

    Thiamine deficiency in rats produces cognitive and memory deficits on spatial tasks that correlate with tissue loss in diencephalon, cortex and white matter

    Behav Brain Res

    (1995 Apr)
  • J.E. LeDoux et al.

    Overlapping projections to the amygdala and striatum from auditory processing areas of the thalamus and cortex

    Neurosci Lett

    (1991)
  • R.G. Mair et al.

    Thiamine deficiency depletes cortical norepinephrine and impairs learning processes in the rat

    Brain Res

    (1985)
  • S. Maren

    Synaptic mechanisms of associative memory in the amygdala

    Neuron

    (2005)
  • F. Mascagni et al.

    Corticoamygdaloid and corticocortical projections of the rat temporal cortex: a Phaseolus vulgaris leucoagglutinin study

    Neurosci

    (1993)
  • A.J. McDonald

    Cortical pathways to the mammalian amygdala

    Prog Neurobiol

    (1998)
  • S. Milanovic et al.

    Production of the Fos protein after contextual fear conditioning of C57BL/6N mice

    Brain Res

    (1998 Feb 16)
  • G.B. Peterson et al.

    Effects of altering outcome expectancies on Pigeons’ delayed conditional discrimination performance

    Learn Motiv

    (1980)
  • A. Pitkänen et al.

    Organization of intra-amygdaloid circuitries in the rat: an emerging framework for understanding functions of the amygdala

    Trends Neurosci

    (1997)
  • W.E. Pratt et al.

    Neurons in rat medial prefrontal cortex show anticipatory rate changes to predictable differential rewards in a spatial memory task

    Behav Brain Res

    (2001)
  • G.J. Quirk et al.

    Fear conditioning enhances different temporal components of tone-evoked spike trains in auditory cortex and lateral amygdala

    Neuron

    (1997)
  • G.J. Quirk et al.

    Fear conditioning enhances short-latency auditory responses of lateral amygdala neurons: parallel recordings in the freely behaving rat

    Neuron

    (1995)
  • D.R. Ramirez et al.

    The role of the GABAA agonist muscimol on memory performance: Reward contingencies determine the nature of the deficit

    Neurobiol Learn Mem

    (2005)
  • S.M. Rodrigues et al.

    Molecular mechanisms underlying emotional learning and memory in the lateral amygdala

    Neuron

    (2004)
  • F. Roullet et al.

    Cue valence representation studied by Fos immunocytochemistry after acquisition of a discrimination learning task

    Brain Res Bull

    (2004)
  • S. Royer et al.

    Bidirectional synaptic plasticity in intercalated amygdala neurons and the extinction of conditioned fear responses

    Neurosci

    (2002)
  • L.M. Savage et al.

    The effect of hippocampal lesions on learning, memory and reward expectancies

    Neurobiol Learn Mem

    (2004)
  • L.M. Savage et al.

    Rats exposed to acute pyrithiamine-induced thiamine deficiency are more sensitive to the amnestic effects of scopolamine and MK-801: examination of working memory, response selection and reinforcement contingencies

    Behav Brain Res

    (1999)
  • P. Apicella et al.

    Neuronal activity in monkey striatum related to the expectation of predictable environmental events

    J Neurophysiol

    (1992)
  • B.W. Balleine et al.

    The effect of lesions of the basolateral amygdala on instrumental conditioning

    J Neurosci

    (2003)
  • C.A. Barnes et al.

    Behavioral and neurophysiological examples of functional sparing in senescent rats

    Canadian J Psychology

    (1987)
  • M.G. Baxter et al.

    The amygdala and reward

    Nat Rev Neurosci

    (2002)
  • A. Bechara et al.

    Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans

    Science

    (1995)
  • C.H. Beck et al.

    Conditioned fear-induced changes in behavior and in the expression of the immediate early gene c-fos: with and without diazepam pretreatment

    J Neurosci

    (1995)
  • H.T. Blair et al.

    Synaptic plasticity in the lateral amygdala: a cellular hypothesis of fear conditioning

    Learn Mem

    (2001)
  • P. Blundell et al.

    Lesions of the Basolateral Amygdala Disrupt Selective Aspects of Reinforcer Representation in Rats

    J Neurosci

    (2001)
  • K.A. Burke et al.

    Conditioned reinforcement can be mediated by either outcome-specific or general affective

    Frontiers in Integrative Neurosci

    (2007)
  • D.T. Cheng et al.

    Functional MRI of human amygdala activity during Pavlovian fear conditioning: stimulus processing versus response expression

    Behav Neurosci

    (2003)
  • D.R. Collins et al.

    Differential fear conditioning induces reciprocal changes in the sensory responses of lateral amygdala neurons to the CS(+) and CS(-)

    Learn Mem

    (2000)
  • L.H. Corbit et al.

    Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer

    J Neurosci

    (2005)
  • L.H. Corbit et al.

    The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell

    J Neurosci

    (2001)
  • H.D. Critchley et al.

    Olfactory neuronal responses in the primate orbitofrontal cortex: analysis in an olfactory discrimination task

    J Neurophysiol

    (1996)
  • A.R. Delamater

    The role of the orbitofrontal cortex in sensory-specific encoding of associations in Pavlovian and instrumental conditioning

    Ann NY Acad Sci

    (2007)
  • T.B. Demarse et al.

    Enhancement of matching acquisition by differential comparison – outcome associations

    J Exp Psych: Animal Behav Processes

    (1993)
  • N.N. Doron et al.

    Cells in the posterior thalamus project to both amygdala and temporal cortex: a quantitative retrograde double-labeling study in the rat

    J Comp Neurol

    (2000)
  • N.N. Doron et al.

    Organization of projections to the lateral amygdala from auditory and visual areas of the thalamus in the rat

    J Comp Neurol

    (1999)
  • S.B. Dunnett et al.

    Delay-dependent short-term memory deficits in aged rats

    Psychophamacology

    (1988)
  • Cited by (50)

    • Implicit outcomes expectancies shape memory process: Electrophysiological evidence

      2020, Biological Psychology
      Citation Excerpt :

      These findings extend the results of the study by Carmona et al. (2020) and provide evidence for a greater efficiency across most of the components analyzed with the DOP. In sum, the neuronal electrical patterns explored in this study revealed that different mechanisms were activated depending on the outcomes condition, which is consistent with previous studies (Carmona et al., 2019; Carmona et al., 2020; Mok, 2012; Mok, Thomas, Lungu, & Overmier, 2009) as well as with the two memory system model proposed by Savage and colleagues (e.g., Savage & Ramos, 2009). To our knowledge, the present findings provide, for the first time, electrophysiological evidence of implicit prospective processes involved in the DOP, which result in an improved visual recognition memory.

    • Does the implicit outcomes expectancies shape learning and memory processes?

      2019, Cognition
      Citation Excerpt :

      One relevant question we might ask is whether being aware of the specific consequences of our actions is a necessary condition for them to have beneficial effects in cognition (as demonstrated by the DOP effect). The two-memory systems theory (e.g., Savage & Ramos, 2009) would claim this not be the case, because expectancies of the specific outcomes are implicitly formed via classical conditioning associations (i.e., sample stimulus-outcome). After several pairings, the presentation of the sample stimulus would activate the representation of its own and unique outcome and this can be used to make the correct choice.

    View all citing articles on Scopus
    View full text