Research report
The role of prelimbic cortex in instrumental conditioning

https://doi.org/10.1016/j.bbr.2003.09.023Get rights and content

Abstract

Numerous studies have implicated human and primate prefrontal cortex in the ability to hold and manipulate goal or outcome-related information in working memory to guide the performance of forthcoming actions. Here we report that cell-body lesions of prelimbic cortex impair the ability of rats to select an action based on previously encoded action–outcome associations. Rats were food deprived and trained to press two levers, one delivering food pellets and the other a sucrose solution. All rats acquired the lever-press response although the initial acquisition in the prelimbic rats was significantly slower than in sham controls. Furthermore, whereas in sham-lesioned rats, post-training devaluation of one of the two outcomes using a specific satiety procedure produced a selective reduction in performance on the lever that in training delivered the prefed outcome, prelimbic rats failed to show a selective devaluation effect and appeared to reduce performance on both levers non-selectively. Importantly, this impairment only emerged in extinction; in subsequent experiments it was found that, when a specific action–outcome association was cued either by presentation of the outcome itself or by presenting a stimulus previously paired with the outcome, rats demonstrated an ability to select the associated action. These results suggest that action–outcome encoding may be intact in prelimbic rats and that the lesion impaired their ability to retain this learning in working memory in order to establish a course of action. Alternatively, the lesion may have altered the relative contribution of action–outcome and outcome–action associations to instrumental performance. On this account, prelimbic lesions affect action–outcome encoding but leave outcome–action associations intact providing the basis for outcome-mediated initiation of an action sufficient, perhaps, to support acquisition and performance in the lesioned rats.

Introduction

In recent years, neuropsychological studies have established a clear constellation of deficits associated with executive processes in humans with frontal cortical damage. This dysexecutive syndrome has been argued to reflect a disorder in the sustained functioning of the prefrontal-basal ganglia-cortical feedback system [13] and distinct symptoms have been argued to be associated with damage to specific components of this system [36], [47], [51]. For example, damage resulting in the disconnection of prefrontal cortex from mediodorsal thalamus has been implicated in Alzheimer’s disease [14], from regions of the striatum in Parkinson’s, Huntington’s, and obsessive compulsive disorders [13], [59], and from the amygdala in various emotional disorders [12], [22].

In normal individuals, prefrontal cortex has long been implicated in storage and executive as well as specific mnemonic, most notably working memory components, of goal-directed actions in humans and non-human primates [37], [39], [45], [65]. Many of these functions have been argued to converge in the control of goal-directed actions; the retrieval of encoded action–goal information into working memory appears likely to be necessary if the integration of sequences of actions and cognitive states is to maintain performance and successfully achieve a selected goal [32], [33], [41], [50]. Numerous studies have, therefore, implicated this region in a large array of cognitive functions, including action selection [53], [63], planning [4], [58], and selective attention [62], [69]. Nevertheless, although these functions very likely contribute to goal-directed action, the role of prelimbic cortex in the acquisition and maintenance of such actions has been largely assumed; few attempts have been made to assess directly the role of prelimbic cortex in tasks known to tap goal-directed processes.

On the basis of constraints established from accounts of human action [35], [66], Dickinson and Balleine [28], [31] argued that, to define any activity as goal-directed, it must be demonstrated that (a) the consequences of the activity constitute a goal for the animal and (b) the performance of the activity depends on it being causal with respect to access to the goal. Performance in the absence of either of these properties identifies the activity as a non-purposive, essentially reflexive, response. Importantly, recent evidence suggests that in rats free-operant instrumental actions satisfy both of these criteria and, therefore, that this conditioning paradigm provides a good animal model of goal-directed action.

Evidence satisfying the goal criterion comes largely from studies assessing post-training devaluation of the instrumental outcome. To take one of many examples, Corbit and Balleine [18] trained hungry rats to press one lever for food pellets and a second lever for a sucrose solution. After training, either the food pellets or the sucrose solution was devalued using a specific satiety procedure, i.e. the rats were allowed freely to consume either the pellet or sucrose outcome for a 1-h period, a treatment that is well known to reduce both the hedonic reactions and consumption elicited by that specific food relative to other, non-prefed foods [8], [42]. Immediately after this treatment the rats were given a choice test on the two levers conducted in extinction, i.e. in the absence of either outcome. On test, the rats demonstrated that they had encoded the specific action–outcome associations and were able to use them to guide performance; they selectively reduced performance of the action that, in training, had delivered the outcome that they were prefed relative to the other action.

There is also considerable evidence that instrumental learning is determined by the contingency between the performance of the action and access to the outcome. Not only will rats stop responding if performance no longer delivers the instrumental outcome, they stop responding even faster if their responding cancels an otherwise freely available food, i.e. leads to the omission of the outcome [23], [29]. Furthermore, in the study described above, Corbit and Balleine [18] were able to show that their rats were sensitive to the contingency between an action and its specific consequences. When this contingency was degraded by ensuring that the delivery of one of the two outcomes was equally probable, whether its associated action was performed or not, rats reduced the performance of that action without modifying performance of the action that earned the other outcome [9], [10], [20].

These findings, amongst many others [7], [15], [30], have established the basis for the claim that the instrumental conditioning paradigm provides an excellent tool for assessing the behavioral and neural determinants of goal-directed actions in rats. As such, given the evidence described above, prelimbic cortex (henceforth PL) should be predicted to be heavily involved in instrumental learning and performance. In support of this claim, lesions of the PL in rats have been reported to produce deficits both in the acquisition and maintenance of instrumental actions in rats [5], [6] as well as in situations where the rats are required to select from multiple possible responses, learn reversals, or switch behavioral strategies [24], [25], [26], [34], [44], [56], [57], [64]. Furthermore, Balleine and Dickinson [9] reported that lesions of the PL rendered rats insensitive to the selective devaluation of the instrumental outcome, an effect recently replicated by Killcross and Coutureau [46] and extended to show that this effect is specific to the PL. Generally, these results are consistent with the claim, advanced by Balleine and Dickinson [9] that, in instrumental conditioning, the PL is critical for encoding the action–outcome association. Nevertheless, given the reports of working memory deficits produced by similar lesions, it remains possible that PL-lesioned rats have difficulty retrieving and/or maintaining action–outcome information in memory sufficiently effectively to guide forthcoming actions. In the following series of experiments, we further assessed the effect of lesions of the PL on instrumental performance and on the ability of rats to encode and utilize action–outcome information based on outcome devaluation (Experiment 1) and contingency degradation (Experiment 2) tests. Generally, these assessments appeared to support the retrieval rather than the encoding account of PL function and so, in an attempt to provide additional evidence for this position, we also assessed the impact of these lesions on stimulus-induced priming of the instrumental actions using a Pavlovian-instrumental transfer design (Experiment 3).

Section snippets

Experiment 1

In Experiment 1, we trained food-deprived rats given either N-methyl-d-aspartate (NMDA)-induced lesions of the prelimbic area or sham surgery to press two levers, one delivering food pellets and the other delivering a 20% sucrose solution. After training, we assessed the rats’ sensitivity to the impact of specific satiety-induced outcome devaluation on performance by allowing them to consume either the pellets or sucrose for 1 h after which we assessed their performance in a brief choice

Experiment 2

In this experiment, we assessed more directly the impact of PL lesions on instrumental learning. This was achieved by examining the rats’ sensitivity to degradation of the specific action–outcome contingencies in a situation in which one or the other outcome was delivered in such a manner that it was equally probable whether its associated action was performed or not. This test allowed us to assess whether unpaired presentations of a specific outcome acts to degrade that action–outcome

Experiment 3

Experiment 3 was conducted for two reasons. First, we were interested in assessing whether the effects of PL lesions were specific to instrumental performance and so we examined the effects of PL lesions on acquisition and performance of another learning task; in this case, standard appetitive Pavlovian conditioning. Second, by establishing cues associated with reward in the Pavlovian-conditioning procedure, we could test a further prediction of a retrieval position advanced to account for the

General discussion

The aim of this series of experiments was to characterize further the effect of lesions of prelimbic cortex on instrumental conditioning in rats. In Experiment 1, we found that initial acquisition of free-operant lever pressing was slower in PL-lesioned rats than in sham controls. Although this effect has previously been reported [5], [6], it has not been consistently observed [9], [46], something that likely reflects differences across studies in training conditions and in the extent of the

Acknowledgements

The research reported in this paper was supported by a grant from the National Institute of Mental Health to BWB (NIMH #56446). The authors thank Sandra Cetl for her assistance with data collection.

References (69)

  • E.A. Asratyan

    Conditioned reflex theory and motivational behavior

    Acta Neurobiol. Exp.

    (1974)
  • Baddley AD. Working memory. Oxford: Oxford University Press;...
  • A.E. Baldwin et al.

    N-Methyl-d-aspartate receptor-dependent plasticity within a distributed corticostriatal network mediates appetitive instrumental learning

    Behav. Neurosci.

    (2000)
  • A.E. Baldwin et al.

    Appetitive instrumental learning requires coincident activation of NMDA and dopamine D1 receptors within the medial prefrontal cortex

    J. Neurosci.

    (2002)
  • Balleine BW. Incentive processes in instrumental conditioning. In: Mowrer R, Klein S, editors. Handbook of contemporary...
  • B.W. Balleine et al.

    The role of incentive learning in instrumental outcome revaluation by sensory-specific satiety

    Anim. Learn. Behav.

    (1998)
  • B.W. Balleine et al.

    The effect of lesions of the basolateral amygdala on instrumental conditioning

    J. Neurosci.

    (2003)
  • Barto AG, Sutton RS. Reinforcement learning in artificial intelligence. In: Donahoe JW, editor. Neural-network models...
  • A. Bechara et al.

    Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making

    J. Neurosci.

    (1999)
  • C.-C. Chu et al.

    The autonomic-related cortex: pathology in Alzheimer’s disease

    Cereb. Cortex

    (1997)
  • Colwill RC, Rescorla RA. Associative structures in instrumental learning. In: Bower GH, editor. The psychology of...
  • R.M. Colwill et al.

    Associations between the discriminative stimulus and the reinforcer in instrumental learning

    J. Exp. Psychol. Anim. Behav. Process

    (1988)
  • J.L. Contreras-Vidal et al.

    A predictive reinforcement model of dopamine neurons for learning approach behavior

    J. Comput. Neurosci.

    (1999)
  • L.H. Corbit et al.

    The role of the hippocampus in instrumental conditioning

    J. Neurosci.

    (2000)
  • L. Corbit et al.

    The role of the nucleus accumbens in instrumental conditioning: evidence for a functional dissociation between accumbens core and shell

    J. Neurosci.

    (2001)
  • L.H. Corbit et al.

    Sensitivity to instrumental contingency degradation is mediated by entorhinal cortex and its efferents through the dorsal hippocampus

    J. Neurosci.

    (2002)
  • L.H. Corbit et al.

    Pavlovian and instrumental incentive processes have dissociable effects on components of a heterogeneous instrumental chain

    J. Exp. Psychol. Anim. Behav. Process

    (2003)
  • A.R. Damasio

    The somatic marker hypothesis and the possible functions of the prefrontal cortex

    Philos. Trans. R. Soc. Lond. B Biol. Sci.

    (1996)
  • J. Davis et al.

    Differential reinforcement of other behavior (DRO): a yoked-control comparison

    J. Exp. Anal. Behav.

    (1971)
  • B. Delatour et al.

    Prelimbic cortex specific lesions disrupt delayed-variable response tasks in the rat

    Behav. Neurosci.

    (1996)
  • B. Delatour et al.

    Lesions of the prelimbic-infralimbic cortices in rats do not disrupt response selection processes but induce delay-dependent deficits: evidence for a role in working memory?

    Behav. Neurosci.

    (1999)
  • Dickinson A. Instrumental conditioning. In: Mackintosh NJ, editor. Animal cognition and learning. London: Academic...
  • Dickinson A, Balleine BW. Causal cognition and goal-directed action. In: Heyes C, Huber L, editors. Evolution of...
  • A. Dickinson et al.

    Omission learning after instrumental pretraining

    Q. J. Exp. Psychol.

    (1998)
  • Cited by (0)

    View full text