Introduction

When we remember an event from our past, it sometimes feels as though we are returned to the place and time of that prior moment. Indeed, Endel Tulving proposed that a core feature of episodic memory was this ability to travel through subjective time, from imagining the future to visualizing the past (e.g., Tulving, 2002). Yet not all memories return to us with a feeling of re-experience. Some moments from our past are more likely than others to be remembered with this phenomenological sense of re-experience, and even for the same event, the extent of re-experiencing can vary from one recall to the next (discussed in Holland & Kensinger, 2010).

One agreed upon factor that can increase this feeling of re-experience is the emotionality of an event: Events of emotional significance are more likely than mundane experiences to be remembered over time and to be recalled vividly. These effects of emotion have been demonstrated in many studies of “flashbulb memory” (Brown & Kulik, 1977), autobiographical memory retrieval, and in assessments of memory for words and pictures presented within a laboratory context (reviewed by Kensinger & Schacter, 2008a). In fact, it has been argued that high memory confidence or vividness may be hallmark characteristics of emotional memory (Phelps & Sharot, 2008; Talarico & Rubin, 2003), existing even when some event details are remembered incorrectly (e.g., Neisser & Harsh, 1992).

To date, most research examining why memories of emotional events differ in their profiles from memories of neutral events has focused on encoding and consolidation – that is, on the way information is initially learned or stored (see Table 1). Yet because many influential models of memory – including encoding specificity (Tulving & Thomson, 1973), transfer appropriate processing (Morris, Bransford, & Franks, 1977), and cortical re-instantiation (Buckner & Wheeler, 2001) – conceive of retrieval as a time during which the operations of encoding are recapitulated, it may be more fruitful to consider the effects of emotion on memory from this standpoint. That is, the likelihood that an emotional experience comes to mind – and the subjective vividness with which that event is remembered – will be influenced not only by processes that act at encoding or at retrieval but by the conjunction of processes engaged at those two times.

Table 1 Prominent theories proposed to explain the effects of arousal on memory, organized by the memory stage at the focus of the theory
Full size table

The model that we eventually propose here capitalizes on a burgeoning literature that has provided evidence that episodic memory retrieval is linked to the reactivation and recapitulation of neural computations and activations engaged during encoding (e.g., Rugg, Johnson, Park, & Uncapher, 2008a, b; Waldhauser, Braun, & Hanslmayr, 2016; Wheeler, Petersen, & Buckner, 2000). We begin by briefly reviewing this literature. We then review key findings from the emotional memory literature, describing the effects of emotional arousal on memory and outlining some of the most influential theories to describe those effects. We then discuss the evidence that many of the effects of emotion during memory retrieval relate specifically to how emotion seems to affect the recapitulation of encoding processes, which may serve both to enhance the storage of emotional events and to affect the subjective quality of their retrieval. We point out that the extant literature suggests not only an effect of emotional arousal on recapitulation, but also an effect of valence. Even when events are of equally high arousal, there appear to be different behavioral and neural profiles for negative events than for positive events. Building from this integration from the literature, we propose a new model for these valence-dependent profiles: Negative Emotional Valence Enhances Recapitulation (NEVER). This model diverges from existing models of emotional memory by emphasizing differences that are related to emotional valence and by proposing that these differences reflect not only the way information is learned or stored but also the way information – sensory aspects, in particular – is brought to mind at the time of retrieval.

The importance of recapitulation and reactivation for episodic memory retrieval

The word “recollect” stems from the Latin recolligere, “to gather or collect again,” implying a memory process by which information from a studied episode is gathered together again at the time of retrieval. This word provides an apt description of episodic memory retrieval insofar as it emphasizes the overlap between encoding and retrieval processes. It has long been known that the operations engaged during encoding will ultimately determine what information is available at retrieval (e.g., Atkinson & Shiffrin, 1968). Extending this idea, classic cognitive models of memory have proposed that the processes engaged at encoding will determine the effectiveness of a retrieval cue to elicit a memory (Tulving & Thomson, 1973; Tulving, 1983) and further, that memory retrieval succeeds when the processes engaged at encoding closely match those engaged at retrieval (e.g., Morris, Bransford, & Franks, 1977; but see Nairne, 2002).

A burgeoning neuroimaging literature has demonstrated that episodic memory retrieval is marked by recapitulation, a re-engagement of the neural activations and patterns that contribute to successful encoding (e.g., Buckner & Wheeler, 2001). This literature has brought into focus the role of sensory engagement in episodic memory retrieval, revealing that the same sensory-specific cortices that were used to encode information are activated again during retrieval, even when the sensory information is not re-presented at retrieval (see reviews by Danker & Anderson, 2010; Rugg et al. 2008a, b). For instance, information that was studied aurally activates auditory cortex at retrieval, while information that was studied visually activates visual cortex; within visual cortex, retrieval activity can differ for information presented statically or in motion, or can depend on the visual field in which information was studied (Gottfried, Smith, Rugg, & Dolan, 2004; Nyberg, Habib, McIntosh, & Tulving, 2000; Slotnick, 2004; Slotnick & Thakral, 2011; Waldhauser, Braun, & Hanslmayr, 2016; see also Wheeler, Petersen, & Buckner, 2000; Vaidya, Zhao, Desmond, & Gabrieli, 2002). These patterns can emerge even when the retrieval cue does not contain information about the modality of the studied information. Moreover, machine learning algorithms can successfully distinguish which classes of stimuli were studied, using only information available at retrieval (e.g., Polyn, Natu, Cohen, & Norman, 2005; also see Kuhl, Rissman, & Wagner, 2012; Leiker & Johnson, 2014, 2015; Norman, Polyn, Detre, & Haxby, 2006).

This recapitulation seems to have ties to memory success: Memory appears to be most successful when this recapitulation occurs with high fidelity (Kuhl et al., 2014; St-Laurent, Abdi, Bondad, & Buchsbaum, 2014). For example, using representational similarity analysis, Wing, Ritchey, and Cabeza (2015) demonstrated that the encoding-to-retrieval similarity in occipito-temporal regions was higher when memory was successful. They also revealed that hippocampal activation at encoding predicted the amount of recapitulation at retrieval (Wing, Ritchey, & Cabeza, 2015) and that the hippocampus mediated the link between memory success and encoding-to-retrieval similarity (Ritchey et al., 2013; see also Bosch et al., 2014 and Walker et al., 2017 for evidence that hippocampal function or integrity may relate to cortical recapitulation).

Studies using electroencephalography (EEG) have similarly demonstrated retrieval-related recapitulation of processes engaged during encoding, marked by context-dependent event-related potentials (ERPs) that are often correlated with successful recollection. The temporal specificity of these ERP studies has provided evidence that these potentials onset relatively early in retrieval (beginning to emerge as early as 300 ms after the onset of the retrieval cue), likely indicating that they are driving the recollective experience and not only reflecting the recovery of episodic details (Johnson, Minton, & Rugg, 2008; Johnson, Price, & Leiker, 2015). These results are broadly consistent with the functional magnetic resonance imaging (fMRI) results of Polyn and colleagues (2005), who found that category-specific activity preceded the recall of information from that category by multiple seconds.

A recent study by Waldhauser, Braun, and Hanslmayr (2016; also see Bowen & Kark, 2016 for a commentary) provided even stronger evidence that recapitulation may precede successful retrieval. In one sample of participants, Waldhauser and colleagues found recapitulation of oscillatory changes (alpha/beta power decreases) linked to successful encoding of objects in the left and right visual fields that was source-localized to the contralateral lateral occipital cortex (LOC). In a separate sample, they applied trial-level rhythmic transcranial magnetic stimulation (TMS) in the alpha/beta frequency range to the LOC before the onset of typical recollection effects (<500 ms); they found that this TMS disrupted participants’ source memory for the spatial context (visual field location) of objects studied in the contralateral visual field. These findings add to a growing literature on the causal role of the visual cortex in memory for visual events (see also van de Ven & Sack, 2013). Moreover, they specifically suggest that recapitulation in sensory cortices can be causally related to recollection of episodic details, such as contextual details, providing the first evidence that early sensory reactivation (prior to recollection) is needed for episodic memory.

The literature just reviewed suggests that recapitulation often occurs at the time of retrieval and can be tied to memory success and to the recollective experience. A related study has revealed that when the networks that were active during encoding are reactivated – either spontaneously during offline periods or in response to environmental cues – such reactivation can trigger the implementation of new storage processes and may strengthen the memory representation. Although the concepts of recapitulation and reactivation are interrelated, in this review we use “recapitulation” to refer to a re-engagement – at the moment someone retrieves or monitors content from memory – of neural activity or patterns that were associated with successful encoding. We use “reactivation” to refer to a re-engagement – which can occur spontaneously during offline states or be induced passively through environmental cues – of any processing which took place during the events (see Morcom, 2014 for elaboration of this distinction).

Wilson and McNaughton (1994) were the first to demonstrate that, after animals learned a spatial task, there was reactivation during sleep in the hippocampal “place cells” (O’Keefe, 1976) that had been active during the performance of the spatial task. Subsequent research has confirmed that memory reactivation during sleep can stabilize or enhance memory in animals (Bendor & Wilson, 2012) and in humans (for reviews see Oudiette & Paller, 2013; Paller & Voss, 2004). These reactivations are thought to promote memory consolidation (for a review see Buhry, Azizi, & Cheng, 2011; Marshall & Born, 2007; Oudiette & Paller, 2013; Paller & Voss, 2004) via repeated and coordinated reactivations of the hippocampus and neocortex (Buzsáki, 1998; Haggerty & Ji, 2014; McClelland et al., 1995; Teyler & Rudy, 2007; Wilson & McNaughton 1994).

Spontaneous reactivations can also occur during dreaming. For example, Wamsley and colleagues (2010) demonstrated that dream imagery of a previously studied virtual navigation task during slow wave sleep (SWS) was associated with better memory performance at test. While reactivations have been demonstrated during rapid eye movement (REM) sleep as well (Louie & Wilson, 2001), they are particularly prominent during SWS (e.g., Rasch et al., 2007). The prevalence of reactivations during this stage of sleep align with evidence that SWS is a time of synchronous activation of the hippocampus and neocortex (Ego-Stengel & Wilson, 2010), providing an optimal neurophysiological environment for memory consolidation (Haggerty & Ji, 2014; Batterink, Creery, & Paller, 2016; see Sirota & Buzsaki, 2005 for a review).

Over the past decade, researchers have leveraged environmental cuing to study memory reactivation during sleep, confirming that reactivation during SWS conveys subsequent benefits to memory. In the typical targeted memory reactivation (TMR) paradigm (see Rasch et al., 2007 for the pioneering study), unique tones or odorants are presented with each event that participants study during wakefulness. During a subsequent period of sleep, only some of the tones or odorants from the study session are covertly presented to the sleeping participant. Upon waking, memory is enhanced for those cued events, compared to the un-cued events. For example, memories cued by tones presented during SWS are enhanced upon waking, and neural changes in the medial temporal lobe induced by reactivation predict subsequent memory performance (van Dongen et al., 2012; reviewed by Ouidette & Paller, 2013).

In the realm of emotional memory, recent work demonstrated that TMR during NREM sleep enhanced memory for highly arousing negative pictures, but not neutral pictures. For negative stimuli, TMR was also associated with increases in theta and spindle oscillations, which are thought to reflect successful reinstatement and memory stabilization, respectively (Lehmann, Schreiner, Seifritz, & Rasch, 2016). These findings suggest reactivation during NREM sleep is modulated by emotion; however, it is not known whether this cuing benefit extends to positive stimuli.

Although there is some evidence to suggest that offline reactivation is most beneficial when it occurs during sleep (e.g., Wamsley et al., 2010), reactivation also has been demonstrated during wakefulness (Foster & Wilson, 2006) and is similarly associated with coordinated activation of the hippocampus and neocortex (Ji & Wilson, 2007). However, it is still an open question whether the types of memories facilitated by reactivation during sleep and wakefulness diverge. One possibility is that reactivation during wakefulness strengthens individual memories (e.g., Deuker et al., 2013) – consistent with recent evidence that reactivation during wakefulness can enhance the ability to discriminate similar events from one another (van den Honert, McCarthy, & Johnson, 2016) – while reactivation during sleep is more likely to promote abstraction or association among related memory traces (see Lehmann, Schreiner, Seifritz, & Rasch; Oudiette et al., 2013 for discussions).

The literature just reviewed emphasizes three points that, we will argue, must be considered in the study of emotional memory. First, memory success is predicted both by the cognitive and neural processes that unfold during encoding and also by the recapitulation of those processes during retrieval. Second, recapitulation of sensory engagement is particularly relevant in the study of episodic memory, and perhaps especially when elucidating what leads to a phenomenology of recollection. Third, memory success over longer delay intervals additionally benefits from reactivation – of operations engaged during the event – between the time of encoding and the time at which memory retrieval is tested. In the sections below, we provide a brief review of the literature on episodic memories of emotional events to highlight why we argue these points are essential to that field of study, and then we turn our attention to describing how our model brings these points into focus.

Episodic memories of emotional events

There is no doubt that “emotional memories” – memories for events that elicited emotion at the time of their occurrence – are marked by the subjective vividness with which people remember those moments in time (see Phelps & Sharot, 2008). Although memories for high-arousal events differ from other memories in their phenomenological qualities at retrieval, most theories of emotional memory have emphasized the role of arousal during the encoding and consolidation phases of memory (see Table 1). There is good reason for the focus on these early stages. An extensive literature has revealed that emotional arousal begins to affect memory processes – such as neurohormonal activation in the amygdala, and amygdala modulation of hippocampal activation – the moment an event is experienced: The experience of arousal during, or soon after, an event leads to long-lasting changes in the durability of a memory (e.g., McGaugh, 2004; Dunsmoor et al., 2015). However, as the prior section underscores, emotional memory enhancements to episodic memory – including enhancements in the recollective phenomenology of a memory – likely require an understanding not only of how emotion modulates these early stages of memory formation, but also how it affects the reactivation and recapitulation of processes engaged during those stages. In the next sub-sections, we briefly review the literature that has examined effects of emotion on encoding and storage (see LaBar & Cabeza, 2006; Mather & Sutherland, 2011; McGaugh, 2000; Murty et al., 2011; Phelps, 2004 for additional reviews) and then describe the emerging evidence for effects of emotion on recapitulation and reactivation.

Encoding and storing emotional events

Emotional arousal can influence the way memories are formed. Physiological arousal during encoding (e.g., skin conductance, heart rate) has been correlated with long-term memory outcomes (Abercrombie, Chambers, Greischar, & Monticelli, 2008; Gavazzeni et al., 2012), and a recent study demonstrated that reducing systemic arousal via administration of propranolol (a beta-adrenergic receptor antagonist) reduced the subjective sense of recollection predominantly if it was administered at encoding (Rimmele et al., 2016). These studies reveal that arousal exerts effects early in memory formation. Indeed, many studies have demonstrated that the enhancing effects of arousal on memory can occur just minutes after an event has occurred (Dolcos, LaBar, & Cabeza, 2004; Kensinger & Corkin, 2004; Talmi & McGarry, 2012; Mickley Steinmetz et al., 2012) and can be associated with greater activation of the amygdala, hippocampus, frontal and temporal cortices, as well as the ventral visual stream during encoding (Murty et al., 2011).

Although the mechanisms underlying these effects of arousal continue to be discussed (see Table 1), it is thought that emotionally arousing stimuli capture attention and encourage elaboration (Hamann, 2001; LaBar & Cabeza, 2006; Nummenmaa, Hyona, & Calvo, 2006; Talmi, 2013; Steinmetz & Kensinger, 2013) even when participants are instructed to focus on non-arousing elements of a visual scene (LaBar, Mesulam, Gitelman, & Weintraub, 2000) or to avoid looking at an emotional picture presented alongside a neutral picture (Calvo & Lang, 2004; Nummenmaa, Hyona, & Calvo, 2006). In at least some instances, the selective orientation toward emotional content can precede – and even drive – the first saccade toward emotional content (Nummenmaam, Hyona, & Calvo, 2006, 2009; Simola, Torniainen, Moisala, Kivikangas, & Krause, 2013; Todd et al., 2012), suggesting early-acting effects of emotional arousal on sensory processing. Critically, these studies have matched the emotional and neutral pictures on measures of visual complexity; thus, visual attention capture by emotional information does not seem to be driven only by differences in low-level visual features. In regard to memory, selective attentional capture may optimize encoding and post-encoding actions that increase the likelihood that goal-relevant information is remembered (e.g., Mather & Sutherland, 2011). For example, emotionally enhanced perceptual vividness at encoding contributes to enhanced vividness at retrieval (Todd, Schmitz, Susskin, & Anderson, 2013). Interestingly, Todd and colleagues (2013) reported emotional and objective visual salience made separable contributions to emotionally enhanced perceptual vividness, suggesting that the emotional meaning of an image results in altered subjective vividness beyond differences in objective salience.

Despite these early actions of arousal, the beneficial effects of emotional arousal on memory tend to increase over time (Sharot, Verfaellie, & Yonelinas, 2007; Sharot & Yonelinas, 2008), with memory for neutral items showing a steeper decay curve than memory for high-arousal items (LaBar & Phelps, 1998; Sharot & Phelps, 2004; Yonelinas & Ritchey, 2015). In fact, many studies report enhancing effects of emotion on memory only when the retention interval between study and test is upwards of 24 h (discussed in Bennion et al., 2013), although enhancements at shorter delays can also occur (discussed in Talmi, 2013). This persistence of emotional memory may relate to the strengthened amygdala-hippocampal connectivity that occurs during the encoding of high-arousal information (Ritchey, Dolcos, & Cabeza, 2008; Fastenrath et al., 2014) or to the ability for the amygdala to bind items and emotions in a way that does not require the hippocampus proper (Yonelinas & Ritchey, 2015).

Consistent with these time-enhanced effects of arousal, there is abundant evidence that effects of arousal persist into the consolidation period and contribute to the preferential preservation of emotional memories. Mather and Sutherland (2011) reviewed ways in which post-encoding arousal can facilitate information that was prioritized during encoding. In many naturalistic experiences, that prioritized content may be emotional, perhaps explaining why – although post-encoding stress or arousal can sometimes facilitate memory for neutral information (e.g., Anderson, Wais, Gabrieli, 2006) – there often is a disproportionate benefit for emotional information (e.g., Cahill & Alkire, 2003; Liu et al., 2008).

Information learned after-the-fact also can retroactively influence the way that emotional events are consolidated (e.g., Dunsmoor et al., 2015; see Kensinger, 2015 for discussion). A systems-consolidation account of emotional memory enhancement suggests arousal-related amygdala engagement modifies the threshold for synaptic modification between distant brain regions, and that this effect persists into consolidation (Hermans et al., 2014; Roozendalal et al., 2009).

Retrieving emotional events

Although the ability for emotion to modulate the mechanisms supporting successful retrieval has not been the focus of most theories of emotional memory, there is evidence for effects of emotion during retrieval. Memories for arousing events are associated with enhanced subjective phenomenology, even when they are not remembered with more accurate detail (Neisser & Harsh, 1992; Phelps & Sharot, 2008; Talarico & Rubin, 2003), and arousal also leads to some distinctions in the retrieval-related neural activation, compared to neutral events. Successful retrieval of high-arousal events often is associated with greater engagement of amygdala and medial temporal-lobe regions, as well as with greater recruitment of visual processing regions as compared to neutral memories (Kark & Kensinger, 2015, Keightley et al., 2011; Kensinger and Schacter, 2007; Mitchell et al., 2006; Taylor et al., 1998). Neuroimaging studies that have used a protracted retrieval epoch have provided evidence that the effects of emotional arousal are larger during the search phase of memory than during later elaboration phases (e.g., Daselaar et al., 2008; Ford, Morris, & Kensinger, 2014), and ERP studies have similarly suggested that emotion can modulate relatively early retrieval-related signatures (e.g., Weymar et al., 2009). Together, this research suggests that arousal may facilitate the search for a memory, making it more likely that affective events come to mind (discussed in Holland & Kensinger, 2010) or are associated with subjective feelings of recollection (discussed in Phelps & Sharot, 2008).

In line with the increased interest in reactivation and recapitulation reviewed earlier, there has been growing interest in understanding whether emotion increases the likelihood that processes engaged during encoding are reactivated during delay periods and/or recapitulated during retrieval. Much of this research has focused on whether the affective processing during an initial event is recapitulated at retrieval. It has long been known that the recapitulation of an affective state during retrieval can increase the likelihood that events consistent with that affective state are retrieved (i.e., mood-dependent and mood-congruent effects; Bower, 1981; Bower & Forgas, 2000; Eich, 1995; reviewed by Buchanan, 2007) but the first neural evidence to support this recapitulation of affect came from ERP studies. The general design of these studies was that individuals would study neutral information in either an affective or non-affective context (e.g., Maratos & Rugg, 2001; Schupp et al., 2000; Smith, et al. 2004). Later, when only the neutral information was used to cue memory, the retrieval-related electrophysiological signatures were still found to differ based upon the affective context of the original event. Specifically, Maratos and Rugg (2001) demonstrated that the affective context in which information had been studied affected a late-posterior potential thought to index the monitoring of retrieved content. Smith, Dolan, and Rugg (2004) replicated the effect of affective context at encoding on this late posterior potential, but also provided additional evidence that the affective context of studied information modulated an earlier component, prior to the parietal old/new effect associated with the recovery of mnemonic content. As stated by Buchanan (2007), these studies suggest that “…mechanisms may act to recapitulate an emotional state early in the retrieval process and then produce a response similar to that elicited by the perception of an emotional stimulus upon successful recollection” (p. 767).

Results from fMRI studies have been consistent with evidence that portions of affective networks can be recapitulated at retrieval. Activation of the amygdala has been shown to be greater when retrieving information studied in affective contexts as compared to those studied in neutral contexts, even when the prompt used to cue retrieval is always neutral (e.g., Smith, Henson, Dolan, & Rugg, 2004). Moreover, there has been enhanced functional connectivity between the amygdala and both the hippocampus and the medial prefrontal cortex (Smith et al., 2006), depending on the specific memory demands of the task. Although fMRI studies do not have the temporal precision to distinguish when in the retrieval process such recapitulation occurs, this research has been able to demonstrate that the affective context at the time of study can influence neural activation at retrieval even when retrieval fails (e.g., Smith et al., 2005, 2006). These results are broadly consistent with the ERP studies: affective recapitulation may occur early in the retrieval epoch (and may not always be associated with successful recovery of information) and also later in the retrieval epoch (in response to the successfully retrieved content).

Although the studies just described focused on whether the affective context present at encoding can be recapitulated at retrieval, there have been a few studies examining whether emotion can influence the likelihood that other types of episodic content – representing the who, what, and where details of an event – are recapitulated at retrieval.Footnote 1 A first study to hint at a role for emotion in the recapitulation of episodic content came from Fenker and colleagues (2005). The researchers presented participants with non-emotional words paired with fearful or neutral facial expressions. At retrieval, participants were re-presented with the words and asked to indicate whether they remembered the word or simply knew it to be familiar from the study episode. Although participants were not specifically cued to retrieve the facial information from the study episode, the results revealed greater activity in the portion of the fusiform gyrus associated with face processing during the recollection of words paired with fearful faces compared to the recollection of words paired with neutral faces. Amygdala activity was also greater during recognition of words paired with fearful faces compared to neutral faces, although this amygdala difference only emerged when examining the words that were recognized on the basis of familiarity. These results suggested that affect could increase the likelihood that sensory details were recapitulated at retrieval, and further hinted that the trials on which recapitulation of sensory content was greatest (i.e., recollected trials) may not be the same as those on which amygdala activation was greatest (i.e., familiar trials). Further evidence that arousal may enhance recapitulation of episodic content came from Ritchey and colleagues (2013); using representational similarity analysis, they found that visual scenes that elicited emotional arousal were associated with enhanced encoding-to-retrieval similarity in neural representations – including within temporo-occipital regions – than were neutral stimuli. Interestingly, Ritchey and colleagues (2013) also found that encoding-to-retrieval neural similarity in the occipital cortex correlated with amygdala activity, but only for successfully retrieved negative trials. This latter result suggests a role for emotional valence, as well as arousal, in the recapitulation of episodic content, a conclusion we return to in the section below.

Role of valence in emotional memory

As outlined earlier, most models of emotional memory have focused on the modulatory influences of arousal (see Table 1). Yet there is evidence for effects of valence, stemming both from behavioral and neural literatures (see Fig. 1). Although valence effects on memory sometimes can be attributable to confounds unrelated to valence (see Supplementary Discussion), there also are theoretically-relevant differences in the ways that positive and negative information are processed and remembered.

Fig. 1
figure 1

The neural processes and behavioral outcomes associated with memory for negative events. Negative valence has been shown to enhance the encoding, storage, and recapitulation of sensory details and to lead to enhancements in memory for sensory detail and in the subjective feeling of re-experiencing an event. Examples of these effects, from the authors’ prior research, are depicted. Data from (clockwise): (a) Mickley and Kensinger, 2008; (b) Bennion et al., 2017; (c) Payne and Kensinger, 2011; (d) Kark and Kensinger 2015a; (e) Kensinger, Garoff-Eaton, et al. 2007b; (f) Holland & Kensinger, 2012

Full size image

Although the behavioral effects of valence have been mixed (see Table 2 and Supplementary Table 1), one of the most consistent patterns is that negative memories tend to be associated with increased recollection (e.g., Dewhurst & Parry, 2000; Ochsner, 2000) and with better memory for at least some item-specific details than positive memories (e.g., D’Argembeau et al., 2005; Kensinger & Choi, 2009; Liu et al., 2008; Wang, 2013b). For instance, when comparing memory for negative stimuli to memory for positive stimuli, individuals are more likely to say that they vividly remember having seen a negative item before (e.g., Dewhurst & Parry, 2000; Ochsner, 2000), and they are better able to remember which negative object exemplar they saw (Kensinger, Garoff-Eaton, & Schacter, 2007b), or whether they imagined or saw a negative item (Kensinger, O’Brien et al., 2007). These effects of valence have been demonstrated when assessing memory for experimenter-controlled stimuli, such as words (Dewhurst & Parry, 2000; Kensinger, O’Brien et al., 2007), objects (Kensinger, Garoff-Eaton et al., 2007b), or pictures (Ochsner, 2000) and also when assessing memory for public events, such as political elections (Holland & Kensinger, 2012) or high-profile sporting events with unexpected outcomes (Kensinger & Schacter, 2006a).

Table 2 A brief summary of studies that have examined the effects of valence on memory while controlling for effects of arousal. Only studies that directly mention controlling for arousal, or that report means for arousal that did not significantly differ between negative and positive, are included below. See Supplementary Table 1 for an expanded summary of these studies and for inclusion of studies that may not have controlled for arousal
Full size table

Consistent with these behavioral differences – but occurring even in paradigms that do not reveal behavioral effects of valence – negative stimuli have shown different routes to successful encoding than positive stimuli. The successful encoding of negative stimuli often relies more heavily on the engagement of sensory processes than does the encoding of positive stimuli (e.g., Mickley & Kensinger, 2008; Mickley Steinmetz & Kensinger, 2009), and this sensory activity often is linked to a detailed, recollective-based memory (e.g., Kensinger & Schacter, 2008a, b; Mickley Steinmetz & Kensinger, 2009). The increased engagement of sensory processes for negative stimuli is generally consistent with evidence that negative valence leads to stronger attentional capture effects than positive valence, even when negative and positive stimuli are matched for arousal and visual complexity (Simola et al., 2013). Prior work also has suggested that the effect of arousal on connectivity between nodes of the emotional memory network depends on valence. Specifically, effective connectivity between the amygdala and sensory-processing regions is increased for successfully-encoded high arousal negative stimuli but decreased for successfully-encoded high arousal positive stimuli (Mickley Steinmetz, Addis, & Kensinger, 2010). These effects arise even when positive and negative images are matched on factors such as arousal, visual complexity, brightness, and category of picture content (e.g., animals, humans, buildings), reducing the likelihood that stimulus-level confounds drive the valence differences. These findings suggest valence-specific effects of arousal during encoding of emotional memories.

The sensory signature for negative memories appears to persist into subsequent phases of memory. A few studies have revealed greater activation in ventral visual processing streams during the retrieval of negative as compared to positive memories (e.g., Markowitsch et al., 2003; Piefke et al., 2003), paralleling the valence differences seen during encoding. This sensory signature at retrieval may become exaggerated with sleep-filled delays that optimize consolidation processes. After a period of sleep compared to a period of wakefulness, there is enhanced sensory connectivity (Payne & Kensinger, 2011) and sensory activity during the retrieval of negative events, even if those events are cued by neutral content (Bennion et al., 2017). There also is a shallower forgetting curve for the specific details of negative information if sleep follows soon after learning than if a comparable period of time spent awake follows learning (e.g., Payne et al., 2012; Weymar, Löw & Hamm, 2011; see also Wagner et al., 2001).

Another clue as to the importance of valence has come from studies of patients with amygdala damage. These patients show difficulty re-experiencing past negative events, while they do not show a comparably large deficit for the retrieval of positive events (Buchanan et al., 2005, 2006; see also Claire et al., 2016). As noted by Buchanan (2007), the reason for this specificity to negative events is uncertain and could either reflect an inability to search for the negative events or to recognize the retrieved events as being of negative valence. This finding is intriguing, because neuroimaging studies often show comparable activation of the amygdala to positive and negative memories (e.g., Hamann & Mao, 2002; see meta-analysis by Costafreda et al., 2008); however, a recent study revealed that during memory retrieval, projections from the basolateral amygdala differed depending upon valence, with negative memories associated with stronger connections between the basolateral and centromedial nuclei (Beyeler et al., 2016). Most fMRI studies would not have the resolution to detect these differences, raising the possibility that there is a greater separation of valence in the amygdala than has previously been detected in human neuroimaging studies.

Together, these studies have emphasized that the way memory processes unfold, from the moment of encoding until the time of retrieval, can depend not only on the arousal of an event but also on its valence (see Fig. 1). For events of high arousal, different behavioral and neural profiles can exist depending on the valence of the event. We recently tested the hypothesis that not only does the valence of high-arousal events affect each separate phase of memory, it also affects the overlap across these phases. We conducted two studies to examine the effect of emotional valence on the way memories are recapitulated during retrieval. In Kark and Kensinger (2015), participants viewed full-color negative, positive, and neutral images from the International Affective Picture System (IAPS; Lang, Bradley, & Cuthbert, 2008) at encoding, and at retrieval were cued with black-and-white line-drawing versions of these images. The use of line drawings at retrieval provided a relatively neutral visual cue for specific items from the study phase. We revealed greater encoding-to-retrieval overlap for negative stimuli compared to positive stimuli in regions associated with visual processing (see row 2 of Fig. 3), and we demonstrated that this overlap can be linked both to memory success and to memory confidence (see row 4 of Fig. 3). Further, we found that the same regions of the fusiform gyrus that were functionally connected with the amygdala during encoding were reactivated at retrieval. Critically, negative and positive IAPS images were pre-matched on arousal and were equally more emotional than neutral images, using normative data. The images were equated for visual complexity, as assessed by the JPEG-file size in kilobytes (Donderi, 2006). Thus, the sensory recapitulation for negative items in this study is not likely due to differences in arousal or low-level visual confounds.

We replicated the general enhancing effect of negative valence on recapitulation processes within visual processing regions in a second study, in which participants studied neutral words in the context of negative, positive, or neutral faces or scenes (Bowen & Kensinger, 2016). As in Kark and Kensinger (2015), there was much greater encoding-to-retrieval overlap within the ventral visual processing stream during the retrieval of neutral words that had previously been studied in negative contexts compared to positive or neutral contexts (see row 2 of Fig. 3). In this study, we extended the results of Kark and Kensinger (2015) in three important ways. First, by using a conceptual (word) cue rather than a sensory (line-drawing) cue, we could confirm that the greater sensory recapitulation for negative memories generalizes across different cue types. Second, we could fully counterbalance the valence of the context that was presented with each word, thereby assuring that any differences in retrieval processes could only be driven by the study context in which those words had been presented. Third, by asking participants to make remember/know distinctions, we could reveal that the effects of valence were robust when recollection occurred (i.e., correct “remember” responses) but not when responses were based on familiarity. In an additional analysis of this dataset (see blue regions depicted in row 4 of Fig. 3), we found that greater sensory recapitulation was associated with recollection more than familiarity specifically for negative stimuli. This pattern is consistent with prior literature connecting recapitulation to the experience of recollection (Fenker et al., 2005; Johnson, Minton, & Rugg, 2008; Johnson, Price, & Leiker, 2015; but see Johnson, McDuff, Rugg, & Norman, 2009 for evidence that recapitulation can sometimes be related to familiarity as well as recollection), but suggests that this relation may be particularly strong for negative stimuli.

Importantly, we demonstrated these valence differences despite very little (Kark & Kensinger, 2015) or absolutely no (Bowen & Kensinger, 2016) emotional information present in the retrieval cue. In Bowen and Kensinger (2016), there also was no instruction for participants to specifically encode or retrieve details of the emotional information. In both of these studies, analyses ruled out the possibility that these effects were better explained by a main effect of arousal, rather than valence. Retrieval-related recapitulation was also not likely to be confounded by low-level visual features, because in Bowen and Kensinger (2016), the valence effects occurred even when comparing trials on which the only difference across valence types was the expression of a face displayed during encoding (see Supplementary Discussion for more consideration of possible confounds).

Taken together, the literature reviewed suggests that negative valence may enhance both the way that sensory cortices are incorporated into emotional memory networks at the time of encoding (and see Murty et al., 2011 for discussion) and also the degree to which sensory regions are recapitulated at the time of retrieval. The findings contribute to the larger recapitulation literature cited previously, providing further evidence for a link between recapitulation in the ventral visual stream and successful memory retrieval of episodic content (e.g., Waldhauser et al., 2016). At the same time, they emphasize that not all types of episodic memories are equally likely to be retrieved with accompanying sensory recapitulation: Even when memory accuracy does not differ as a function of valence (see Bowen & Kensinger, 2016 for further discussion), sensory recapitulation can be enhanced for negative memories and can be more likely linked to recollection of negative events.

A new model of emotional memory that emphasizes the importance of valence

The literature reviewed suggests that arousal can have different (Beyeler et al., 2016), even opposing (Mickley Steinmetz, Addis, & Kensinger, 2010), effects on the emotional memory network depending on the valence of the stimuli. And even when events are matched on arousal, there can be different behavioral profiles (see top portion of Table 2 for examples), and consistently different neural profiles (see Fig. 1), associated with memories for positive and negative events. These different neural profiles include differences in amygdala connectivity (Beyeler et al., 2016) and, in studies of human episodic memory, suggest that connections between the amygdala and sensory processing regions may be strengthened not merely by arousal but by arousal associated with a negative event (Kark & Kensinger, 2015; Mickley Steinmetz, Addis, & Kensinger, 2010).

Some of these effects of valence may occur relatively quickly; a recent study (Xie & Zhang, 2016) suggested that negative valence enhances the resolution of visual information held in working memory. Such a pattern suggests that – even when memory storage is only required for a few seconds – valence can affect the fidelity of stored, sensory details. One possibility to explain these effects is that valence may affect the re-entrant processing of visual information. Re-entrant signals can enter sensory cortices from many regions, including the amygdala, and have been proposed to tune the properties of neurons within visual cortices to make them more sensitive to emotional content (e.g., Vuilleumier & Driver, 2007). Although such re-entrant processing generally has been proposed to occur for arousing stimuli, the extant data raise the intriguing possibility that such re-entrant effects may be exaggerated for negative stimuli with high arousal.

Although asymmetries in the impact of positive and negative emotions have frequently been described (e.g., Baumeister et al., 2001; Bless & Schwarz, 1999; Ohira et al., 1998; Robinson-Riegler & Winton, 1996; Storbeck & Clore, 2005; Taylor, 1991), there have not been mechanistic accounts of these asymmetries within the episodic memory literature. Yet the extant literature suggests the need for a model of emotional memory that emphasizes the importance of event valence. We do not question that theories that focus on emotional arousal may sometimes be sufficient: For instance, there is compelling evidence that the memory-narrowing effects of emotion may be dependent on arousal rather than valence (e.g., Mather & Sutherland, 2011). Nevertheless, there is mounting evidence that valence can affect the behavioral profiles and neural correlates of episodic memory. The effects of emotional valence appear to be particularly pronounced within sensory regions, suggesting that valence-focused models of emotional memory may become particularly relevant when trying to understand the ways that sensory processes become incorporated in emotional memory networks. This perspective broadens the focus of models of emotional memory: Most existing models have honed in on the role of the amygdala, or on interactions between the amygdala and other medial temporal-lobe regions (e.g., McGaugh, 2000; Phelps, 2004; Yonelinas & Ritchey, 2015), while less emphasis has been given to the previously-reviewed evidence that sensory processes can be strongly affected by emotion, and specifically by emotional valence.

When considering the plausible accounts for the effects of valence on the incorporation of sensory processes into emotional memory networks, we were inspired by the aforementioned articles on reactivation and recapitulation. There is no doubt that emotion affects the way that events are experienced, encoded, and initially consolidated (McGaugh, 2000; Murty et al., 2011). Yet the phenomena of reactivation and recapitulation – coupled with evidence for the modification or strengthening of negative emotional memories over time (Dunsmoor et al., 2015; Kensinger, 2015; Yonelinas & Ritchey, 2015) – suggest a need to consider longer time-courses for the effects of emotion on memory. The incorporation of reactivation and recapitulation into models of emotional memory may prove particularly fruitful for explaining how negative emotional memories develop a rich phenomenology and for understanding the durability of negative emotional memories.

Our consideration of the literature described above has led us to propose a new model: NEVER (Fig. 2). This model builds off of the evidence that negative valence has effects on encoding, storage, and retrieval (Fig. 1). This model emphasizes the importance of considering (a) effects of emotional valence on memory, (b) effects of emotion that extend long past the initial encoding episode, and (c) the importance of considering how sensory processes are integrated into emotional memory networks.

Fig. 2
figure 2

Negative Emotional Valence Enhances Recapitulation. The figure presents the hypothesized links between negative valence and encoding, storage, retrieval, and behavioral outcomes (black arrows) and the interconnections among memory phases that are predicted to be intensified by negative valence (gray symbols). The numbers refer to portions of the model that have received some support from data in the literature (see first column of Fig. 3) while the letters refer to portions of the model that remain to be tested (see column 3 of Fig. 3). The gray arrow and question-mark between memory for sensory details and subjective feeling of re-experience indicates a possible link, but one that may be subject to boundary conditions. The gray arrows back from recapitulation reflect the ability for retrieval to serve as an opportunity both to re-encode details and to reactivate consolidation processes. The question-marks on those lines signify uncertainty as to whether negative valence enhances the strength of those links

Full size image

Description of model: Negative Emotional Valence Enhances Recapitulation (NEVER)

We sought to develop a model of emotional memory that would incorporate the following features:

  • Provide an explanation for valence differences in memory, in the absence of stimulus confounds or arousal differences. Many models of emotional memory suggest that when valence differences in memory are revealed, they are driven by confounds: negative stimuli may be remembered better because they are more arousing, or because they have more memorable or distinctive features. This may sometimes (even often) be the case, but we propose that there are other instances when valence differences in memory cannot be explained by such confounds.

  • Incorporate mechanisms that operate beyond encoding and early-stage consolidation, and emphasize interactions across phases of memory.

  • Extend the focus beyond the amygdala, hippocampus, and other medial temporal regions, while not denying key roles for these regions.

The specific model we developed was based on our review of the literature, and on the results from Kark and Kensinger (2015) and Bowen and Kensinger (2016). This model, depicted in Fig. 2, proposes that negative valence enhances the encoding of sensory information, such that sensory details of negative events will become incorporated into memory traces more often than sensory details of positive events. The model further proposes that negative valence also will enhance the recapitulation of those sensory details at the time of retrieval. In other words, not only is the sensory information more likely to become incorporated into a memory trace for a negative event than a positive event, that sensory content also is more likely to be recapitulated at the time of retrieval. These differences may intensify the subjectively vivid phenomenology associated with negative memories.

Although the model proposes that these valence differences will exist even over short delays, the model further proposes that – over offline periods enabling consolidation processes to unfold optimally – the sensory features of negative events will be more likely to be reactivated than the sensory features of positive events. This reactivation will solidify the incorporation of sensory features within a negative memory trace and can lead to exaggerated valence differences in memory over longer delays.

It is relevant to note that the model includes two routes to the aforementioned valence differences. One route is that negative valence enhances the encoding and early storage of sensory information, and by doing so, it influences the information available when later storage and retrieval processes occur. If this were the only explanation of valence differences, then valence would have relatively early actions on memory processes, and those early actions would have downstream consequences that would lead to the effects of valence on storage, retrieval, and behavioral outcomes. In Fig. 2, this route is depicted by the black arrows in the model: Negative valence enhances sensory encoding, and because sensory encoding triggers downstream consolidation (likely via reactivation) and recapitulation, these too are enhanced by negative valence. This explanation is generally consistent with evidence that depth- or style-of-processing can affect the information that is encoded and later recapitulated (e.g., Rugg et al. 2008a, b). It also is generally consistent with other theories of emotional memory that ascribe most of the modulatory effects to early on in the learning or storage of the memories.

Where our model diverges more from extant models is through a second explanation, not mutually exclusive with the first, that negative valence enhances the links among phases; the gray symbols in the model reflect those enhancements. By this explanation, negative valence enhances the likelihood that encoded sensory details are consolidated (reactivated) and recapitulated at retrieval. This aspect of the model places more emphasis on later-stage processing than most models of emotional memory.

The model we propose incorporates both of these routes. We propose that negative valence enhances sensory-focused encoding, thereby affecting the types of information that are available as storage processes unfold and as retrieval occurs (black arrows). We also propose that negative valence enhances the likelihood that encoded information is reactivated during offline periods that enable consolidation or recapitulated during retrieval (gray symbols). More specifically, we propose that negative valence enhances the likelihood that sensory details are reactivated and recapitulated.

We further hypothesize that these effects of valence on recapitulation have consequences for the quality of a memory, even when they do not have consequences for the ability to discriminate retrieval cues as old or new. We propose that the enhanced recapitulation for negative memories will be linked to a richer subjective quality of a memory (e.g., the vividness or confidence in a memory) as well as to some objectively-quantifiable aspects of a memory (e.g., retrieval of some sensory event features).

The role of the amygdala in emotional memory has been well documented in prior research, and it is therefore worth noting how we think about amygdala processes within our model. We do not take the strong stance that all amygdala (or amygdala-hippocampal) modulation of memory relates to recapitulation. In fact, the literature suggests that many of the modulatory influences of the amygdala are likely to be related principally to arousal and to extend equally to negative and positive memories; thus, these modulatory influences would not factor into this model to explain valence differences in memory. However, we do propose that there may be an important exception to this valence-invariance in amygdala modulation: There may be valence differences in how amygdala and sensory regions interact during encoding – with negative valence associated with stronger connectivity than positive valence (Kark & Kensinger, 2015) – and these differences may set the stage for some of the valence-based differences in recapitulation that become apparent at the time of retrieval. Indeed, we presented evidence (Kark & Kensinger, 2015b) that during successful encoding, there is enhanced connectivity between the amygdala and right lateral occipital cortex as arousal increases for negative stimuli, but there is decreased connectivity between these regions as arousal increases for positive stimuli (see region of right lateral occipital cortex depicted in yellow in row 1 of Fig. 3). This result replicates the key finding of Steinmetz, Addis, and Kensinger (2011) capitalizing on within-subject variability in arousal ratings and neural activity for the analysis as compared to the between-subject variability tapped by Steinmetz and colleagues. Thus, the amygdala may play a key role in the first route to valence differences described at the outset of this section: When events are negative and arousing, amygdala connectivity to sensory regions may increase, and this connectivity may aide in the incorporation of sensory regions into memory traces that can later be reactivated and recapitulated.

Fig. 3
figure 3

A number of aspects of the NEVER model are supported by the datasets that led us to generate the model. Four aspects of the model are delineated in the first (left-most) column, along with a brief description (in the second column) of the evidence supporting each aspect. The numbers 1–4 are placed on the model (Fig. 2) to demarcate the portion of the model to which each of these four aspects relates. The model generates additional predictions that have not yet been tested. Here, we highlight four predictions (third column) and suggest the next steps for testing those predictions in the fourth (right-most) column. Letters A–D are placed on the model (Fig. 2) to demarcate the portion of the model to which each of these predictions relates. neg = negative, pos = positive

Full size image

Future directions

Some key aspects of this model are already reasonably well supported by the datasets that led us to propose this model. These aspects are outlined in the two leftmost columns of Fig. 3. Of course, further tests of these aspects – using different stimuli, methods, and participant samples – are required to determine the reproducibility and generality of the results. Other aspects of the model have not yet been tested; some of these, ripe for testing, are outlined in the two right columns of Fig. 3. In particular, the model generates a number of new questions about how valence influences the interactions among phases of memory; these are the questions that we highlight, because we think this focus on processes that unfold and interact over all phases of memory will be essential for an understanding of emotional modulation of memory.

We also denote by gray arrows and question-marks (Fig. 2) some of the open questions with regard to how these processes interact. For instance, the literature reviewed earlier emphasizes that retrieval can serve as a new opportunity for encoding and can trigger storage processes. There are insufficient data from which to generate hypotheses as to whether negative valence modulates these links: Does negative valence affect the likelihood that retrieval triggers re-encoding or storage processes? Because these iterative encoding-retrieval-encoding processes can enable memories to be updated and inferences to be made (Bridge & Voss, 2014; Zeithamova et al., 2012), understanding the role of emotional valence in these iterations is likely to be of importance for both basic and clinical research. Indeed, we began the title of our paper with “NEVER forget” both to serve as a mnemonic for our model’s acronym and because of the intriguing possibility that the enhanced recapitulation for negative information could lead to enhanced storage – and thus increased durability – for those memories. To the extent that negative memories show a shallower forgetting curve than positive memories (and see Wang, 2014 for some evidence), this effect could be explained by the enhanced reactivation and recapitulation of negative memories. However, retrieval can also trigger memory modification, or deletion. The reconsolidation literatureFootnote 2 reveals that modifications or deletions can occur for both appetitive and aversive memories (e.g., Lane et al., 2015; Reichelt & Lee, 2013) but there are scant assessments of valence differences within the human episodic memory literature. It therefore remains to be determined under what circumstances an enhancement of recapitulation processes will be beneficial to the durability of memories for negative events.

Possible clinical relevance

An overarching goal of this manuscript has been to present how advances in the study of human episodic memory can be applied to the study of emotional memory. We have been struck by the artificial separations often present between literatures studying emotional memory from the perspective of affective neuroscience and literatures studying episodic memory from the perspective of cognitive neuroscience. It is our hope that this review has revealed some of the interconnections between these literatures, and that this revelation will be useful to those interested in clinical and translational applications as well as those focused on basic science.

Emotional memory biases have been observed in a range of psychopathological conditions (for a review see Haas & Canli, 2008) and have been linked with atypical encoding, consolidation, and retrieval processes. For example, patients with depression tend to remember the bad and forget the good, and there is some evidence that these negative memory biases are associated with disruptions in sleep-dependent memory consolidation processes (Goerke, Muller, & Cohrs, 2017; Harrington, Pennington, & Durrant, 2017). Patients with post-traumatic stress disorder (PTSD) show enhanced memory for trauma-related stimuli and poorer memory for emotionally neutral stimuli compared to controls (Brewin, Kleiner, Vasterling, & Field, 2007) and aberrant activation of occipital cortex during negative memory retrieval (Whalley et al., 2013). Indeed, distressing sensory re-experiencing of negative memories is a common feature across a range of disparate psychiatric conditions, from PTSD and anxiety disorder to obsessive-compulsive disorder and eating disorders (Brewin, Gregory, Lipton, & Burgess, 2010). Together, clinical evidence suggests that the development and maintenance of psychological conditions is at least in part related to abnormalities across multiple phases of memory, and to alterations in the content of negative memories. The NEVER model provides a framework for researchers to understand and test for valence-specific recapitulation effects in healthy individuals or how they might go awry in psychopathology.

Concluding remarks

It is our hope that this model will serve as a launch-point for further research. We hope it will catalyze additional research aimed at understanding the importance of emotional valence than has historically occurred, and with more of a focus in understanding how emotion may affect processes that unfold long after the occurrence of the original event. Because many clinical disorders are marked by changes in how emotional events from long ago are remembered, this focus would appear to be one with clinical impact as well as basic science importance.