The relationship between eye movements and subsequent recognition: Evidence from individual differences and amnesia

Abstract

There is consistent agreement regarding the positive relationship between cumulative eye movement sampling and subsequent recognition, but the role of the hippocampus in this sampling behavior is currently unknown. It is also unclear whether the eye movement repetition effect, i.e., fewer fixations to repeated, compared to novel, stimuli, depends on explicit recognition and/or an intact hippocampal system. We investigated the relationship between cumulative sampling, the eye movement repetition effect, subsequent memory, and the hippocampal system. Eye movements were monitored in a developmental amnesic case (H.C.), whose hippocampal system is compromised, and in a group of typically developing participants while they studied single faces across multiple blocks. The faces were studied from the same viewpoint or different viewpoints and were subsequently tested with the same or different viewpoint. Our previous work suggested that hippocampal representations support explicit recognition for information that changes viewpoint across repetitions (Olsen et al., 2015). Here, examination of eye movements during encoding indicated that greater cumulative sampling was associated with better memory among controls. Increased sampling, however, was not associated with better explicit memory in H.C., suggesting that increased sampling only improves memory when the hippocampal system is intact. The magnitude of the repetition effect was not correlated with cumulative sampling, nor was it related reliably to subsequent recognition. These findings indicate that eye movements collect information that can be used to strengthen memory representations that are later available for conscious remembering, whereas eye movement repetition effects reflect a processing change due to experience that does not necessarily reflect a memory representation that is available for conscious appraisal. Lastly, H.C. demonstrated a repetition effect for fixed viewpoint faces but not for variable viewpoint faces, which suggests that repetition effects are differentially supported by neocortical and hippocampal systems, depending upon the representational nature of the underlying memory trace.

Introduction

Measures derived from eye movements have been used to characterize memory encoding, to reveal the nature of the representations that are stored in memory and to relate both to hippocampal function (reviewed by Hannula et al., 2010, Ryan and Cohen, 2003). The current investigation explored the contribution of eye movements to subsequent item memory as well as the role of the hippocampal system in eye movement sampling behavior and subsequent item memory.

Item recognition in healthy, neurologically intact individuals is likely supported by both the hippocampus as well as the neocortex, with the medial temporal lobe (MTL) cortices playing a central role (Brown and Aggleton, 2001, Cohen and Eichenbaum, 1993, Davachi, 2006, Mayes et al., 2007). Our recent work has indicated that recognition memory for items is more heavily dependent on the hippocampus when items are presented and encoded from differing viewpoints (Olsen et al., 2015). We reported that compared to control participants, H.C., an individual with developmental amnesia due to hippocampal system compromise, demonstrated intact recognition for faces that were repeatedly studied from the same viewpoint, and impaired recognition for faces that were studied from multiple viewpoints. These results suggest that item recognition memory was differentially supported by the neocortex and hippocampus within the same experimental paradigm, depending on the presentation format (fixed or variable viewpoints) of study items. More generally, such findings suggest that the hippocampal system provides the ability to flexibly bind the features within an item, and performs a relational binding function, which supports memory for items that are physically modified across study repetitions.

Eye movement sampling behavior is functional for learning and memory such that subsequent recognition is higher when viewers are allowed to move their eyes during encoding compared to when they are required to maintain fixation (Henderson, Williams, & Falk, 2005). Other work has shown that increased cumulative sampling of visual stimuli is associated with better subsequent recognition for items (Loftus, 1972) in both healthy younger and older adults (Chan et al., 2011, Firestone et al., 2007).

Despite research showing that eye movements are functional for the recognition of items, and that the hippocampus can contribute to item recognition, there is little research that examined the relationship between eye movement sampling behavior and later recognition for items that specifically depend on the hippocampus. In particular, it is unknown whether eye movement sampling would particularly benefit item representations that rely predominantly on hippocampal function. Moreover, it remains to be determined whether amnesic people would engage in more sampling behavior to compensate for impaired hippocampal function.

While considerable research suggests that eye movement sampling supports memory acquisition, another line of well-established research suggests that eye movement behavior can reflect the online expression of memory. Eye movement sampling is sensitive to prior experience, such that upon repeated exposures to an item, a repetition effect is observed: the previously viewed items are sampled with fewer eye fixations compared to novel items. This effect has been reported in numerous studies, and in various populations including younger adults, older adults, people with prosopagnosia, and memory-impaired individuals (Althoff and Cohen, 1999, Althoff et al., 1999, Bate et al., 2008, Heisz and Ryan, 2011, Ryan et al., 2000, Smith and Squire, 2008).

In contrast to the consistent positive relationship between cumulative eye movement sampling and recognition memory, the relationship between the eye movement repetition effect and recognition is not straightforward, nor is the relationship between the repetition effect and its underlying neural substrates. Some studies have reported that the repetition effect can occur in the absence of explicit recognition, whereas others have found that the repetition effect is eliminated in individuals who have impaired recognition, such as hippocampal amnesics (Ryan et al., 2000; cf. Smith & Squire, 2008). Moreover, some research has shown that eye movement repetition effects are hippocampal-dependent (Smith and Squire, 2008, Smith et al., 2006), whereas other work has reported intact repetition effects in hippocampal amnesia that are presumably driven by neocortical regions (Althoff and Cohen, 1999, Ryan et al., 2000). It may be the case that, depending on the particular paradigm, eye movement repetition effects can be supported by either hippocampal and/or neocortical memory representations. Paradigm differences may affect the relationship between eye movement repetition effects and subsequent recognition in healthy individuals, and influence the extent to which eye movement repetition effects are impaired in amnesia.

The current study investigated the relationship between cumulative eye movement sampling, the eye movement repetition effect, and subsequent recognition memory for faces, as well as the relation between the hippocampal system and these eye movement measures. We used the same paradigm as in our previous study (Olsen et al., 2015), which provided the novel opportunity to investigate both the acquisition and expression of memory, as indexed by eye movements, and their associations with recognition memory that can be supported differentially by hippocampal or neocortical memory representations. We tested H.C. and a group of demographically matched, typically developing adults to allow for a thorough exploration of the relationship between eye movements, the hippocampal system, and recognition.

The differential reliance on distinct MTL structures for face recognition on this task provides different predictions regarding the relationship between cumulative sampling and subsequent recognition. Consistent with previous literature, we predicted that increased eye movement sampling would be associated with better recognition memory for faces presented from a fixed viewpoint, in both H.C. and controls, as memory for fixed viewpoint faces can be supported by neocortex. Increased sampling was also predicted to benefit recognition memory for faces presented across variable viewpoints, but only for control participants, as our previous work indicated memory for faces which are presented across viewpoints depends on an intact hippocampal system (Olsen et al., 2015). In contrast, we expected that the relationship between sampling behavior and subsequent recognition for variable viewpoint faces would be “broken” in H.C., as the primary neural structure supporting those memory representations is compromised. If this is the case, then equivalent sampling for subsequently remembered and subsequently forgotten faces should be observed in H.C. Additionally, it was expected that H.C. would demonstrate greater cumulative eye movement sampling relative to controls during encoding, potentially as a compensatory mechanism to support later recognition.

The relationship between eye movement repetition effects and subsequent memory may similarly be influenced by differential contributions from the MTL cortex and hippocampus on the fixed viewpoint and variable viewpoint conditions, respectively. If the MTL cortex enables repetition effects for faces studied from a fixed viewpoint (Althoff and Cohen, 1999, Althoff et al., 1999), a positive relationship would be expected in both controls and H.C. If repetition effects for faces studied from a variable viewpoint are dependent on the hippocampus, a positive relationship would also be expected for controls but not for H.C. If, however, repetition effects for both types of face presentations hinge on conscious recognition, we would expect positive correlations between repetition effects and recognition for both fixed and variable viewpoint faces for H.C. and the controls. A final possibility is that repetition effects are unrelated to conscious recognition, in which case no correlation would be expected for explicit recognition of either fixed or variable viewpoint faces in either H.C. or controls.