Eye movements during visual mental imagery

doi:10.1016/S1364-6613(02)01931-9

Trends in Cognitive Sciences

Volume 6, Issue 7, 1 July 2002, Pages 271-272

https://doi.org/10.1016/S1364-6613(02)01931-9 Get rights and content

Abstract

It has long been debated whether eye movements play a functional role in visual mental imagery. A recent paper by Laeng and Teodorescu presents new evidence that eye movements are stored as a spatial index that is used to arrange the component parts correctly when mental images are generated.

References (7)

B. Laeng et al.
Eye scanpaths during visual imagery reenact those of perception of the same visual scene
Cogn. Sci.
(2002)
D. Noton et al.
Scanpaths in eye movements during pattern perception
Science
(1971)
D. Noton et al.
Scanpaths in saccadic eye movements while viewing and recognizing patterns
Vis. Res.
(1971)

There are more references available in the full text version of this article.

Cited by (62)

Perceptual reality monitoring: Neural mechanisms dissociating imagination from reality
2022, Neuroscience and Biobehavioral Reviews
Citation Excerpt :
However, while retinal invariance might prove a useful cue to the absence of external input, the presence of eye gaze-contingent effects is not a reliable cue to its presence. Perhaps surprisingly, mental imagery is often also accompanied by content-specific eye-movements (Gurtner et al., 2021; Martarelli and Mast, 2021; Mast and Kosslyn, 2002) and while the changes in external input associated with eye-movements should not influence purely internally generated sensory representations, recent findings have shown that internally generated representations are also re-mapped during eye-movements (Brincat et al., 2021). This means that the way (eye) movements alter sensory processing might be similar during perception and imagery and unless a reality monitoring system has direct access to what is happening at the retina, this might not tell apart imagination and reality.
There is increasing evidence that imagination relies on similar neural mechanisms as externally triggered perception. This overlap presents a challenge for perceptual reality monitoring: deciding what is real and what is imagined. Here, we explore how perceptual reality monitoring might be implemented in the brain. We first describe sensory and cognitive factors that could dissociate imagery and perception and conclude that no single factor unambiguously signals whether an experience is internally or externally generated. We suggest that reality monitoring is implemented by higher-level cortical circuits that evaluate first-order sensory and cognitive factors to determine the source of sensory signals. According to this interpretation, perceptual reality monitoring shares core computations with metacognition. This multi-level architecture might explain several types of source confusion as well as dissociations between simply knowing whether something is real and actually experiencing it as real. We discuss avenues for future research to further our understanding of perceptual reality monitoring, an endeavour that has important implications for our understanding of clinical symptoms as well as general cognitive function.
Early is left and up: Saccadic responses reveal horizontal and vertical spatial associations of serial order in working memory
2021, Cognition
Maintaining serial order in working memory is crucial for cognition. Recent theories propose that serial information is achieved by positional coding of items on a spatial frame of reference. In line with this, an early-left and late-right spatial-positional association of response code (SPoARC) effect has been established. Various theoretical accounts have been put forward to explain the SPoARC effect (the mental whiteboard hypothesis, conceptual metaphor theory, polarity correspondence, or the indirect spatial-numerical association effect). Crucially, while all these accounts predict a left-to-right orientation of the SPoARC effect, they make different predictions regarding the direction of a possible vertical SPoARC effect. In this study, we therefore investigated SPoARC effects along the horizontal and vertical spatial dimension by means of saccadic responses. We replicated the left-to-right horizontal SPoARC effect and established for the first time an up-to-down vertical SPoARC effect. The direction of the vertical SPoARC effect was in contrast to that predicted by metaphor theory, polarity correspondence, or by the indirect spatial-numerical association effect. Rather, our results support the mental whiteboard-hypothesis, according to which positions can be flexibly coded on an internal space depending on the task demands. We also found that the strengths of the horizontal and vertical SPoARC effects were correlated, showing that some people are more prone than others to use spatial references for position coding. Our results therefore suggest that context templates used for position marking are not necessarily spatial in nature but depend on individual strategy preferences.
“Looking-at-nothing” during sequential sensorimotor actions: Long-term memory-based eye scanning of remembered target locations
2018, Vision Research
Citation Excerpt :
This result indicates that participants used the LTM information acquired during the visual-targets phase to shift attention and gaze sequentially to the empty target locations in the memory-based action phase. Taken together, when the motor target locations were no longer visually marked, participants scanned the empty target locations on the screen – a behavior known as looking-at-nothing (Ferreira et al., 2008; Johansson et al., 2011; Laeng & Teodorescu, 2002; Mast & Kosslyn, 2002; Richardson & Spivey, 2000; Richardson et al., 2009), but less precise and less anticipatory than with visual targets available. The observed gaze patterns are in line with Foerster et al.'s (2012) cup-stacking study.
Before acting humans saccade to a target object to extract relevant visual information. Even when acting on remembered objects, locations previously occupied by relevant objects are fixated during imagery and memory tasks – a phenomenon called “looking-at-nothing”. While looking-at-nothing was robustly found in tasks encouraging declarative memory built-up, results are mixed in the case of procedural sensorimotor tasks. Eye-guidance to manual targets in complete darkness was observed in a task practiced for days beforehand, while investigations using only a single session did not find fixations to remembered action targets. Here, it is asked whether looking-at-nothing can be found in a single sensorimotor session and thus independent from sleep consolidation, and how it progresses when visual information is repeatedly unavailable. Eye movements were investigated in a computerized version of the trail making test. Participants clicked on numbered circles in ascending sequence. Fifty trials were performed with the same spatial arrangement of 9 visual targets to enable long-term memory consolidation. During 50 consecutive trials, participants had to click the remembered target sequence on an empty screen. Participants scanned the visual targets and also the empty target locations sequentially with their eyes, however, the latter less precise than the former. Over the course of the memory trials, manual and oculomotor sequential target scanning became more similar to the visual trials. Results argue for robust looking-at-nothing during procedural sensorimotor tasks provided that long-term memory information is sufficient.
Spontaneous ocular positioning during visual imagery in patients with hemianopia and/or hemineglect
2016, Neuropsychologia
Spontaneous eye movements during imagery are not random and can be used to study and reveal mental visualization processes (Fourtassi et al., 2013; Johansson et al. 2006). For example, we previously showed that during memory recall of French towns via imagery healthy individuals looks straight ahead when recalling Paris and their subsequent gaze positions are significantly correlated with the real GPS coordinates of the recalled towns. This correlation suggests that memory retrieval is done via depictive representations as it is never found when the towns are recalled using verbal fluency. In the present paper we added to this finding by showing that the mental image is spontaneously centered on the head or body midline.
In order to investigate the capacities of visual imagery in patients, and by extension, the role of primary visual cortex and fronto-parietal cortex in spatial visual imagery, we recorded gaze positions during memory recall of French towns in an imagery task, a non-imagery task (verbal fluency), and a visually-guided task in five patients with left or right hemianopia and in four patients with hemineglect (two with left hemianopia and two without).
The correlation between gaze position and real GPS coordinates of the recalled towns was significant in all hemianopic patients, but in patients with hemineglect this was only the case for towns located on the right half of the map of France. This suggests hemianopic patients can perform spatially consistent mental imagery despite direct or indirect unilateral lesions of the primary visual cortex. In contrast, the left-sided towns recalled by hemineglect patients, revealed that they have some spatial inconsistency or representational difficulty.
Hemianopic patients positioned and maintained their gaze in their contralesional hemispace, suggesting that their mental map was not centered on their head or body midline. This contralesional gaze positioning appeared to be a general compensation strategy and was not observed in patients with neglect (with or without hemianopia). Instead, neglect patients positioned their gaze in their ipsilesional hemispace and only when performing the visual imagery task.
These findings are discussed in the context of the role of occipital and fronto-parietal cortices in the neuroanatomical model of visual imagery developed by Kosslyn et al. (2006).
What do eyes reveal about the mind?. Algorithmic inference of search targets from fixations
2015, Neurocomputing
Citation Excerpt :
For example, it is known that eye movements follow the road tangent in driving [45], some saccades occur to avoid obstacles (predictive saccades in walking [54]), and eye movements are sensitive to the value of visual items [59]. Eye movements are also indicators of abstract thought processes, for instance in arithmetic and geometric problem solving [18], list sorting, and mental imagery [53]. These findings highlight the intricate links between the mind, the body׳s actions, and the world around us.
We address the question of inferring the search target from fixation behavior in visual search. Such inference is possible since during search, our attention and gaze are guided toward visual features similar to those in the search target. We strive to answer two fundamental questions: what are the most powerful algorithmic principles for this task, and how does their performance depend on the amount of available eye movement data and the complexity of the target objects? In the first two experiments, we choose a random-dot search paradigm to eliminate contextual influences on search. We present an algorithm that correctly infers the target pattern up to 50 times as often as a previously employed method and promises sufficient power and robustness for interface control. Moreover, the current data suggest a principal limitation of target inference that is crucial for interface design: if the target pattern exceeds a certain spatial complexity level, only a subpattern tends to guide the observers׳ eye movements, which drastically impairs target inference. In the third experiment, we show that it is possible to predict search targets in natural scenes using pattern classifiers and classic computer vision features significantly above chance. The availability of compelling inferential algorithms could initiate a new generation of smart, gaze-controlled interfaces and wearable visual technologies that deduce from their users׳ eye movements the visual information for which they are looking. In a broader perspective, our study shows directions for efficient intent decoding from eye movements.
Scrutinizing visual images: The role of gaze in mental imagery and memory
2014, Cognition
Gaze was monitored by use of an infrared remote eye-tracker during perception and imagery of geometric forms and figures of animals. Based on the idea that gaze prioritizes locations where features with high information content are visible, we hypothesized that eye fixations should focus on regions that contain one or more local features that are relevant for object recognition. Most importantly, we predicted that when observers looked at an empty screen and at the same time generated a detailed visual image of what they had previously seen, their gaze would probabilistically dwell within regions corresponding to the original positions of salient features or parts. Correlation analyses showed positive relations between gaze’s dwell time within locations visited during perception and those in which gaze dwelled during the imagery generation task. Moreover, the more faithful an observer’s gaze enactment, the more accurate was the observer’s memory, in a separate test, of the dimension or size in which the forms had been perceived. In another experiment, observers saw a series of pictures of animals and were requested to memorize them. They were then asked later, in a recall phase, to answer a question about a property of one of the encoded forms; it was found that, when retrieving from long-term memory a previously seen picture, gaze returned to the location of the part probed by the question. In another experimental condition, the observers were asked to maintain fixation away from the original location of the shape while thinking about the answer, so as to interfere with the gaze enactment process; such a manipulation resulted in measurable costs in the quality of memory. We conclude that the generation of mental images relies upon a process of enactment of gaze that can be beneficial to visual memory.

View all citing articles on Scopus

View full text

Trends in Cognitive Sciences

Research updateEye movements during visual mental imagery

Abstract

Cogn. Sci.

Scanpaths in eye movements during pattern perception

Science

Scanpaths in saccadic eye movements while viewing and recognizing patterns

Vis. Res.

Research update
Eye movements during visual mental imagery