Original ArticlesInvoluntary top-down control by search-irrelevant features: Visual working memory biases attention in an object-based manner
Introduction
Many everyday tasks involve successive visual-search episodes with changing targets. Before writing a note you have to look not only for a pencil, but also for a piece of paper in your vicinity. Surprisingly, mechanisms of visual search have mostly been investigated by means of experiments with constant search targets throughout a series of search trials (Treisman and Gelade, 1980, Wolfe, 1994, Wolfe, 2010, Zelinsky, 2012). In this case, due to repetition of the same search target in each trial, a representation of relevant features of the target object can be recoded and retained in long-term memory (LTM, Woodman, Carlisle, & Reinhart, 2013). If the target changes from one search episode to the next, as is the case in most everyday tasks, time might not suffice to recode a representation of the search target into an LTM template. Instead, a short-term memory (STM) also called working memory (WM) representation of the search target should be formed after target specification and maintained until the target is found (Woodman et al., 2013). This working memory representation of the search target is often called search template (e.g., Olivers, Peters, Houtkamp, & Roelfsema, 2011).
Experimental evidence differentiates between STM/WM-based search for trial-wise varying targets and LTM-based search for constant targets over trials. Firstly, searches for changing targets are less efficient than searches for a constant target (Bravo and Nakayama, 1992, Wolfe et al., 2004). Second, there is evidence from human and non-human primate electrophysiology that STM and LTM representations of a search object differ. Woodman et al. (2013) measured contralateral-delay activity (CDA) while humans performed visual searches. The CDA is a relative negativity at parietal, occipital, and temporal electrodes that is supposed to indicate maintenance of visual representations in WM (Vogel & Machizawa, 2004). Importantly, CDA activity stayed high throughout a series of searches with changing targets, while it decreased over the course of repeated searches for a constant target. Furthermore, in a monkey study by Rossi, Harris, Bichot, Desimone, and Ungerleider (2001), constant target searches were not affected by prefrontal lesions, while changing target searches were impaired. Thus, WM templates for changing search targets and LTM templates for constant search targets are retained differently in the primate brain (Woodman et al., 2013).
How might a visual search target be retained in WM? A major viewpoint of visual working memory (VWM) stresses (D’Esposito & Postle, 2015) that transiently available visual information needs to be kept active over a short period of time until it is used for a subsequent action. VWM can be considered as a capacity-limited visual memory system with short-term retention and manipulation functions (the visuo-spatial sketchpad in Baddeley, 1986, Baddeley and Hitch, 1974). Its capacity seems to be limited to three to four items (Cowan, 2000, Fukuda et al., 2010, Luck and Vogel, 1997, Luck and Vogel, 2013, but see also Ma, Husain, & Bays, 2014). Importantly, the VWM capacity is limited more by the number of objects rather than by the number of visual features (Luck and Vogel, 1997, Vogel et al., 2001; but see also Alvarez and Cavanagh, 2004, Hardman and Cowan, 2015, Palmer et al., 2015). Based on these findings, some theories assume that bound features of an object rather than separated features are represented in VWM (Luck and Vogel, 1997, Luck and Vogel, 2013). A similar assumption has been raised by theories that emphasize the binding function of attention (Kahneman, Treisman, & Gibbs, 1992) and theories that view visual attention and working memory in the service of action control (Schneider, 1995, Schneider, 2013) and advanced cognitive operations (Oberauer, 2009).
Given these considerations on the nature of VWM, for visual search tasks with changing targets (e.g., trial-wise), a template of a search target could be retained in VWM as a visual object with integrated features. Alternatively, the template in VWM could contain only search-relevant features of the search target. In the former case, that is, if an object with integrated features is stored as search template in VWM, then not only task-relevant but also task-irrelevant features might exert an attentional biasing effect. This consideration seems plausible if VWM objects are conceptualized as bidirectional loops between visual feature maps and a VWM map with a limited number (about 4) of object pointers (Bundesen, Habekost, & Kyllingsbaek, 2005). Feedback connections from the VWM map could mediate the attentional bias effect in the visual feature maps. In this case, all features of a VWM template are likely kept active through these reverberating loops and should bias attentional selection in an object-based manner. Consequently, even completely irrelevant features of an active VWM representation should bias attention via involuntary top-down control signals such as a bias by color of a VWM template when searching for a template-matching shape. So far, this question has not been studied experimentally, that is, whether all or only relevant features of a VWM template bias attention during the corresponding visual search with trial-wise changing targets.
In the present study, we demonstrate in three experiments that a search-irrelevant feature, here color, of a VWM template biases attentional selection, arguing for involuntary object-based top-down control by irrelevant features. In all experiments, a search trial started with a colored cue depicting the search object at fixation. Afterwards, two colored objects appeared left and right from fixation. Participants had to saccade to the search object according to its identity and irrespective of its color. Overt saccadic eye movements were used as proxies for covert visual attention as it is known that a covert shift of attention obligatorily precedes each saccade (Deubel & Schneider, 1996). In a first experiment, the target object appeared in the same color as the search cue in half of the trials. Although object identity and not color defined the target, first saccades went more often and faster towards the target if it appeared in the task-irrelevant cue’s color rather than in any other color. As the distractor did never appear in the cue color, color might have been strategically used for search. In a second experiment, we therefore included a cue-distractor color-match condition. Cue-colored targets and distractors captured the eyes when they were accompanied by different-colored counterparts. Still, participants might have used the color voluntarily, as color was equally often valid and invalid. Therefore, in a third experiment, the target never appeared in the irrelevant cue color, while the distractor matched the cue color in half the trials. Furthermore, participants were informed that the cue color was never informative and even misleading in fifty percent of the trials. Although, it would have been strategically beneficial to ignore any cue-colored object in this case, first saccades landed still more often on a cue-colored than on a different-colored distractor. The results of these three experiments strongly support the view that VWM-search templates work obligatorily in an object-based manner during visual search tasks with trial-wise changing targets. All features of a VWM template, including the irrelevant features, seem to bias attentional selection due to an involuntary top-down regulation. Task knowledge cannot be used to eliminate the attentional and oculomotor capture from search-irrelevant template features completely.
Section snippets
Experiment 1
In Experiment 1, we investigated whether the search-irrelevant color feature of a trial-wise varying target biases covert attention during visual search. Participants had to saccade to a target object in a search display. A search cue indicated the identity of the target object prior to each trial. Therefore, the search cue, that is, the target identity varied from trial to trial. The cue object and the search objects were always colored. However, color was not relevant for visual search
Experiment 2
As in Experiment 1, participants had to saccade to a target object defined by its identity and not by its color. To investigate whether a search-irrelevant VWM-template feature does even bias attention towards a distractor, not only the target, but also the distractor matched the cue’s color in some trials. In addition, we added a condition in which both, target and distractor matched the search-irrelevant cue color. This condition was introduced to reveal whether faster saccadic responses are
Experiment 3
In Experiments 1 and 2, participants might have voluntarily used color to bias attention allocation towards cue-colored items. Participants had not been informed about the color-match distributions in the previous two experiments and the target matched the cue’s color in half of the trials. In addition, target selection according to object identity was a difficult task indicated by the performance level of about 60% correct first saccades in the no color-match baseline. Thus, biasing attention
General discussion
When a search task requires changing the search target from one search to the next, humans seem to rely on VWM to bias attention towards template-matching objects in the search region (Conci et al., 2013, Hollingworth, 2012, Wolfe, 1994, Woodman and Chun, 2006). While top-down bias signals are configured to favor task-relevant features (Bundesen, 1990, Desimone and Duncan, 1995, Foerster and Schneider, 2015), whole objects in terms of bound features seem to be retained in VWM (Luck and Vogel,
Acknowledgement
This research was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG).
References (62)
- et al.
A neural theory of visual attention and short-term memory (NTVA)
Neuropsychologia
(2011) - et al.
Saccade target selection and object recognition: Evidence for a common attentional mechanism
Vision Research
(1996) - et al.
Discrete capacity limits in visual working memory
Current Opinion in Neurobiology
(2010) - et al.
The reviewing of object files: Object-specific integration of information
Cognitive Psychology
(1992) - et al.
Visual working memory capacity: From psychophysics and neurobiology to individual differences
Trends in Cognitive Sciences
(2013) Design for a working memory
- et al.
Different states in visual working memory: When it guides attention and when it does not
Trends in Cognitive Sciences
(2011) - et al.
Automatic guidance of attention from working memory
Trends in Cognitive Sciences
(2008) - et al.
Working memory can guide pop-out search
Vision Research
(2006) - et al.
A feature-integration theory of attention
Cognitive Psychology
(1980)
Visual search
Current Biology
How fast can you change your mind? The speed of top-down guidance in visual search
Vision Research
The capacity of visual short-term memory is set both by visual information load and by number of objects
Psychological Science
Working memory
The role of attention in different visual-search tasks
Perception and Psychophysics
A theory of visual attention
Psychological Review
A neural theory of visual attention: Bridging cognition and neurophysiology
Psychological Review
Object-based implicit learning in visual search: Perceptual segmentation constrains contextual cueing
Journal of Vision
The magical number 4 in short-term memory: A reconsideration of mental storage capacity Dep
Behavioral and Brain Sciences
The cognitive neuroscience of working memory
Annual Review of Psychology
Neural mechanisms of selective visual attention
Annual Review of Neuroscience
Interactions between visual working memory and selective attention
Psychological Science
Competition in visual working memory for control of search
Visual Cognition
Long-term memory-based control of attention in multi-step tasks requires working memory: Evidence from domain-specific interference
Frontiers in Psychology
Expectation violations in sensorimotor sequences: Shifting from LTM-based attentional selection to visual search
Annals of the New York Academy of Sciences
Standard errors and confidence intervals in within-subjects designs: Generalizing Loftus and Masson (1994) and avoiding the biases of alternative accounts
Psychonomic Bulletin and Review
Object-based encoding in visual working memory: Evidence from memory-driven attentional capture
Scientific Reports
Do the contents of working memory capture attention? Yes, but cognitive control matters
Journal of Experimental Psychology: Human Perception and Performance
Remembering complex objects in visual working memory: Do capacity limits restrict objects or features?
Journal of Experimental Psychology. Learning, Memory, and Cognition
Guidance of visual search by memory and knowledge
Cited by (33)
Negative cues minimize visual search specificity effects
2022, Vision ResearchCitation Excerpt :Early attention seems to be highly influenced by VWM contents, whereas later attention may be more heavily influenced by goals and the other items in the search display. Also assessing the early direction of attention, Foerster & Schneider (2018) cued participants with colored realistic silhouettes; however, color was irrelevant to search. They observed a decreased likelihood of the first search saccade going to the target and an increase in initial saccade latency when the cue matched the color of the distractor in the two-item search display (Foerster & Schneider (2018)).
Object features reinstated from episodic memory guide attentional selection
2020, CognitionCitation Excerpt :The percentage of choice errors in the no-match condition was intermediate (3.3%). We replicated the results from Foerster and Schneider (2018) with our stimulus set. In the saccade task, correct target selection was facilitated when the target in the choice display appeared in the same color as in the memory display, although color was irrelevant.
Hide and seek: Directing top-down attention is not sufficient for accelerating conscious access
2020, CortexCitation Excerpt :Consequently, only the to-be-remembered feature dimension of a memorized object enhances conscious access of matching visual input (Experiment 5 of Gayet et al., 2013; Experiment 3 of Gayet, Brascamp, Van der Stigchel, & Paffen, 2015), and attentional capture by matching visual input (e.g., Experiment 4 of Olivers et al., 2006). In contrast, when observers are instructed to search for a particular target shape (independent of its color), as cued by a colored shape, search times are reduced when the eventual (shape-matching) target also matches the incidental color of the cued shape (Foerster & Schneider, 2018). This reveals that, unlike the case of a memorization instruction, in case of a search instruction, the attentional template can encompass more feature dimensions than only the feature dimension that is explicitly cued.
The contents of visual working memory delay the perceived offset of matching visual stimuli
2019, Acta Psychologica