Involuntary top-down control by search-irrelevant features: Visual working memory biases attention in an object-based manner

Abstract

Many everyday tasks involve successive visual-search episodes with changing targets. Converging evidence suggests that these targets are retained in visual working memory (VWM) and bias attention from there. It is unknown whether all or only search-relevant features of a VWM template bias attention during search. Bias signals might be configured exclusively to task-relevant features so that only search-relevant features bias attention. Alternatively, VWM might maintain objects in the form of bound features. Then, all template features will bias attention in an object-based manner, so that biasing effects are ranked by feature relevance. Here, we investigated whether search-irrelevant VWM template features bias attention. Participants had to saccade to a target opposite a distractor. A colored cue depicted the target prior to each search trial. The target was predefined only by its identity, while its color was irrelevant. When target and cue matched not only in identity (search-relevant) but also in color (search-irrelevant), saccades went more often and faster directly to the target than without any color match (Experiment 1). When introducing a cue-distractor color match (Experiment 2), direct target saccades were most likely when target and cue matched in the search-irrelevant color and least likely in case of a cue-distractor color match. When cue and target were never colored the same (Experiment 3), cue-colored distractors still captured the eyes more often than different-colored distractors despite color being search-irrelevant. As participants were informed about the misleading color, the result argues against a strategical and voluntary usage of color. Instead, search-irrelevant features biased attention obligatorily arguing for involuntary top-down control by object-based VWM templates.

Introduction

Many everyday tasks involve successive visual-search episodes with changing targets. Before writing a note you have to look not only for a pencil, but also for a piece of paper in your vicinity. Surprisingly, mechanisms of visual search have mostly been investigated by means of experiments with constant search targets throughout a series of search trials (Treisman and Gelade, 1980, Wolfe, 1994, Wolfe, 2010, Zelinsky, 2012). In this case, due to repetition of the same search target in each trial, a representation of relevant features of the target object can be recoded and retained in long-term memory (LTM, Woodman, Carlisle, & Reinhart, 2013). If the target changes from one search episode to the next, as is the case in most everyday tasks, time might not suffice to recode a representation of the search target into an LTM template. Instead, a short-term memory (STM) also called working memory (WM) representation of the search target should be formed after target specification and maintained until the target is found (Woodman et al., 2013). This working memory representation of the search target is often called search template (e.g., Olivers, Peters, Houtkamp, & Roelfsema, 2011).

Experimental evidence differentiates between STM/WM-based search for trial-wise varying targets and LTM-based search for constant targets over trials. Firstly, searches for changing targets are less efficient than searches for a constant target (Bravo and Nakayama, 1992, Wolfe et al., 2004). Second, there is evidence from human and non-human primate electrophysiology that STM and LTM representations of a search object differ. Woodman et al. (2013) measured contralateral-delay activity (CDA) while humans performed visual searches. The CDA is a relative negativity at parietal, occipital, and temporal electrodes that is supposed to indicate maintenance of visual representations in WM (Vogel & Machizawa, 2004). Importantly, CDA activity stayed high throughout a series of searches with changing targets, while it decreased over the course of repeated searches for a constant target. Furthermore, in a monkey study by Rossi, Harris, Bichot, Desimone, and Ungerleider (2001), constant target searches were not affected by prefrontal lesions, while changing target searches were impaired. Thus, WM templates for changing search targets and LTM templates for constant search targets are retained differently in the primate brain (Woodman et al., 2013).

How might a visual search target be retained in WM? A major viewpoint of visual working memory (VWM) stresses (D’Esposito & Postle, 2015) that transiently available visual information needs to be kept active over a short period of time until it is used for a subsequent action. VWM can be considered as a capacity-limited visual memory system with short-term retention and manipulation functions (the visuo-spatial sketchpad in Baddeley, 1986, Baddeley and Hitch, 1974). Its capacity seems to be limited to three to four items (Cowan, 2000, Fukuda et al., 2010, Luck and Vogel, 1997, Luck and Vogel, 2013, but see also Ma, Husain, & Bays, 2014). Importantly, the VWM capacity is limited more by the number of objects rather than by the number of visual features (Luck and Vogel, 1997, Vogel et al., 2001; but see also Alvarez and Cavanagh, 2004, Hardman and Cowan, 2015, Palmer et al., 2015). Based on these findings, some theories assume that bound features of an object rather than separated features are represented in VWM (Luck and Vogel, 1997, Luck and Vogel, 2013). A similar assumption has been raised by theories that emphasize the binding function of attention (Kahneman, Treisman, & Gibbs, 1992) and theories that view visual attention and working memory in the service of action control (Schneider, 1995, Schneider, 2013) and advanced cognitive operations (Oberauer, 2009).

Given these considerations on the nature of VWM, for visual search tasks with changing targets (e.g., trial-wise), a template of a search target could be retained in VWM as a visual object with integrated features. Alternatively, the template in VWM could contain only search-relevant features of the search target. In the former case, that is, if an object with integrated features is stored as search template in VWM, then not only task-relevant but also task-irrelevant features might exert an attentional biasing effect. This consideration seems plausible if VWM objects are conceptualized as bidirectional loops between visual feature maps and a VWM map with a limited number (about 4) of object pointers (Bundesen, Habekost, & Kyllingsbaek, 2005). Feedback connections from the VWM map could mediate the attentional bias effect in the visual feature maps. In this case, all features of a VWM template are likely kept active through these reverberating loops and should bias attentional selection in an object-based manner. Consequently, even completely irrelevant features of an active VWM representation should bias attention via involuntary top-down control signals such as a bias by color of a VWM template when searching for a template-matching shape. So far, this question has not been studied experimentally, that is, whether all or only relevant features of a VWM template bias attention during the corresponding visual search with trial-wise changing targets.

In the present study, we demonstrate in three experiments that a search-irrelevant feature, here color, of a VWM template biases attentional selection, arguing for involuntary object-based top-down control by irrelevant features. In all experiments, a search trial started with a colored cue depicting the search object at fixation. Afterwards, two colored objects appeared left and right from fixation. Participants had to saccade to the search object according to its identity and irrespective of its color. Overt saccadic eye movements were used as proxies for covert visual attention as it is known that a covert shift of attention obligatorily precedes each saccade (Deubel & Schneider, 1996). In a first experiment, the target object appeared in the same color as the search cue in half of the trials. Although object identity and not color defined the target, first saccades went more often and faster towards the target if it appeared in the task-irrelevant cue’s color rather than in any other color. As the distractor did never appear in the cue color, color might have been strategically used for search. In a second experiment, we therefore included a cue-distractor color-match condition. Cue-colored targets and distractors captured the eyes when they were accompanied by different-colored counterparts. Still, participants might have used the color voluntarily, as color was equally often valid and invalid. Therefore, in a third experiment, the target never appeared in the irrelevant cue color, while the distractor matched the cue color in half the trials. Furthermore, participants were informed that the cue color was never informative and even misleading in fifty percent of the trials. Although, it would have been strategically beneficial to ignore any cue-colored object in this case, first saccades landed still more often on a cue-colored than on a different-colored distractor. The results of these three experiments strongly support the view that VWM-search templates work obligatorily in an object-based manner during visual search tasks with trial-wise changing targets. All features of a VWM template, including the irrelevant features, seem to bias attentional selection due to an involuntary top-down regulation. Task knowledge cannot be used to eliminate the attentional and oculomotor capture from search-irrelevant template features completely.