Abstract
Attention operates perceptually on items in the environment, and internally on objects in visuospatial working memory. In the present study, we investigated whether spatial and temporal constraints affecting endogenous perceptual attention extend to internal attention. A retro-cue paradigm in which a cue is presented beyond the range of iconic memory and after stimulus encoding was used to manipulate shifts of internal attention. Participants’ memories were tested for colored circles (Experiments 1, 2, 3a, 4) or for novel shapes (Experiment 3b) and their locations within an array. In these experiments, the time to shift internal attention (Experiments 1 and 3) and the eccentricity of encoded objects (Experiments 2–4) were manipulated. Our data showed that, unlike endogenous perceptual attention, internal shifts of attention are not modulated by stimulus eccentricity. Across several timing parameters and stimuli, we found that shifts of internal attention require a minimum quantal amount of time regardless of the object eccentricity at encoding. Our findings are consistent with the view that internal attention operates on objects whose spatial information is represented in relative terms. Although endogenous perceptual attention abides by the laws of space and time, internal attention can shift across spatial representations without regard for physical distance.
Similar content being viewed by others
Imagine you are expecting friends to pick you up and you are looking for cars stopping in front of your house. When your friends unexpectedly arrive across the street, it may require several honks before you redirect your attention to them. In experimental settings, cuing paradigms use spatial cues to direct perceptual attention toward the location where an item is likely to appear. A vast perceptual cuing literature demonstrates that allocating attention to a location prior to stimulus onset enhances object detection and processing for the cued item (reviewed in Carrasco, 2011; Kastner & Ungerleider, 2000; Posner & Petersen, 1990). Spatial cues and subsequent allocation of attention also improve transfer into visuospatial working memory (VWM), resulting in improved performance for items appearing at attended locations (Schmidt, Vogel, Woodman, & Luck, 2002; Woodman, Vecera, & Luck, 2003).
The perceptual cuing literature testifies to the considerable quantity of research attempting to disentangle the orienting of endogenous and exogenous attention (e.g. Egeth & Yantis, 1997; Eriksen & Hoffman, 1972; Jonides, 1981; Nakayama & Mackeben, 1989; Posner, 1980; Treisman & Gelade, 1980). In general, the orienting of endogenous, voluntary, or top-down attention is initiated by a central cue that indicates the location of a possible future target. Conversely, exogenous, reflexive, or bottom-up attentional orienting is typically initiated by a brief peripheral cue that is presented at the location of the target. The debate has been ongoing as to whether shifts of endogenous and exogenous attention are independent of (Kwak et al., 1991; Remington & Pierce, 1984; Sagi & Julesz, 1985; Sperling & Weichselgartner, 1995) or dependent on (Egly & Homa, 1991; Rizzolatti, Riggio, Dascola, & Umiltà, 1987; Shulman, Remington, & Mclean, 1979; Tsal, 1983) spatial distance. Recent work by Chakravarthi and VanRullen (2011) has demonstrated that exogenous attention shifts quickly (100 ms) and is minimally dependent on distance, whereas endogenous attention shifts more slowly (250 ms) and depends on distance (Chakravarthi & VanRullen, 2011;, see also Carlson, Hogendoorn, & Verstraten, 2006). In the present article, we examined whether time and distance factors that are thought to influence endogenous perceptual cuing also extend to attentional cuing for items already in VWM.
The cuing literature has recently expanded after several published reports demonstrating that spatial cues presented after encoding improve VWM performance, even though no new information is provided to the participant (Berryhill, Richmond, Shay, & Olson, 2011; Dell'Acqua, Sessa, Toffanin, Luria, & Jolicoeur, 2010; Delvenne, Cleeremans, & Laloyaux, 2010; Griffin & Nobre, 2003; Landman, Spekreijse, & Lamme, 2003; Lepsien, Griffin, Devlin, & Nobre, 2005; Lepsien & Nobre, 2006; Lepsien & Nobre, 2007; Lepsien, Thornton, & Nobre, 2011; Makovski & Jiang, 2007; Makovski, Sussman, & Jiang, 2008; Matsukura, Luck, & Vecera, 2007; Nobre, 2008; Nobre, Rao, & Chelazzi, 2006; Sligte, Scholte, & Lamme, 2008, 2009; Sligte, Vandenbroucke, Scholte, & Lamme, 2010; Sligte, Wokke, Tesselaar, Scholte, & Lamme, 2011; Vandenbroucke, Sligte, & Lamme, 2011). In other words, spatial precues can shift attention to possible future targets, and spatial retro-cues can shift attention to the representations of items currently stored in VWM. We use the term perceptual attention to refer to endogenous attentional orienting and subsequent processing of items in the environment, and internal attention to refer to endogenous attentional processing of items in VWM. The majority of previous VWM experiments testing internal attention use retroactive cues (retro-cues) that are centrally presented at least 1,000 ms after encoding, but before the retrieval phase (Berryhill et al., 2011; Delvenne et al., 2010; Germeys et al., 2010; Griffin & Nobre, 2003; Landman et al., 2003; Lepsien et al., 2005; Lepsien & Nobre, 2006; Lepsien & Nobre, 2007; Lepsien et al., 2011; Makovski & Jiang, 2007; Makovski et al., 2008; Matsukura et al., 2007; Nobre, 2008; Nobre et al., 2006; Sligte et al., 2008, 2009; Sligte et al., 2010; Sligte et al., 2011; Vandenbroucke et al., 2011). On the face of it, the retro-cue appears to be largely an endogenous shift of internal attention. However, recent findings indicate that purely exogenous (abrupt onset retro-cues) and purely endogenous (number retro-cues mapped to a spatial location) cues do not elicit a retro-cue benefit; instead endogenous/exogenous hybrids such as arrow retro-cues are required (Berryhill et al., 2011). It is also essential to note that the retro-cue paradigm and associated attentional cuing effects are distinct from Sperling’s classic partial report research because the timing is well beyond the defined limits of iconic memory (<500 ms) (Sperling, 1960). The retro-cue paradigm’s longer delay interval (>900 ms) ensures that the cues are operating on the contents of VWM rather than on the icon. Across a variety of stimuli, timing, and design parameters, a growing literature reports robust VWM performance improvements (~5-17 %) provided by retro-cuing (Berryhill et al., 2011; Dell'Acqua et al., 2010; Delvenne et al., 2010; Germeys et al., 2010; Griffin & Nobre, 2003; Landman et al., 2003; Lepsien et al., 2005; Lepsien & Nobre, 2006; Lepsien & Nobre, 2007; Lepsien et al., 2011; Makovski & Jiang, 2007; Makovski et al., 2008; Matsukura et al., 2007; Nobre, 2008; Nobre et al., 2006; Sligte et al., 2008, 2009; Sligte et al., 2010; Sligte et al., 2011; Vandenbroucke et al., 2011).
There is currently some discussion regarding the degree of similarity between attentional shifts of perceptual and internal attention. Perceptual attention provides a benefit by enhancing the processing of cued items (Carrasco, 2011; Kastner & Ungerleider, 2000; Posner & Petersen, 1990). The mechanism of internal attention is less clear. Current hypotheses include enhancement of the VWM representation (Griffin & Nobre, 2003; Lepsien et al., 2005), improved ability to inhibit distractor items (Lepsien et al., 2005), increased VWM durability (Makovski et al., 2008), and/or greater protection from decay (Matsukura et al., 2007). The prima facie differences between perceptual and internal attention make it seem unlikely that there is a single underlying attentional mechanism (reviewed in Chun, Golomb, & Turk-Browne, 2011). However, recent neuroimaging findings show that VWM and perception-related activations are more similar than different. Several groups identify activations in early visual areas (V1–V4) during extended periods of VWM maintenance (Ester, Serences, & Awh, 2009; Harrison & Tong, 2009; Munneke, Heslenfeld, & Theeuwes, 2010; Serences, Ester, Vogel, & Awh, 2009). Specifically, Harrison and Tong (2009) and Serences et al. (2009) determined that multivoxel activity patterns exhibited by these early visual areas could be used to predict the contents of VWM (Harrison & Tong, 2009; Serences et al., 2009). Similar results were reported by Munneke et al. (2010), who demonstrated that VWM maintenance evoked retinotopically organized responses in early visual areas (V1–V3), indicating that orienting to spatial locations either perceptually, or in VWM, produced similar neural activity (Munneke et al., 2010). These data elegantly demonstrate that early visual areas are involved in more than visual perception, but instead they may also be key contributors to higher order functions such as VWM. Furthermore, event-related potentials (ERP) and fMRI studies comparing neural activations associated with shifts of attention within perception and VWM report highly similar activations in parietal and frontal regions (Griffin & Nobre, 2003; Kuo, Rao, Lepsien, & Nobre, 2009; Lepsien & Nobre, 2006; Nobre et al., 2004). Specifically, Kuo et al. (2009) used ERPs to demonstrate that perceptual and VWM target selection produced highly similar neural responses that followed a similar time course (Kuo et al., 2009). Behavioral research also reports similar performance improvements in studies directly comparing pre- and retro-cuing benefits associated with orienting perceptual (~14–19 %) and internal (~11–17 %) attention (Griffin & Nobre, 2003; Nobre et al., 2004). Thus, parsimony would suggest that pre- and retro-cues tap into similar attentional mechanisms.
However, not all of the behavioral data are consistent with the single-mechanism hypothesis. Berryhill et al. (2011) reported no retro-cue benefit when the number or peripheral retro-cues were used, even though these cues induce shifts of perceptual attention. Makovski and Jiang (2007) also noted behavioral differences while using single or multiple retro-cues to indicate one or more locations. They found that unlike perceptual attention, in which multiple precues provide performance benefits to multiple locations (e.g. Kramer & Hahn, 1995; Makovski & Jiang, 2007), there was no retro-cueing benefit when the retro-cue indicated more than one item. They concluded that internal attention is less flexible than perceptual attention and suggested that there are separable, partially overlapping mechanisms at work (Makovski & Jiang, 2007). Consequently, it is particularly important to identify differences between perceptual and internal attention that can then be used to clarify where the underlying attentional mechanisms differ.
In the present study, we tested two important aspects of endogenous perceptual orienting: the time course of an attentional shift and the relationship between eccentricity and attentional shifting. It takes time for perceptual attention to shift across space (reviewed in Carrasco, 2011; Egeth & Yantis, 1997; Kinchla, 1992; Posner, 1980). Since more time is provided to shift attention from a central cue to a peripheral target, attentional benefits increase slowly (as compared with exogenously cued attention) before reaching asymptote between 200 and 400 ms (Carlson et al., 2006; Chakravarthi & VanRullen, 2011; Cheal & Lyon, 1991; Eriksen & Collins, 1969; Muller & Rabbitt, 1989; Posner, Nissen, & Ogden, 1978; Shulman et al., 1979). Thus, precues improve perceptual performance over a measurable amount time. Research on iconic memory also shows that cuing benefits take time to develop (Becker, Pashler, & Anstis, 2000). Specifically, cues presented immediately after stimulus offset facilitate item detection rapidly, whereas benefits for item identification require more time (Becker et al., 2000). In the VWM domain, only two papers have described the development of the retro-cue benefit over time (Germeys et al., 2010; Landman et al., 2003). Research by both Landman (2003) and Germeys et al. (2010) has indicated that the retro-cue benefit takes time to develop (Germeys et al., 2010; Landman et al., 2003), but to our knowledge there are no studies specifically examining this question.
Another well-studied aspect of perceptual attention is the effect of eccentricity on shifts of attention. Shulman et al. (1979) showed that as the distance between fixation and the test location lengthened, the amount of time needed for attention to shift to the target increased linearly, indicating that perceptual attention shifts in an analog fashion (Shulman et al., 1979; but see (Kwak et al., 1991; Sagi & Julesz, 1985). In other words, a shift from one location to another is continuous and involves attention traveling though the intermediate locations (Shulman et al., 1979). The velocity of spatial attention was quantified by Tsal (1983), who established that perceptual attention shifts at a constant velocity of ~8 ms/º of visual angle (Tsal, 1983). These results are intuitive: short shifts require less time than large shifts. Consequently, the magnitude and the time course of attentional cuing benefits are predictable (reviewed in Carrasco, 2011; Egeth & Yantis, 1997; Kinchla, 1992; Posner, 1980). To our knowledge there are no parallel studies describing how time and distance interact to modulate shifts of attention among representations held in VWM. This is in contrast to the perceptual and long-term memory domains, which show effects of spatial distance. One well-known example from long-term memory research is that recollections involving traversing greater physical distances take longer amounts of time than those involving crossing shorter distances, even when the distance is task irrelevant (reviewed in Kosslyn, 1981). Therefore, spatial distance matters when completing endogenous shifts of perceptual attention and when retrieving episodic long-term memories, but it is not clear whether it matters when shifting attention between items in VWM.
In the present study, we investigated whether internal attention is constrained by time and distance. The predictions were straightforward: If shifts of internal attention are similar to shifts of endogenous perceptual attention, the retro-cue benefit should be greater when there is more time to switch attention. Second, as a corollary, increasing object eccentricity should increase the time required for internal shifts of attention. Alternatively, if perceptual and internal attention are subject to different constraints, we predict nonlinear effects of time and distance.
Experiment 1
Second delay effects on the retro-cue benefit
We used a retro-cue paradigm to investigate the time course over which the retro-cue benefit developed. If shifts of internal attention are time-consuming and related to the amount of spatial distance covered, as they are for endogenous shifts of perceptual attention, the magnitude of the retro-cue benefit should grow as the post-retro-cue delay duration increases.
Method
Experimental design and trial sequence
Two randomly interleaved retro-cue conditions were tested: neutral and valid. The neutral retro-cue (50 %) was uninformative and served as a control condition; the valid retro-cue (50 %) was always predictive of the test location. Neutral and valid retro-cues were equally likely.
Participants were instructed to remember the color and location conjunction of each stimulus presented in the memory display; see Fig. 1. They were informed that the neutral (X) retro-cues were not informative, but that the valid (arrow) retro-cues would indicate the to-be-probed location. Each trial began with a fixation cross (1,500 ms), followed by the memory display (300 ms). The memory display consisted of four equiluminant color patches out of a set of 10 possible colors. Although four items are not necessarily above the capacity of VWM (Cowan, 2001; Luck & Vogel, 1997), previous work by several groups has also used a set size of four (Berryhill et al., 2011; Dell'Acqua et al., 2010; Griffin & Nobre, 2003; Lepsien et al., 2005; Lepsien & Nobre, 2006; Lepsien & Nobre, 2007; Makovski & Jiang, 2007; Makovski et al., 2008; Matsukura et al., 2007; Nobre, 2008; Nobre et al., 2006). Secondly, Makovski et al. (2008) and colleagues have previously demonstrated that the magnitude of the retro-cue benefit is not modulated by memory load, meaning the retro-cue benefit does not increase with set sizes greater than four (Makovski et al., 2008). The stimuli were 3 ° × 3 ° of visual angle and were located at 6 ° from fixation. This was followed by a first delay period (1,000 ms). Next, either the neutral or valid retro-cue (100 ms) appeared at central fixation. The neutral cue was a capital “X” (2.2 ° W × 2.0 ° H of visual angle), and the valid cue was an arrow (3 ° L × 2.1 ° W of visual angle). The stimuli and cues were not presented at the same physical location to avoid the possibility of overwriting. A second delay period of variable duration (100, 200, 300, 400, 500, 600, 700 ms) followed the retro-cue. The second delay durations were randomized and equally probable. Next, a memory probe screen appeared. The probe screen preserved the spatial arrangement of the memory display by indicating stimulus locations with empty annuli as placeholders, whereas the probe location was filled with a stimulus from the memory array. This practice is often used in previous retro-cue literature (Becker et al., 2000; Berryhill et al., 2011; Landman et al., 2003; Sligte et al., 2008, 2009; Sligte et al., 2010; Sligte et al., 2011; Vandenbroucke et al., 2011). Recent research by Sligte et al. (2008) also demonstrated that perceptual organization of the memory probe does not reduce the retro-cue benefit (Sligte et al., 2008). The task was to decide whether the probe stimulus matched the object shown during encoding. Participants responded by pressing the “Y” key if the color-location conjunction matched (50 %) the original memory display, and by pressing the “N” key if it did not match (50 %). Responses were unspeeded. Participants completed 250 randomly selected trials in five 50-trial blocks that were separated by rest breaks. Prior to the experiment, participants conducted a 10 trial practice block.
Trial sequence (Experiments 1–3): After a fixation cross (1,500 ms), the four-item stimulus array appeared (Experiment 1, all items presented at 6 ° eccentricity; Experiment 1a, all items presented at 6 ° eccentricity; Experiment 2, items presented at 3 °, 5 °, 7 °, and 9 ° eccentricity; Experiment 3, items presented at 3 ° and 12 ° eccentricity). Stimulus array was followed by the first delay period (1,000 ms). Next, either a neutral or valid retro-cue flashed (100 ms) and was followed by a second delay (Experiment 1, 100, 200, 300, 400, 500, 600, 700 ms; Experiment 1a, 100, 300, 600 ms; Experiment 2, 300 ms; Experiment 3, 146.63, 159.96, 173.29, 186.62, 200 ms). After the second delay period, a probe item appeared, and participants judged whether it matched what had been shown in the stimulus array
Articulatory suppression
During all of the experiments, participants were instructed to repeat a single syllable three-letter word out loud throughout the experiment to avoid verbal encoding of the colors. A different word was specified at the beginning of each block.
Equipment
Participants were tested individually in a room with dimmed lighting. They sat approximately 57 cm from a 17-in. LCD computer monitor (Dell 1707 FPc, Samsung SyncMaster 172 N). The experiment was programmed in ePrime (Psychology Software Tools, PA, USA).
Participants
A total of 25 volunteers from the University of Nevada psychology subject pool participated (ages 18–23 years, M = 20.2, 10 male) in exchange for course extra credit. The Internal Review Board of the University of Nevada approved all of the experimental protocols.
Results
Across experiments, VWM raw accuracy was the primary dependent measure. Parallel analyses examining d’ and reaction time (RT) are not discussed because they were consistent with the accuracy data.
A repeated measures ANOVA with cue type (valid, neutral) and second delay duration (100, 200, 300, 400, 500, 600, 700 ms) as within-subjects factors revealed a main effect of cue type, F(1, 24) = 71.76, p < .001; see Fig. 2. As was expected, performance was significantly better in the valid retro-cue condition than in the neutral retro-cue condition. An interaction between cue type and delay duration was observed, F(6, 144) = 2.44, p = .03. The interaction was due to significantly greater performance in the valid retro-cue condition than in the neutral cue condition for all second delay durations greater than 300 ms (ps < .02, Bonferroni corrected pairwise comparisons). A second repeated-measures ANOVA comparing the 300–700 ms conditions confirmed that there was no interaction between cue type and delay duration, F(4, 96) = 1.56, p = .19. In summary, there was no difference in the magnitude of the retro-cue effect for second delay durations greater than 300 ms. Because there was a numerical difference in performance at 300 ms and 700 ms, we also conducted a paired t test between these two second delay durations and confirmed that there was no significant difference between them, t 24 = 1.15, p = .26.
Results of Experiment 1. Bars represent proportion correct for valid (black) and neutral (white) retro-cue conditions. Error bars reflect the standard errors of the means
Discussion
In Experiment 1, we tested the hypothesis that internal attention, similar to endogenous perceptual attention, takes time to shift to a cued item in VWM. We varied the duration between the offset of the retro-cue and the VWM retrieval stage. The data produced two findings. First, there was a numerical benefit, but not a significant retro-cue benefit when the second delay was shorter than 300 ms. A significant retro-cue benefit emerged only at durations greater than 300 ms. These results are consistent with the predictions of retro-cuing in iconic memory (Becker et al., 2000), and they extend earlier VWM findings (Germeys et al., 2010; Landman et al., 2003). Internal attention requires a measurable amount of time to shift within VWM. According to these data, the minimal amount of internal shift time rests approximately between 100–300 ms. This finding mirrors the time required for shifts of endogenous attention in perception (Carlson et al., 2006; Chakravarthi & VanRullen, 2011; Cheal & Lyon, 1991; Eriksen & Collins, 1969; Muller & Rabbitt, 1989; Posner et al., 1978; Shulman et al., 1979). Our data provide a new caveat in understanding internal attention. Once the minimum time to shift attention was given, the magnitude of the retro-cue benefit did not significantly increase as the second delay duration increased. Once a threshold time was surpassed, additional time did not continue to benefit VWM performance. Although both types of attention follow similar time courses for the onset of cuing benefits, there is an important difference in how these two develop over time. Additional time improves cuing benefits linearly for perceptual attention (Cheal & Lyon, 1991; Egly & Homa, 1991; Eriksen & Collins, 1969; Muller & Rabbitt, 1989; Posner et al., 1978; Tsal, 1983). Our results indicate that additional time above threshold does not provide additional cuing benefits for internal attention.
Experiment 1a
Eye tracking control
The purpose of Experiment 1a was to alleviate concerns that eye movements had a significant impact on the retro-cue benefit due to trans-saccadic memory formation and improved probe processing and/or integration or interruption masking. Trans-saccadic memory refers to the retaining of visual information during eye movements (reviewed in De Graef & Verfaillie, 2002; Deubel, Schneider, & Bridgeman, 2002; Rayner, 1998). Irwin and Gordon (1998) and Irwin (1992) performed a series of experiments demonstrating beneficial acquisition of information during a saccade (Irwin & Gordon, 1998) and that the capacity of trans-saccadic memory is approximately four items (Irwin, 1992; Irwin & Gordon, 1998). Because voluntary initiation of a saccades requires approximately 100–200 ms (Abrams & Jonides, 1988; Fischer & Ramsperger, 1984; reviewed in Rayner, 1998), there was the possibility that participants initiated a saccade during stimulus presentation. The consequence of this would be that retro-cue benefits could be attributable to attention operating on the aggregate representations from VWM and trans-saccadic memory. Eye movements occurring after the valid retro-cue may have introduced several confounds. For example, eye movements might have improved processing of the probe item or produced masking (integration or interruption). Although the valid retro-cue aims to orient attention within VWM, because of its directional nature, it may also influence the processing of items to be perceived in the future. The valid retro-cue may have encouraged participants to reposition their gaze in preparation for the probe stimulus. If this were to occur, as with precuing, the detection and processing of the probe item would be improved. The predictions were as follows: If the retro-cue benefit is produced by internal attentional processes, ensuring that participants maintained fixation should replicate the findings observed in Experiment 1. Alternatively, if retro-cue benefits are modulated by trans-saccadic memory, enhanced probe processing, or masking, the retro-cue effect will be reduced when fixation is monitored.
Method
Experimental design and trial sequence
The procedure used in Experiment 1 was followed except that the duration after cue offset was 100, 300, and 600 ms. This change was implemented in order to determine the effects of masking (100 ms) and to provide adequate time for internal attention to shift (300 and 600 ms) (based on Experiments 1 and 2). Both the 300- and 600-ms durations provided enough time for possible eye movements after cue presentation, allowing this experiment to examine the possible effects of eye movements prior to probe presentation.
Equipment
Eye movements were monitored using the infrared SensoMotoric Instruments (SMI) iViewX™ 2 “Red” eye tracking system. During eye tracking, participants sat 63 cm (iView designated) from the integrated 22-in. LCD monitor and bottom-mounted remote eye tracking unit. A chin rest was used to stabilize head movement. Participants were explicitly instructed to maintain central fixation, and gaze position was monitored during the entire trial sequence beginning at stimulus onset and concluding at probe presentation onset. Trials were eliminated if the participant deviated more than 1 ° of visual angle from central fixation. Participants completed 180 trials.
Participants
Eight new students from the University of Nevada psychology subject pool participated in exchange for course extra credit (ages: 19–26 years, M = 23.25, three male)
Results
A repeated measures ANOVA evaluated the factors of retro-cue type (valid, neutral) and second delay duration (100, 300, 600 ms). There was a main effect of retro-cue type revealing a significant retro-cue benefit, F(1, 7) = 29.20, p < .001; see Fig. 3. There was also a main effect of second delay duration, F(1, 7) = 4.72, p = .027. Importantly, there was a significant interaction between retro-cue type and second delay duration, F(2, 14) = 6.74, p = .009. The interaction was due to significantly better accuracy in the valid retro-cue condition than in the neutral retro-cue condition for all second delay durations greater than 100 ms (ps < .034, Bonferroni corrected pairwise comparisons). A paired t test comparing the 300- and 600-ms conditions confirmed that there was no significant difference in performance for these conditions, t 7 = .22, p = .76. In summary, as in Experiment 1, once the retro-cue effect reached significance, there was no additional benefit for longer second delay durations.
Results of Experiment 1a. Bars represent proportion correct for valid (black) and neutral (white) retro-cue conditions. Error bars reflect the standard errors of the means
Furthermore, when we examined the discarded trials—those in which eye movements were made—the same pattern of results was observed [main effect of retro-cue type, F(1, 6) = 34.65, p < .001; interaction between retro-cue and second delay duration, F(1, 6) = 7.33, p = .008; greater accuracy for second delay durations greater than 100 ms (ps < .044, Bonferroni-corrected pairwise comparisons)]. One participant made so few eye movements that her data were not included in this analysis.
Discussion
In Experiment 1a, we furthered previous retro-cue studies in which eye movements were monitored (Griffin & Nobre, 2003). We controlled for eye movement and varied the duration between the offset of the retro-cue and VWM retrieval stage. Here, we replicated the results from Experiment 1: The retro-cue developed over time, but once established, it did not benefit from longer second delay durations. Importantly, we confirmed that the retro-cue benefit is not due to contamination from eye movements. Monitoring fixation during stimulus presentation quells concerns that the retro-cue benefit was artificially inflated by trans-saccadic memory. Second, monitoring fixation after retro-cue presentation alleviates concerns that enhanced probe processing or masking (integration or interruption) impeded the retro-cue benefit at short delay durations. Finally, even if fixation was not maintained, the same pattern of results emerged, confirming that eye movements were not able to account for the retro-cue benefit.
Experiment 2
Object eccentricity effects on the retro-cue benefit
In Experiment 1, we varied the second delay duration and maintained the eccentricity of the stimulus array and found that shifting internal attention takes time. We inverted these conditions in Experiment 2. Here, we maintained the second delay duration and varied the eccentricity of the stimulus array. If internal attention is constrained by distance, placing stimuli farther from fixation will require longer times for attention to shift, and the magnitude of the retro-cue benefit will diminish. If internal attention is not constrained by distance, the magnitude of the retro-cue benefit will be constant across stimulus eccentricities. Our approach was to apply the briefest second delay duration that produced a significant retro-cue benefit (300 ms) in Experiment 1 and to assess the magnitude of the retro-cue benefit as a function of eccentricity.
Method
Experimental design and trial sequence
The procedure used in Experiment 1 was followed except that the second delay remained constant during the experiment. Object eccentricity varied: Items were presented at 3 °, 5 °, 7 °, or 9 ° of visual angle from fixation.
Participants
Twenty new participants (ages 18–33 years, M = 21.25, six male) from the University of Nevada psychology subject pool participated in exchange for course extra credit.
Results
The factors of cue type (valid, neutral) and stimulus eccentricity (3 °, 5 °, 7 °, 9 °) were subjected to a repeated measures ANOVA that revealed a retro-cue benefit, F(1, 19) = 73.20, p < .001, but no effect of stimulus eccentricity, F < 1, p = ns, and no interaction of cue type and eccentricity, F(3, 57) = 1.08, p = .37; see Fig. 4.
Results of Experiment 2. Bars represent proportion correct for valid (black) and neutral (white) retro-cue conditions. Error bars reflect the standard errors of the means
Discussion
In Experiment 2, we tested whether internal attention takes longer to shift to items presented at greater eccentricity. If internal attention operates in the same way as endogenous perceptual attention, then there should be a gradient in the magnitude of the retro-cue effect. Yet, the present results indicate that there was no significant difference in the magnitude of the retro-cue benefit across eccentricities. It should be noted that in an effort to further investigate the effect of eccentricity on shifts of internal attention, we tested several other second delay durations (146.63, 173.29, 200, and 500 ms). The results of those experiments mirrored those of Experiment 2 and confirmed that as long as the time to shift attention surpassed threshold ( >146.63 ms), internal attention provided a constant benefit to VWM regardless of object eccentricity. This unexpected reduction in the time required for internal attention to shift from Experiment 1 ( >300 ms) may be explained by individual participant differences and/or changes in participants’ strategy based on the task and second delay durations. Both of these alternatives look to be promising avenues for future research.
The present data suggest that the location of objects at encoding does not affect how internal attention shifts in VWM. This is a surprising difference from the well-documented relationship between shifts of endogenous perceptual attention and distance where further distances require more time for attention to shift (Egly & Homa, 1991; Shulman et al., 1979; Tsal, 1983).
Experiment 3
Manipulation of eccentricity and second delay duration
The purpose of Experiment 3 was to extend the results of Experiments 1 and 2 by investigating the converging influence of time and space on the retro-cue benefit using a within-subjects design. To accomplish this, we varied stimulus eccentricity and second delay duration to maximize the likelihood of observing interactions in the space–time relationship of internal attention. The predictions were as follows: If internal attention shifts independently of stimulus eccentricity, then the retro-cue benefit will remain constant across encoding eccentricities. If there is an interaction between object eccentricity and internal attention, there should be a measurable gradient in the magnitude of the retro-cue benefit so that items presented closer to central fixation benefit more and at earlier second delay durations than items encoded at greater eccentricity.
Method
Experimental design and trial sequence
The paradigm made several changes from that used in Experiment 2. First, two eccentricities were tested (3 °, 12 °). Twelve degrees, instead of 9 °, was used to enhance the likelihood of observing eccentricity effects. Second, the second delay durations in Experiment 3a were: 146.63, 159.96, 173.29, 186.62, and 200 ms (9–13 frames at a 75-Hz monitor refresh rate). These were shortened by one frame in Experiment 3b: 133.3, 146.63, 159.96, 173.29, and 186.62 ms (8–12 frames). In Experiment 3b, the color patches were replaced with an equal number of bilaterally symmetrical novel shape stimuli (10 total objects); see Supplementary Fig. 1A. We employed this new stimulus set to allay concerns that any previously reported effects were specific to the color patch stimuli used in Experiments 1 and 2. EPrime outputs were used to monitor the accuracy of the second delay duration length, and any trials showing an unexpected delay were excluded from analysis.
Participants
Two groups of students from the University of Nevada psychology subject pool participated in exchange for course extra credit: Experiment 3a, 25 volunteers (ages: 18–29 years, M = 22.12, 10 male); Experiment 3b, 20 volunteers (ages: 18–33 years, M = 20.89, three male).
Results
In Experiments 3a and 3b, we used a repeated measures ANOVA to evaluate the factors of cue type (valid, neutral), object eccentricity (3 °, 12 °), and second delay duration (Experiment 3a, 146.63 – 200 ms; Experiment 3b, 133.3 – 186.62 ms). In Experiment 3a, there was a main effect of cue type revealing the retro-cue benefit, F(1, 24) = 31.55, p < .001; see Fig. 4. No other main effects [eccentricity, F(1, 24) = 1.10, p = .30; delay duration, F(4, 96) < 1, p = ns] or any two-way or three-way interactions reached significance (all ps > .31). The results from Experiment 3b were similar. There was a significant retro-cue benefit shown by the main effect of cue type, F(1, 19) = 18.92, p < .001; see Supplementary Fig. 1B. Here, however, there was also a main effect of object eccentricity, F(1, 19) = 13.71, p = .002, due to overall superior VWM performance in the 3 ° condition (71.8 %) as compared with the 12 ° condition (67.1 %). The key result here is that as in Experiment 3a, there was no main effect of second delay duration, and none of the interactions reached significance. Most importantly, there was no interaction between eccentricity and second delay duration. In short, in both Experiment 3a and 3b there was no interaction between second delay duration and distance on the magnitude of the retro-cue benefit. Figure 5.
Accuracy results of Experiment 3a. Bars represent proportion correct for valid (black) and neutral (white) retro-cue conditions. Error bars reflect the standard errors of the means
Discussion
In Experiment 3, we tested whether spatiotemporal factors affect internal shifts of attention. We varied both the eccentricity of objects at VWM encoding and the second delay duration in an attempt to discover eccentricity effects on the retro-cue benefit. We found no evidence of a gradient in the magnitude of the effect across second delay durations. In Experiment 3a, there was no difference in the magnitude of the retro-cue benefit when the stimuli were presented at 3 ° or 12 ° or when participants had more time to shift internal attention. In Experiment 3b, we replicated this finding using novel shapes instead of color patch stimuli. Interestingly, we note that in Experiments 3a and 3b, the retro-cue benefit was numerically larger for stimuli presented at 12 ° during shorter second delay durations (Experiment 3a, 146.63, 159.96, and 173.29 ms; Experiment 3b, 133.3 and 146.63 ms). These results strengthen the conclusion that object eccentricity and second delay duration did not interact to modulate the retro-cue benefit, and are evidence that the inability to find a statistically significant gradient was not simply a power concern.
However, the results of Experiment 3b did reveal a significant main effect of eccentricity: Overall performance at 3 ° was superior to performance at 12 °. This main effect may be due to the shape stimuli, which would be more difficult to resolve at greater eccentricities, consequently lowering performance accuracy for more distal presentation arrays. Alternatively, perceptual organization may have benefited 3 ° presentations by increasing grouping effects (as compared with items presented at 12 °) (Jiang, Olson, & Chun, 2000; Woodman et al., 2003). The results of Experiments 3a and 3b also revealed significant retro-cue benefits at second delay durations as short as 146.63 (3a) and 133.33 (3b) ms. These results deviate from the minimum time required to shift internal attention observed in Experiments 1 and 2. Future research is necessary to determine whether this was due to experimental context effects improving participants’ performance when all of the second delay durations were very brief.
As in Experiment 2, the present findings support the hypothesis that for shifts of internal attention, space and time do not interact the way they do for shifts of endogenous perceptual attention. It could also be the case that internal attention is indeed influenced by time and distance, but it shifts so quickly that we might not be able to detect it with our equipment. In Experiment 3, we presented stimuli at 3 ° and 12 ° and used second delay durations limited by a 75-Hz monitor refresh rate. Based on these constraints, we calculated that internal attention would have to shift 9 ° in less than 13.33 ms, a rate of ~1.5 ms/degree, to account for our observed lack of eccentricity effects on the retro-cue benefit. Therefore, if a relationship exists between internal attention and object eccentricity in this paradigm it is shifting at over five times the rate of perceptual attention (8 ms/degree). Another alternative explanation is that our failure to observe eccentricity effects resulted from possible task irrelevance of the eccentricity information.
Experiment 4
Eccentricity effects on endogenous perceptual and internal attention
The purpose of Experiment 4 was to allay several lingering concerns. First, although previous perceptual attention (Chakravarthi & VanRullen, 2011; Egly & Homa, 1991; Shulman et al., 1979; Tsal, 1983) and episodic memory (Kosslyn, Ball, & Reiser, 1978) research have reported effects of eccentricity when distance was task irrelevant, we cannot discount the possibility that spatial eccentricity was insufficiently task relevant. Second, to strengthen our comparison of internal and perceptual attention, in Experiment 4, we provided a direct within-subjects contrast of eccentricity effects on shifts of both endogenous perceptual and internal attention. We included pre- and retro-cues and manipulated object eccentricity. The predictions were as follows: If endogenous perceptual and internal attention are differentially modulated by stimulus eccentricity, the retro-cue benefit will remain constant across encoding eccentricities, whereas precuing benefits will be larger for items presented closer to fixation. If they are similarly modulated by eccentricity, the effects of eccentricity should be consistent across pre- and retro-cued trials.
Method
Experimental design and trial sequence
The paradigm used in Experiment 4 modified that used in Experiment 3. The greatest change was to include both pre- and retro-cue conditions. Cue conditions were randomly interleaved so that the upcoming cue type (pre- or retro-) was unknown to the participants. Again, two eccentricities were tested (3 °, 12 °), but instead of presenting object displays with uniform eccentricity, here, each stimulus array contained two stimuli presented at 3 ° and two stimuli presented at 12 ° in a mixed-eccentricity array. A presentation of stimuli at 3 ° or 12 ° was equally likely at each location: Participants were unaware of at what eccentricity upcoming stimuli would be presented. This arrangement boosts the relevance of eccentricity information because it forces participants to encode multiple objects at varying eccentricities within the same array. Because our stimuli and cues utilized in Experiments 1–3 were larger than those commonly used, we shrank both the stimuli (2.4 ° × 2.4 ° of visual angle) and retro-cues (neutral retro-cue “X” [1.4°W × 1.4°H] and the valid retro-cue [1.4°L × 1.1°W] of visual angle) to fall more in line with previously reported stimuli and cue size (Dell'Acqua et al., 2010; Makovski & Jiang, 2007; Makovski et al., 2008). Stimulus presentation duration was also shortened to 200 ms to alleviate concerns about possible eye movements and to replicate stimulus durations used in previous research (Lepsien et al., 2005). Duration after cue onset was selected according to reported approximate shift time requirements for both pre- (100 ms) (Guzzon, Brignani, Miniussi, & Marzi, 2010; Tipples, 2002; Tsal, 1983) and retro- (300 ms) (Experiments 1 and 2; Becker et al., 2000) cuing. These durations were selected to provide adequate time for internal attention to shift (based on Experiments 1 and 2) while not providing excessive time for endogenous perceptual attention to shift (Tsal, 1983). Participants completed 250 trials in five 50-trial blocks that were separated by rest breaks.
Participants
Twelve new students from the University of Nevada psychology subject pool participated in exchange for course extra credit (ages: 18–31 years, M = 22.33, three male).
Results
A repeated measures ANOVA evaluated the factors of cue type (neutral, valid 3 °, valid 12 °) and cue presentation (precue, retro-cue). As was expected, there was a main effect of cue type revealing significantly better performance during valid rather than neutral cues, F(2, 22) = 13.91, p < .001; see Fig. 6. There was also a main effect of cue presentation: Performance on precued trials was significantly better than on retro-cued trials, F(1, 11) = 16.41, p = .002. Paired t tests revealed that this effect was driven by significantly more accurate performance in the 3 ° precue condition versus the 3 ° retro-cue condition, t 11 = 5.25, p < .001. There were no significant differences for neutral pre- and retro-cues, t 11 = 1.35, p = .21, or valid 12 ° pre- and retro-cues, t 11 = 1.24, p = .24. The interaction of cue type and cue presentation was not significant, F(2, 22) = 3.13, p < .064. To clarify whether there were eccentricity effects in the pre- and retro-cue presentations, we conducted individual ANOVAs for each. Both the precue, F(2, 22) = 10.15, p = .001) and retro-cue conditions, F(2, 22) = 7.26, p = .004) revealed significant effects of cue type. However, there was no significant difference between the 3 ° and 12 ° conditions (p = .72, Bonferroni-corrected pairwise comparisons) for the retro-cue presentations, suggesting that eccentricity did not matter. In contrast, the same comparison was significant for the precue presentation (p = .021, Bonferroni-corrected pairwise comparisons). In other words, eccentricity did not modulate the retro-cue benefit, but it did modulate the precue benefit.
Accuracy results of Experiment 4. Bars represent proportion correct for 3 ° (gray), 12 ° (black), and neutral (white) conditions
Discussion
In Experiment 4, we tested whether spatiotemporal factors differentially affect endogenous perceptual and internal attention. We varied both the eccentricity of objects at VWM encoding (3 °, 12 °) and the type of cue used to shift attention (pre-, retro-). Eccentricity modulated only precuing performance, with memory performance for objects presented at 3 ° being significantly better than for objects presented at 12 °. This was not the case for retro-cuing where there was no difference in performance for the 3 ° or 12 ° conditions. The present findings extend the results from Experiments 2 and 3 by further supporting the hypothesis that unlike perceptual attention (Chakravarthi & VanRullen, 2011; Egly & Homa, 1991; Rizzolatti et al., 1987; Shulman et al., 1979; Tsal, 1983), space and time do not interact to modulate shifts of internal attention.
An alternative interpretation of these results must be addressed. Both the pre- and retro- cue conditions utilized only a single second delay duration (instead of multiple second delay durations as in Experiments 1–3), creating the possibility that differential performance in the 3 ° versus 12 ° precue condition was not due to an effect of eccentricity, but instead may have resulted from better grouping for more closely positioned items, or generally improved performance for items closer to fixation. To eliminate this concern, multiple second delay durations would need to be included in the experiment. However, the second delay durations were chosen to optimize the likelihood of observing an eccentricity effect. The times were selected on the basis of previously reported attentional shift times for perceptual (Guzzon et al., 2010; Tipples, 2002; Tsal, 1983) and internal (Experiments 1 and 2; Becker et al., 2000) attention. Furthermore, if performance was modulated by item proximity to fixation or grouping effects, it seems reasonable to expect that these benefits would similarly influence both the pre- and retro-cue conditions. Results from Experiments 3a and 4 indicate that there was no difference in VWM performance in the 3 ° and 12 ° retro-cue conditions. Consequently, we conclude that the observed difference between pre- and retro-cue performance is important and not simply due to the use of one second delay duration.
General discussion
To gain leverage on behavioral similarities and differences between endogenous perceptual and internal attention, we tested whether shifts of internal attention were subject to the spatiotemporal constraints that govern shifts of perceptual attention. Endogenous perceptual shifts of attention require longer durations to traverse greater distances (Chakravarthi & VanRullen, 2011; Egly & Homa, 1991; Rizzolatti et al., 1987; Shulman et al., 1979; Tsal, 1983). Thus, if internal attention is similarly constrained, it seemed reasonable to predict that internal shifts of attention would also need more time to shift across greater spatial distances. However, there is also evidence suggesting that internal attention is not always similarly directed (Berryhill et al., 2011) and may be less flexible than perceptual attention and therefore subject to possible control from a different mechanism (Chun et al., 2011; Makovski & Jiang, 2007). Our data yielded two main findings. First, similar to endogenous perceptual attention, internal attention takes a measurable, minimum amount of time to shift. Second, unlike endogenous perceptual attention, the eccentricity of objects at encoding does not affect how quickly internal attention can shift among representations in VWM. Thus, once the minimal, quantal amount of time is allocated, internal attention appears to be impervious to the original spatial configuration of the stimuli. In other words, shifts of internal attention are not constrained by spatial distance (Appendix 1).
In Experiments 1 and 2, we expanded on previous VWM research (Germeys et al., 2010; Landman et al., 2003) and demonstrated that similar to shifts of attention within iconic memory (Becker et al., 2000), attentional shifts within VWM are not instantaneous; it took a small, but measurable, amount of time to produce the retro-cue benefit. However, increasing the time beyond this minimal value did not enhance the retro-cue benefit. These results deviate from the continued cuing benefit observed in perceptual attention when more time is permitted (Cheal & Lyon, 1991; Egly & Homa, 1991; Eriksen & Collins, 1969; Muller & Rabbitt, 1989; Posner et al., 1978; Tsal, 1983). In Experiment 1a, we incorporated eye tracking to demonstrate that the retro-cue benefit remained when participants strictly maintained central fixation. These data confirm that the results were not artifacts produced by saccadic eye movements or masking. In Experiment 2, we tested effects of object eccentricity during encoding on subsequent shifts of internal attention. Object eccentricity at encoding had no effect on the magnitude of the retro-cue benefit. Beyond a minimum threshold duration, retro-cues provided similar benefits across eccentricities. This finding demonstrates a behavioral dissimilarity from endogenous perceptual attention in which shifting attention over further distances requires more time (Chakravarthi & VanRullen, 2011; Egly & Homa, 1991; Rizzolatti et al., 1987; Shulman et al., 1979; Tsal, 1983). In Experiment 2, we also further refined our estimate of the threshold minimum time required for internal shifting downward to ~175 ms. To test whether time and encoding eccentricity jointly modulated the retro-cue benefit, in Experiment 3, we manipulated both factors of object eccentricity and second delay duration. Using two different stimulus sets, we found no significant interaction between encoding eccentricity and second delay duration on the magnitude of the retro-cue benefit. This confirms the idea that internal attention is not constrained by the eccentricity of objects during encoding. The absence of an interaction between distance and time deviates from the linear relationship observed when cuing perceptual attention (Cheal & Lyon, 1991; Egly & Homa, 1991; Eriksen & Collins, 1969; Muller & Rabbitt, 1989; Posner et al., 1978; Tsal, 1983). Lastly, in Experiment 4, we directly compared endogenous perceptual attention with internal attention, and the results further highlighted the behavioral differences reported in Experiments 2 and 3. In Experiment 4, we also provided further evidence suggesting that time and distance limit perceptual attention more strongly than internal attention (Appendix 1).
Several alternative explanations for a small retro-cue benefit at shorter delay durations require discussion. One alternative may be that a minimum time is required for perceptual processing of the cue. Previous research has demonstrated that arrow precues can shift attention in as little as 100 ms after cue onset (Guzzon et al., 2010; Tipples, 2002). This means that our shortest second delay duration of 100 ms provided adequate time (200 ms total after cue onset) for cue processing. Another alternative is that our use of brief delay durations did not provide sufficient time for the retro-cue to protect the fragile visual short-term memory (VSTM) representation from interference produced by the subsequent memory probe (Sligte et al., 2008). Another concern is that the findings were due to integration, interruption, or object-substitution masking (see reviews by Bachmann, 1994; Enns & Di Lollo, 2000; Kahneman, 1968; Kolers, 1983). For several reasons, we think it is unlikely that integration masking was caused by the retro-cue or probe stimulus. First, integration masking requires spatiotemporal confusion of multiple objects presented in rapid succession, resulting in both objects being perceived as part of the same representation (Francis & Cho, 2008; Scheerer, 1973; reviewed in Bachmann, 1994; Enns & Di Lollo, 2000; Kahneman, 1968; Kolers, 1983). However, in our paradigm, the extended cue (100 ms) and probe (until response) durations were not presented in rapid succession. This makes it more likely that there was ample time for separate cue and probe processing. Second, in these experiments, we used second delay durations at or above the reported time course of integration masking, (0–100 ms for either forward or backward (Francis & Cho, 2008; Kahneman, 1968; Scheerer, 1973; reviewed in Enns & Di Lollo, 2000). Third, there was no spatial overlap between the retro-cue and probe stimuli presentations. There may also be concerns related to interruption masking. The second delay durations fell in the range of iconic memory. Thus, there is a possibility that even though the retro-cue and probe item did not spatially overlap, a combination of eye movements and the arrival of the probe item may have truncated processing of the retro-cue via interruption masking (Bachmann & Allik, 1976; Spencer & Shuntich, 1970; reviewed in Enns & Di Lollo, 2000). The strongest evidence that interruption masking and/or eye movements cannot explain the present data is the eye movement control study (Experiment 1a). These results demonstrate that the time course and magnitude of the retro-cue benefit did not change when fixation was rigorously maintained, or when fixation was explicitly broken.
Because object substitution masking depends on the distribution of attention, eye tracking and nonoverlapping displays cannot control for this type of masking (Enns, 2004; Enns & Di Lollo, 2000; Enns & DiLollo, 1997). The object substitution model of masking centers around disruption of cortical re-entrant processing (Di Lollo, Enns, & Rensink, 2000; Enns, 2004; Enns & Di Lollo, 2000; Fahrenfort, Scholte, & Lamme, 2007). It is thought to arise because of signal conflict between ongoing processing in early visual areas (V1) and re-entrant signals originating in extrastriate (V2–V5) areas (Di Lollo et al., 2000; Enns & Di Lollo, 2000; Fahrenfort et al., 2007). Importantly, object substitution masking does not require the spatial overlap of object presentations (Enns & Di Lollo, 2000; Enns & DiLollo, 1997). Enns and Di Lollo (1997) reported that effects of four dot substitution masking were significant when attention was distributed amongst multiple targets (left, right, center), and were eliminated when a single target was presented centrally (Enns & DiLollo, 1997). We think that our paradigm minimized the likelihood of backward object substitution masking of the retro-cue because it was presented as a single central target. A final concern remains regarding forward object substitution masking of the probe stimulus by the neutral retro-cue. In the neutral cue condition, distribution of attention was necessary because the location of the upcoming probe stimulus was unknown. If masking effects were present, we would expect to see significantly lower VWM performance during neutrally cued trials at short second delay durations (100–200 ms). This was not the case, since performance on neutrally cued trials did not significantly differ over time when collapsing across second delay durations [Experiment 1, F(6, 144) = .519, p = .793].
The present findings are intertwined within the perceptual attention literature, and also within the literature describing how information is represented in VWM. Researchers have long studied how the interaction of time and distance affect perceptual attention. Still, it remains contested as to whether shifts of perceptual attention are analog (Egly & Homa, 1991; Rizzolatti et al., 1987; Shulman et al., 1979; Tsal, 1983) or abrupt (Kwak et al., 1991; Remington & Pierce, 1984; Sagi & Julesz, 1985; Sperling & Weichselgartner, 1995) in nature. Recent work suggests that under certain circumstances, both views may be correct. Work by Chakravarthi and VanRullen (2011) and Carlson et al. (2006) indicated that distance is a factor when endogenous shifts of attention are involved, but not for exogenous shifts of attention (Carlson et al., 2006; Chakravarthi & VanRullen, 2011). Although definitive answers about the true nature of shifts of perceptual attention remain elusive, the present findings further the notion that shifts of internal attention do not fit within any of the currently proposed models of perceptual attention. These results indicate that unlike endogenous perceptual attention, internal attention shifts in a quantal fashion and is unaffected by spatial distance. This may seem similar to exogenous perceptual attention, but the retro-cue benefit emerges with central cues whereas exogenous attention is defined by an attentional shift to a peripheral cue. Most importantly, results from Experiment 4, in which we directly compared perceptual and internal attention, indicate that there are significant differences in VWM performance for each type of attention when eccentricity is manipulated. Similar to a proposal by Vecera (1997), the results from Experiment 4 indicate that regardless of whether perceptual attention shifts in an analog or abrupt fashion, object eccentricity remains a significant factor (Vecera, 1997). Therefore, it seems reasonable to conclude that the results of Experiment 4 illustrate a behavioral effect of distance on shifts of perceptual attention because object eccentricity was an important modulating factor. Together with research testing the efficacy of various types of retro-cues (Berryhill et al., 2011) and the divisibility of internal attention by varying the number of retro-cued items (Makovski & Jiang, 2007), these results are consistent with the notion that internal attention exhibits behavioral dissimilarities to perceptual attention.
The present results also contribute to the VWM literature and are consistent with previous research that suggests that late object-based attentional selection is spatially invariant. Vecera and Farah (1994) suggested that there are two forms of object-based selection. They proposed that early attentional selection individuates object features but links them to a spatial location, whereas late-stage attentional selection is object based and independent of spatial location (Vecera, 1997; Vecera & Farah, 1994). Several groups have clearly demonstrated that spatial and object information remain separate during iconic memory (Awh & Jonides, 2001; Kramer, Weber, & Watson, 1997; Matsukura & Vecera, 2011; Vecera, 1997; Vecera & Farah, 1994) and VWM (Matsukura & Vecera, 2009). At this later stage, objects are thought to be an integrated whole (e.g., a tomato), rather than a lose assemblage of features (e.g., red, round, on the plate; Duncan, 1984; Kahneman, Treisman, & Gibbs, 1992; Matsukura & Vecera, 2011; Vecera & Farah, 1994; Vogel, Woodman, & Luck, 2001). In support of this idea, research has demonstrated that increases in the number of features that an object possesses has minimal effect on the capacity estimates of VWM, indicating that objects are stored whole, not as a list of features (e.g. Awh, Barton, & Vogel, 2007; Luck & Vogel, 1997; Vogel et al., 2001). However, we note that there is debate on this topic, and some researchers have concluded that the number of object features has a large effect on VWM capacity (e.g., Alvarez & Cavanagh, 2004).
Our data extend this literature by suggesting that internal attention operates on object-based representations that are partially spatially invariant. These data support a spatial-invariance model that includes relative spatial location as a component feature of object-based representations in VWM, rather than absolute spatial location. Olson and Marshuetz (2005) demonstrated that relative location (e.g., the telephone is to the left of the computer) is encoded as part of object representations in VWM because only relative but not absolute spatial information (e.g., the telephone is 8 in. to the left of the computer) interfered with object VWM performance (Olson & Marshuetz, 2005). Our data are consistent with the idea that relative spatial information is included in object representations in VWM. The preservation of relative spatial information accounts for directional retro-cues’ ability to shift internal attention toward specific object representations in VWM. The relative nature of the spatial information concurrently allows shifts of internal attention to be generally unaffected by the eccentricity of the objects at encoding.
These findings also add constraints to two theoretical frameworks explaining the retro-cue benefit. Matsukura et al.’s (2007) protection account states that internal attention serves to prevent the cued item from decay. Fragile VSTM describes a type of memory accessed via retro-cuing that is independent from iconic and robust VWM (Sligte et al., 2008). Our manipulation of the second delay duration indicates that it takes ~200 ms to protect the retro-cued item, or to access items in fragile VSTM. These mechanisms will also need to include a relative representation of distance, because our results indicate that eccentricity information is represented in relative terms. Therefore, items protected within VWM (Matsukura et al., 2007), or maintained in fragile VSTM (Sligte et al., 2008) must also preserve distance in relative terms.
In conclusion, the goal of these present studies was to investigate how internal attention shifts among items represented in VWM. The broader goal was to discover behavioral similarities and differences between perceptual and internal forms of attention. Importantly, the present data provide further evidence suggesting that internal attention possesses distinct behavioral characteristics and has advantages of its own. Our findings demonstrate that unlike endogenous perceptual attention, internal attention is not subject to the physical constraint of object distance at encoding. Regardless of how far from fixation an item may be remembered, internal attention requires only a short, quantal amount of time to shift and provide a cuing benefit. In short, perceptual attention abides by the laws of space and time, but internal attention can shift over a desk or a desert with equal alacrity.
References
Abrams, R. A., & Jonides, J. (1988). Programming saccadic eye-movements. Journal of Experimental Psychology. Human Perception and Performance, 14, 428–443.
Alvarez, G. A., & Cavanagh, P. (2004). The capacity of visual short-term memory is set both by visual information load and by number of objects. Psychological Science, 15, 106–111.
Awh, E., Barton, B., & Vogel, E. K. (2007). Visual working memory represents a fixed number of items regardless of complexity. Psychological Science, 18, 622–628. doi:10.1111/j.1467-9280.2007.01949.x
Awh, E., & Jonides, J. (2001). Overlapping mechanisms of attention and spatial working memory. Trends in Cognitive Sciences, 5, 119–126.
Bachmann, T. (1994). Psychophysiology of visual masking: The fine structure of conscious experience. Commack, NY: Nova Science.
Bachmann, T., & Allik, J. (1976). Integration and interruption in masking of form by form. Perception, 5, 79–97.
Becker, M. W., Pashler, H., & Anstis, S. M. (2000). The role of iconic memory in change-detection tasks. Perception, 29, 273–286.
Berryhill, M. E., Richmond, L. L., Shay, C. S., & Olson, I. R. (2011). Shifting attention among working memory representations: Testing cue type, awareness, and strategic control. The Quarterly Journal of Experimental Psychology, 65(3), 426–438. doi:10.1080/17470218.2011.604786
Carlson, T. A., Hogendoorn, H., & Verstraten, F. A. (2006). The speed of visual attention: What time is it? Journal of Vision, 6, 1406–1411. doi:10.1167/6.12.6/6/12/6/
Carrasco, M. (2011). Visual attention: The past 25 years. Vision Research, 51, 1484–1525. doi:10.1016/j.visres.2011.04.012
Chakravarthi, R., & VanRullen, R. (2011). Bullet trains and steam engines: Exogenous attention zips but endogenous attention chugs along. Journal of Vision 11(4):12, 1–12. doi:10.1167/11.4.1212
Cheal, M., & Lyon, D. R. (1991). Central and peripheral precuing of forced-choice discrimination. The Quarterly Journal of Experimental Psychology. A, 43, 859–880.
Chun, M. M., Golomb, J. D., & Turk-Browne, N. B. (2011). A taxonomy of external and internal attention. Annual Review of Psychology, 62, 73–101. doi:10.1146/annurev.psych.093008.100427
Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. The Behavioral and Brain Sciences, 24, 87–185.
De Graef, P., & Verfaillie, K. (2002). Transsaccadic memory for visual object detail. Brain's Eye: Neurobiological and Clinical Aspects of Oculomotor Research, 140, 181–196.
Dell'Acqua, R., Sessa, P., Toffanin, P., Luria, R., & Jolicoeur, P. (2010). Orienting attention to objects in visual short-term memory. Neuropsychologia, 48, 419–428. doi:10.1016/j.neuropsychologia.2009.09.033
Delvenne, J. F., Cleeremans, A., & Laloyaux, C. (2010). Feature bindings are maintained in visual short-term memory without sustained focused attention. Experimental Psychology, 57, 108–116. doi:10.1027/1618-3169/A000014
Deubel, H., Schneider, W. X., & Bridgeman, B. (2002). Transsaccadic memory of position and form. Brain's Eye: Neurobiological and Clinical Aspects of Oculomotor Research, 140, 165–180.
Di Lollo, V., Enns, J. T., & Rensink, R. A. (2000). Competition for consciousness among visual events: The psychophysics of reentrant visual processes. Journal of Experimental Psychology. General, 129, 481–507.
Duncan, J. (1984). Selective attention and the organization of visual information. Journal of Experimental Psychology. General, 113, 501–517.
Egeth, H. E., & Yantis, S. (1997). Visual attention: Control, representation, and time course. Annual Review of Psychology, 48, 269–297. doi:10.1146/annurev.psych.48.1.269
Egly, R., & Homa, D. (1991). Reallocation of visual-attention. Journal of Experimental Psychology. Human Perception and Performance, 17, 142–159.
Enns, J. T. (2004). Object substitution and its relation to other forms of visual masking. Vision Research, 44, 1321–1331. doi:10.1016/j.visres.2003.10.024S0042698903007727
Enns, J. T., & Di Lollo, V. (2000). What's new in visual masking? Trends in Cognitive Sciences, 4, 345–352.
Enns, J. T., & DiLollo, V. (1997). Object substitution: A new form of masking in unattended visual locations. Psychological Science, 8, 135–139.
Eriksen, C. W., & Collins, J. F. (1969). Temporal course of selective attention. Journal of Experimental Psychology, 80, 254–261.
Eriksen, C. W., & Hoffman, J. E. (1972). Some characteristics of selective attention in visual perception determined by vocal reaction-time. Perception & Psychophysics, 11, 169–171.
Ester, E. F., Serences, J. T., & Awh, E. (2009). Spatially global representations in human primary visual cortex during working memory maintenance. Journal of Neuroscience, 29, 15258–15265. doi:10.1523/JNEUROSCI.4388-09.2009
Fahrenfort, J. J., Scholte, H. S., & Lamme, V. A. (2007). Masking disrupts reentrant processing in human visual cortex. Journal of Cognitive Neuroscience, 19, 1488–1497. doi:10.1162/jocn.2007.19.9.1488
Fischer, B., & Ramsperger, E. (1984). Human express saccades: Extremely short reaction-rimes of goal directed eye-movements. Experimental Brain Research, 57, 191–195.
Francis, G., & Cho, Y. S. (2008). Effects of temporal integration masking on the shape of visual backward functions. Journal of Experimental Psychology. Human Perception and Performance, 34, 1116–1128. doi:10.1037/0096-1523.34.5.1116
Germeys, F., De Graef, P., Van Eccelpoel, C., & Verfaillie, K. (2010). The visual analog: Evidence for a preattentive representation across saccades. Journal of Vision, 10, 9. doi:10.1167/10.10.9
Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations. Journal of Cognitive Neuroscience, 15, 1176–1194. doi:10.1162/089892903322598139
Guzzon, D., Brignani, D., Miniussi, C., & Marzi, C. A. (2010). Orienting of attention with eye and arrow cues and the effect of overtraining. Acta Psychologica, 134, 353–362. doi:10.1016/j.actpsy.2010.03.008
Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature, 458, 632–635. doi:10.1038/nature07832
Irwin, D. E. (1992). Memory for position and identity across eye-movements. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 307–317.
Irwin, D. E., & Gordon, R. D. (1998). Eye movements, attention and trans-saccadic memory. Visual Cognition, 5, 127–155.
Jiang, Y., Olson, I. R., & Chun, M. M. (2000). Organization of visual short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 683–702.
Jonides, J. (1981). Voluntary versus automatic control over the mind's eye's movement. In J. B. Long & A. D. Baddely (Eds.), Attention and Performance, IX (pp. 187–203). Hillsdale, NJ: Erlbaum.
Kahneman, D. (1968). Method, findings, and theory in studies of visual masking. Psychological Bulletin, 70, 404–425.
Kahneman, D., Treisman, A., & Gibbs, B. J. (1992). The reviewing of object files: Object-specific integration of information. Cognitive Psychology, 24, 175–219.
Kastner, S., & Ungerleider, L. G. (2000). Mechanisms of visual attention in the human cortex. Annual Review of Neuroscience, 23, 315–341.
Kinchla, R. A. (1992). Attention. Annual Review of Psychology, 43, 711–742. doi:10.1146/annurev.ps.43.020192.003431
Kolers, P. A. (1983). Perception and representation. Annual Review of Psychology, 34, 129–166.
Kosslyn, S. M. (1981). The medium and the message in mental imagery—A theory. Psychological Review, 88, 46–66.
Kosslyn, S. M., Ball, T. M., & Reiser, B. J. (1978). Visual images preserve metric spatial information: Evidence from studies of image scanning. Journal of Experimental Psychology. Human Perception and Performance, 4, 47–60.
Kramer, A. F., & Hahn, S. (1995). Splitting the beam: Distribution of attention over non contiguous regions of the visual field. Psychological Science, 6, 381–386.
Kramer, A. F., Weber, T. A., & Watson, S. E. (1997). Object-based attentional selection— Grouped arrays or spatially invariant representations?: Comment on Vecera and Farah (1994). Journal of Experimental Psychology. General, 126, 3–13.
Kuo, B. C., Rao, A., Lepsien, J., & Nobre, A. C. (2009). Searching for targets within the spatial layout of visual short-term memory. Journal of Neuroscience, 29, 8032–8038. doi:10.1523/Jneurosci.0952-09.2009
Kwak, H. W., Dagenbach, D., & Egeth, H. (1991). Further evidence for a time-independent shift of the focus of attention. Perception & Psychophysics, 51, 455–464.
Landman, R., Spekreijse, H., & Lamme, V. A. F. (2003). Large capacity storage of integrated objects before change blindness. Vision Research, 43, 149–164.
Lepsien, J., Griffin, I. C., Devlin, J. T., & Nobre, A. C. (2005). Directing spatial attention in mental representations: Interactions between attentional orienting and working-memory load. NeuroImage, 26, 733–743. doi:10.1016/j.neuroimage.2005.02.026
Lepsien, J., & Nobre, A. C. (2006). Cognitive control of attention in the human brain: Insights from orienting attention to mental representations. Brain Research, 1105, 20–31. doi:10.1016/j.brainres.2006.03.033
Lepsien, J., & Nobre, A. C. (2007). Attentional modulation of object representations in working memory. Cerebral Cortex, 17, 2072–2083. doi:10.1093/cercor/bhl116
Lepsien, J., Thornton, I., & Nobre, A. C. (2011). Modulation of working-memory maintenance by directed attention. Neuropsychologia, 49, 1569–1577. doi:10.1016/J.Neuropsychologia.2011.03.011
Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279–281. doi:10.1038/36846
Makovski, T., & Jiang, Y. V. (2007). Distributing versus focusing attention in visual short-term memory. Psychonomic Bulletin & Review, 14, 1072–1078.
Makovski, T., Sussman, R., & Jiang, Y. V. (2008). Orienting attention in visual working memory reduces interference from memory probes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, 369–380. doi:10.1037/0278-7393.34.2.369
Matsukura, M., Luck, S. J., & Vecera, S. P. (2007). Attention effects during visual short-term memory maintenance: Protection or prioritization? Perception & Psychophysics, 69, 1422–1434.
Matsukura, M., & Vecera, S. P. (2009). Interference between object-based attention and object-based memory. Psychonomic Bulletin & Review, 16, 529–536. doi:10.3758/Pbr.16.3.529
Matsukura, M., & Vecera, S. P. (2011). Object-based selection from spatially invariant representations: Evidence from a feature-report task. Attention, Perception, & Psychophysics, 73, 447–457. doi:10.3758/s13414-010-0039-9
Muller, H. J., & Rabbitt, P. M. (1989). Reflexive and voluntary orienting of visual attention: Time course of activation and resistance to interruption. Journal of Experimental Psychology. Human Perception and Performance, 15, 315–330.
Munneke, J., Heslenfeld, D. J., & Theeuwes, J. (2010). Spatial working memory effects in early visual cortex. Brain and Cognition, 72, 368–377. doi:10.1016/j.bandc.2009.11.001
Nakayama, K., & Mackeben, M. (1989). Sustained and transient components of focal visual-attention. Vision Research, 29, 1631–1647.
Nobre, A. C. (2008). Spatial attention can bias search in visual short-term memory. Frontiers in Human Neuroscience, 1:4, 1–9. doi:10.3389/neuro.09.004.2007
Nobre, A. C., Coull, J. T., Maquet, P., Frith, C. D., Vandenberghe, R., & Mesulam, M. M. (2004). Orienting attention to locations in perceptual versus mental representations. Journal of Cognitive Neuroscience, 16, 363–373. doi:10.1162/089892904322926700
Nobre, A. C., Rao, A. L., & Chelazzi, L. (2006). Selective attention to specific features within objects: Behavioral and electrophysiological evidence. Journal of Cognitive Neuroscience, 18, 539–561.
Olson, I. R., & Marshuetz, C. (2005). Remembering "what" brings along "where" in visual working memory. Perception & Psychophysics, 67, 185–194.
Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3–25.
Posner, M. I., Nissen, M. J., & Ogden, W. C. (1978). Attended and unattended processing modes: The role of set for spatial location. Modes of perceiving and processing information. Hillsdale, N.J.: Erlbaum.
Posner, M. I., & Petersen, S. E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13, 25–42. doi:10.1146/annurev.ne.13.030190.000325
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372–422.
Remington, R., & Pierce, L. (1984). Moving attention: Evidence for time-invariant shifts of visual selective attention. Perception & Psychophysics, 35, 393–399.
Rizzolatti, G., Riggio, L., Dascola, I., & Umiltà, C. (1987). Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention. Neuropsychologia, 25, 31–40.
Sagi, D., & Julesz, B. (1985). Fast noninertial shifts of attention. Spatial Vision, 1, 141–149.
Scheerer, E. (1973). Integration, interruption and processing rate in visual backward masking. I. Review. Psychologische Forschung, 36, 71–93.
Schmidt, B. K., Vogel, E. K., Woodman, G. F., & Luck, S. J. (2002). Voluntary and automatic attentional control of visual working memory. Perception & Psychophysics, 64, 754–763.
Serences, J. T., Ester, E. F., Vogel, E. K., & Awh, E. (2009). Stimulus-specific delay activity in human primary visual cortex. Psychological Science, 20, 207–214. doi:10.1111/j.1467-9280.2009.02276.x
Shulman, G. L., Remington, R. W., & Mclean, J. P. (1979). Moving attention through visual space. Journal of Experimental Psychology. Human Perception and Performance, 5, 522–526.
Sligte, I. G., Scholte, H. S., & Lamme, V. A. (2008). Are there multiple visual short-term memory stores? PLoS One, 3, e1699. doi:10.1371/journal.pone.0001699
Sligte, I. G., Scholte, H. S., & Lamme, V. A. (2009). V4 activity predicts the strength of visual short-term memory representations. Journal of Neuroscience, 29, 7432–7438. doi:10.1523/JNEUROSCI.0784-09.2009
Sligte, I. G., Vandenbroucke, A. R., Scholte, H. S., & Lamme, V. A. (2010). Detailed sensory memory, sloppy working memory. Frontiers in Psychology, 1, 175. doi:10.3389/fpsyg.2010.00175
Sligte, I. G., Wokke, M. E., Tesselaar, J. P., Scholte, H. S., & Lamme, V. A. (2011). Magnetic stimulation of the dorsolateral prefrontal cortex dissociates fragile visual short-term memory from visual working memory. Neuropsychologia, 49, 1578–1588. doi:10.1016/j.neuropsychologia.2010.12.010
Spencer, T. J., & Shuntich, R. (1970). Evidence for an interruption theory of backward masking. Journal of Experimental Psychology, 85, 198–203.
Sperling, G. (1960). The information available in brief visual presentations. Washington, D.C.: American Psychological Association.
Sperling, G., & Weichselgartner, E. (1995). Episodic theory of the dynamics of spatial attention. Psychological Review, 102, 503–532.
Tipples, J. (2002). Eye gaze is not unique: Automatic orienting in response to uninformative arrows. Psychonomic Bulletin & Review, 9, 314–318.
Treisman, A. M., & Gelade, G. (1980). Feature-integration theory of attention. Cognitive Psychology, 12, 97–136.
Tsal, Y. (1983). Movements of attention across the visual-field. Journal of Experimental Psychology. Human Perception and Performance, 9, 523–530.
Vandenbroucke, A. R., Sligte, I. G., & Lamme, V. A. (2011). Manipulations of attention dissociate fragile visual short-term memory from visual working memory. Neuropsychologia, 49, 1559–1568. doi:10.1016/j.neuropsychologia.2010.12.044
Vecera, S. P. (1997). Grouped arrays versus object-based representations: Reply to Kramer et al. Journal of Experimental Psychology. General, 126, 14–18.
Vecera, S. P., & Farah, M. J. (1994). Does visual-attention select objects or locations. Journal of Experimental Psychology. General, 123, 146–160.
Vogel, E. K., Woodman, G. F., & Luck, S. J. (2001). Storage of features, conjunctions, and objects in visual working memory. Journal of Experimental Psychology. Human Perception and Performance, 27, 92–114.
Woodman, G. F., Vecera, S. P., & Luck, S. J. (2003). Perceptual organization influences visual working memory. Psychonomic Bulletin & Review, 10, 80–87.
Author note
We express our sincerest thanks to Dr. Gideon Caplovitz for his invaluable help with eye tracker troubleshooting and paradigm design. Thank you also to Dr. Thomas Nickles, Kevin Jones, Dwight Peterson, and Terina Metoyer for their comments, suggestions, and help throughout the course of the project. This work was supported by faculty start-up funds generously provided to M.E.B. by the University of Nevada, Reno.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(DOC 84 kb)
Appendix
Rights and permissions
About this article
Cite this article
Tanoue, R.T., Berryhill, M.E. The mental wormhole: Internal attention shifts without regard for distance. Atten Percept Psychophys 74, 1199–1215 (2012). https://doi.org/10.3758/s13414-012-0305-0
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13414-012-0305-0