Visual working memory (VWM) is characterized by temporarily storing a limited amount of visual information (e.g., Baddeley, 2010; Cowan, 2005; L. Huang, 2010; Jiang, Makovski, & Shim, 2009; Xu & Chun, 2006; Zhang & Luck, 2008), usually holding about 3–4 simple objects (Fukuda, Awh, & Vogel, 2010). Constrained by this capacity limit, the human brain has developed certain selection mechanisms to extract information efficiently from the enriched environment (e.g., Alvarez, 2011; Gao, Li, Yin, & Shen, 2010; Woodman, Vecera, & Luck, 2003; Xu & Chun, 2009).

A few studies have examined these selecting mechanisms by focusing on the fate of the task-irrelevant features (e.g., shape), which together with the target feature (e.g., color) pertain to the same object. Behavioral, event-related potential (ERP) and functional magnetic resonance imaging (fMRI) studies (e.g., Gao et al., 2010; Hyun, Woodman, Vogel, Hollingworth, & Luck, 2009) have convergingly found that the irrelevant simple feature can be selected into VWM (i.e., object-based encoding). For instance, in a change detection task in which the change of an irrelevant feature is manipulated, a significant “irrelevant-change distracting effect” was consistently observed (e.g., Gao et al., 2010; Hyun et al., 2009; but see Woodman & Vogel, 2008). That is, the change of the task-irrelevant feature in the test array dramatically influences the behavioral performance on the target feature, suggesting that the irrelevant feature was selected into VWM.

However, the aforementioned studies tested the selection mechanism by restricting memory load to within the VWM capacity (i.e., 3–4 simple objects, a low load). Since in our daily life information selection frequently takes place in a situation in which the available information exceeds the VWM capacity, whether the selection mechanism employed is similar to that in the low-load condition remains unknown. Some evidence has shown that in visual perception, two different mechanisms are adopted in low- and high-load conditions (Lavie, Hirst, de Fockert, & Viding, 2004). Recently, Xu (2010) examined this issue in VWM by asking participants to remember distinct colors while ignoring simple shapes in an fMRI study. By focusing on activation of the lateral occipital complex (LOC), which tracks the processing of shapes in the brain, she found that, consistent with previous VWM studies, the LOC was significantly activated under low VWM load (i.e., remembering two objects), suggesting that the irrelevant shape was selected into VWM involuntarily. However, in line with findings in perception research (e.g., Lavie et al., 2004), the activation in LOC was attenuated, or even suppressed, under high VWM load (i.e., remembering six objects), implying that the irrelevant shape was not extracted in this condition.

Xu’s (2010) finding is particularly important because it fits well with the perceptual-load theory, and thereby offers further evidence on the intimate interaction between perception and VWM. Nonetheless, this conclusion needs further verification, considering that the result with high VWM load was a null effect. In particular, since her interest was only in the neural marker, Xu did not directly manipulate the change of the irrelevant shape, and thus was unable to examine the storage of the irrelevant shape from the behavioral performance. Moreover, Xu took LOC as the only region of interest, yet other regions might be responsible for the processing of irrelevant information. Finally, recent findings on the storage of ensemble representations in VWM further imply the necessity of reexamining this issue. It has been found that, beyond individual features (or items) (Fukuda et al., 2010) and their spatial layout (Jiang, Olson, & Chun, 2000), the ensemble properties of the features in a display are also extracted into VWM, which can considerably enhance visual cognition (Alvarez, 2011; Brady & Alvarez, 2011; Brady, Konkle, & Alvarez, 2011). The ensemble representation is suggested to be constructed by sampling across all of the displayed items, even including the items that are not the focus of attention and not the target of the task (see Alvarez, 2011, for a state-the-art review; Alvarez & Oliva, 2009; Huang & Sekuler, 2010). Therefore, quite possibly, even if the individual irrelevant shape could not enter into VWM, its ensemble information might.

To this end, in the present study we reexamined the fate of task-irrelevant information in a high-VWM-load condition by focusing on the presence of an “irrelevant-change distracting effect,” while using the same feature dimensions (i.e., color as target and shape as the irrelevant dimension) used by Xu (2010). We predicted that if irrelevant information were encoded into VWM, then a distracting effect would be obtained; otherwise, there should be no such effect.

Experiment 1: Whole-probe method

We first examined the fate of irrelevant features under high VWM load by using a whole-probe method, which had been adopted in previous studies examining the irrelevant-change distracting effect (e.g., Hyun et al., 2009). This method keeps the general configuration of memory and the test array constant, while changing one individual feature when a change takes place. Following Xu’s (2010) study, the participants were required to remember six colors of objects while ignoring the shapes.

Method

Participants

A group of 16 Zhejiang University undergraduates (8 male, 8 female; ages 18–25 years) participated in the experiment with signed consent forms. All had normal color vision and normal or corrected-to-normal visual acuity.

Stimuli and apparatus

Colored shapes were used as stimuli (1.64° × 1.64° of visual angle, from a viewing distance of 60 cm), and color was the target dimension. A set of seven distinct shapes and seven colors (see Fig. 1) were used in the experiment. Six of the objects were evenly distributed (separated by angles of 60°) around an invisible circle with a radius of 2.86° to the center of the gray (RGB: 128, 128, 128) screen of a 17-in. CRT monitor.

Fig. 1
figure 1

The seven shapes and colors used in Experiment 1

Full size image

Procedure and analysis

Each trial began with a 200-ms fixation cross to warn the participant that a trial would start (Fig. 2). After a 100- to 200-ms blank interval, a memory array was presented on the screen for 200 ms and was followed by a 1,000-ms blank interval. Finally, a test array was presented on screen until a response was initiated. If no response was made within 2,000 ms, a new trial started automatically. Both the memory and test arrays contained six objects. To keep the participants’ eyes fixated at the center of the screen, the fixation mark was displayed at the centre of the screen throughout the whole trial. Participants were instructed to detect whether a color changed in the test array as compared to the memory array, while ignoring the shape, which would otherwise impair their performance. The intertrial blank interval was 1,000–1,400 ms. Seating in a dark and sound-shielded room, the participants were instructed to press “F” on the keyboard if a color changed, and “J” if no change occurred. Both response accuracy and reaction time (RT) were emphasized and recorded.

Fig. 2
figure 2

Schematic illustration of a single trial in Experiment 1. Here, both the relevant color and the irrelevant feature shape changed

Full size image

The color and shape of the changed item in the test array were changed independently, with a probability of 50%. When a change occurred, a new feature that had not been used in the memory array was adopted in the test array. Therefore, there were two types of relationship between the memory and test arrays, in terms of the irrelevant feature: Shapes were identical (no change) or the shape of one object changed in the test array (irrelevant change). When both the shape and color changed, both changes occurred on the same object. There were 64 trials in each condition (32 trials with a color change and 32 with no color change), resulting in 128 trials in total, which were presented randomly. The experiment was divided into two blocks with a 5-min break in between, lasting about 15 min.

A one-way analysis of variance (ANOVA) with Change Type (no change or irrelevant change) as within-subjects factor was conducted on both accuracy and RTs. Instead of simply comparing the accuracy between the two change types (e.g., Gao et al. 2010), the accuracy was determined in terms of signal detection theory, which allowed us to examine the sensitivity (d') to the change and the response bias (criterion).

Results and discussion

As is shown in Fig. 3, the irrelevant change influenced the behavioral performance, particularly in terms of d'. Confirming the observation, the AONVA revealed a significant main effect of change type on d' [F(1, 15) = 7.16, p < .025, η 2 p = .32], suggesting that the irrelevant change impaired detection of the target change. Although the participants exhibited a higher tendency to respond “changed” under irrelevant-change condition, no difference was found between the no-change and irrelevant-change conditions in criterion [F(1, 15) < 1]. No difference was found in RTs, either [F(1, 15) = 1.18, p = .30, η 2 p = .07].

Fig. 3
figure 3

Results of Experiment 1 for (a) d', (b) criterion, and (c) RTs in the no-change and irrelevant-change conditions. The smaller the criterion, the greater the tendency to respond “changed”

Full size image

In contrast to Xu (2010), where the irrelevant shape could not be selected into VWM under high VWM load, Experiment 1 demonstrated an irrelevant-change distracting effect due to the irrelevant shape change. However, there was a critical difference between our Experiment 1 and Xu’s study: We used a whole-probe method, while Xu used a partial-probe method (i.e., presenting the probe at the screen center). This method difference may have resulted in the discrepancy in findings. Furthermore, in the whole-probe method, the ensemble representation of the irrelevant shapesFootnote 1 was changed under the irrelevant-change condition, and consequently might have affected performance, particularly since the six objects were displayed within the parafovea. However, in the partial-probe method, the ensemble information of the irrelevant shapes was lost in the test phase, regardless of the irrelevant change, which might have erased any distracting effect revealed in Experiment 1 even using the same manipulation. Therefore, the findings of Experiment 1 were possibly restricted to paradigms using the whole-probe method. Experiment 2 used a partial-probe method to examine this issue.

Experiment 2: Partial-probe method

To examine whether the findings of Experiment 1 were restricted to the whole-probe method, a partial-probe method was adopted. Beyond presenting the probe at the screen center (Exp. 2a), as had been done in Xu (2010), we also displayed the probe at a location that had been taken in the memory array (Exp. 2b), since it has been shown that the location of probe affected task performance (e.g., Hollingworth, 2006). If the distracting effect was restricted to the whole-probe method, we would not observe any distracting effect due to the irrelevant change. Otherwise, we would once again find the distracting effect. In addition, a low-load condition (two objects) was added so that we could have a direct comparison with Xu (2010).

Method

A group of 36 undergraduates (16 male, 20 female; ages 18–25 years) participated in Experiment 2. Either two or six colored shapes constituted the memory array, yet only one item was displayed as a probe (Fig. 4a), which was presented either at the screen center (Exp. 2a; 18 participants: 8 male, 10 female) or at a location selected from the memory array (Exp. 2b; 18 participants: 8 male, 10 female). A 200-ms fixation was presented to mark the start of a trial, then disappeared from the screen in order to avoid a possible forward mask for the probe (particularly in Exp. 2a). A three-way mixed ANOVA with Change Type (no change or irrelevant change) and Set Size (two or six objects) as within-subjects factors and Probe Type (Exp. 2a or 2b) as a between-subjects factor was conducted on d', criterion (c), and RTs. The other aspects were the same as in Experiment 1.

Fig. 4
figure 4

(a) Schematic illustration of a single trial in Experiments 2a and 2b. The black dash circle in the memory array denotes the changed objects, yet it is not shown during the experiment. (b–d) The results of Experiment 2 for (b) d', (c) criterion, and (d) RTs in the no-change and irrelevant-change conditions. The smaller the criterion, the greater the tendency to respond “changed”

Full size image

To have a more stringent test for the effect of the irrelevant change, we also ran a mixed ANOVA by taking Change Type as a within-subjects factor and Probe Type (Exp. 1 or 2) as a between-subjects factor for remembering six objects.

Results

Remembering two versus six objects

Relative to remembering two objects, remembering six objects led to lower sensitivity (Fig. 4b), a higher tendency to respond “changed” (Fig. 4c), and longer RTs (Fig. 4d). The irrelevant change affected performance, particularly on the criterion.

The ANOVA on d' revealed that it was significantly higher for two objects (d' = 2.96) than for six objects (d' = 1.22) [set size: F(1, 34) = 442.35, p < .01, η 2 p = .93], yet neither the main effect of change type [F(1, 34) < 1, p = .72, η 2 p = .004] nor the Change Type × Set Size interaction [F(1, 34) = 1.83, p = .19, η 2 p = .05] was significant.

As for the criterion, the ANOVA also revealed a significant main effect of set size [F(1, 34) = 7.8, p < .01, η 2 p = .19], suggesting that the participants exhibited a greater tendency to respond “changed” for six objects (c = 0.78) than for two objects (c = 1.13). The irrelevant change affected the criterion by creating a greater tendency to respond “changed” for an irrelevant change (c = 0.73) than for no change (c = 1.18) [change type: F(1, 34) = 15.80, p < .01, η 2 p = .32]. The change type was further modulated by set size [Change Type × Set Size: F(1, 34) = 7.00, p < .025, η 2 p = .17]. Paired t tests elaborating this interaction showed that, regardless of memory load, the irrelevant change significantly lowered the criterion (both ps < .005). However, the distracting effect (irrelevant change – no change) was significantly greater for two objects (0.75) than for six objects (0.15) [t(35) = 2.69, p < .025].

The ANOVA on RTs yielded a significant main effect of set size [F(1, 34) = 88.24, p < .01, η 2 p = .72], suggesting that the participants responded more slowly for six objects (RT = 795 ms) than for two objects (RT = 692 ms). A significant main effect of change type was also revealed [F(1, 34) = 10.49, p < .005, η 2 p = .24], implying that the irrelevant change (RT = 750 ms) delayed the response relative to no change (RT = 738 ms). However, the effect of change type was not modulated by set size [Change Type × Set Size: F(1, 34) < 1, p > .5, η 2 p = .01].

All other effects were nonsignificant (ps > .15, η 2 p < .05).

Whole probe versus partial probe for remembering six objects

Since the above analysis did not find any difference between Experiments 2a and 2b (i.e., no probe type effect at all), we pooled the data across the two subexperiments and compared them with those of Experiment 1. The ANOVA revealed a significant Change Type × Probe Type interaction on d' [F(1, 50) = 7.23, p = .01, η 2 p = .13], confirming our findings that the sensitivity was considerably impaired by irrelevant change in the whole-probe method, but not in the partial-probe method. Replicating previous findings (Jiang et al., 2000), a significant main effect of probe type was revealed on criterion [F(1, 50) = 44.22, p < .001, η 2 p = .47], indicating that the participants exhibited a greater tendency to respond “changed” with the partial-probe (c = 0.78) than with the whole-probe (c = 1.90) method. Finally, the ANOVA on RTs demonstrated that the response was significantly longer under irrelevant change (RT = 848 ms) than under no change (RT = 829 ms) [F(1, 50) = 6.80, p < .025, η 2 p = .12]. The other effects were nonsignificant (ps > .05, η 2 p < .05).

Discussion

By using the same method of presenting the probe as Xu (2010; our Exp. 2a) and a similar one (our Exp. 2b), we consistently found that the irrelevant change lowered the criterion and lengthened the RT, regardless of the memory load. These results suggest that the irrelevant shape was encoded into VWM and that the result revealed in Experiment 1 was not specific to the whole-probe method. The analysis over our Experiments 1 and 2 further confirmed this conclusion.

General discussion

The goal of the present study was to investigate whether irrelevant high-discriminable information could be selected into VWM under high VWM load. By using the same feature dimensions (i.e., distinct color and shape) as in Xu (2010), in Experiment 1 we showed that under high VWM load, an irrelevant change of shape impaired performance on the target in a whole-probe change detection task. Experiment 2 confirmed the conclusion of Experiment 1 using a partial-probe method. These results together suggest that irrelevant high-discriminable features can be stored in VWM under high VWM load (at least for six objects).

Replicating previous behavioral (e.g., Hyun et al., 2009), ERP (e.g., Gao et al., 2010), and fMRI (e.g., Xu, 2010) studies, we showed that an irrelevant high-discriminable feature could be extracted into VWM under low VWM load. However, in contrast to the finding of Xu (2010) that this object-based encoding vanished under high VWM load, in two experiments we consistently found evidence suggesting that the object-based encoding existed even under high VWM load. Importantly, this finding was revealed using the same object features used in Xu’s study, and it was found not only using new probe presentation procedures (Exps. 1 and 2b), but also using the same probe presentation manner Xu had used (Exp. 2a). Therefore, the present study implies that the involuntary selection of an irrelevant high-discriminable feature (at least for shape) is a fairly robust process.

Since Xu (2010) provided evidence that the LOC could reflect the storage of irrelevant shape information (i.e., the result under low VWM load), why were the findings under high VWM load different between Xu’s study and the present one? While we do not have clear answers at this stage, four possible explanations should be considered. First, Xu assumed that LOC was the only neural substrate in charge of the shape processing. However, previous neuroimaging studies have demonstrated that other regions are also involved in shape processing (e.g., Denys et al., 2004; Yee, Roe, & Courtney, 2010). It will be interesting to check these additional regions in the future. Second, the LOC may be not sensitive enough to the processing activity of the irrelevant shape, since the shape was not the attentional focus, and its neural activation was thus fairly weak. Indeed, Xu found that statistically significant LOC activation was exhibited only during the encoding phase of the task, not during the maintenance phase, which led Xu to suggest that the object-based encoding was rather transient. Third, perhaps the different task settings in the two studies made the participants adopt different encoding strategies. In Xu’s study, the irrelevant shape did not change through a trial, yet in the present study it changed on 50% of trials. The participants, hence, may not have selected the irrelevant shape in Xu’s study, but did select it in our study. The available evidence does not support this possibility: Xu found evidence that the irrelevant shape was processed in the low-load condition, and we had informed the participants that encoding the irrelevant shape would be harmful to task performance. Finally, since Experiment 2 revealed a reduced distracting effect on criterion under high load, perhaps the VWM load in Xu’s study was higher. Obviously, this alternative was not possible, particularly considering that the participants in our study needed to suppress the response to the irrelevant change. Of course, future work may need to test object-based encoding under higher VWM load.

Unlike the predictions of perceptual-load theory, which claim that an irrelevant object should be excluded from processing under high processing load, we found that the irrelevant feature was selected into VWM in our conditions. However, it should be noted that the present finding should not be taken as evidence implying that VWM and visual perception have distinct processing mechanisms. On the one hand, perceptual-load theory mainly elaborates the processing of objects (targets plus distractors) presented at distinct spatial locations, while the present study investigated the processing of visual features (relevant plus irrelevant features) occupying the same spatial locations. Both the study of perceptual load (Chen, 2003) and our recent study on VWM (Zhou et al., 2011) have provided evidence suggesting that the processing mechanisms for the two conditions are different. On the other hand, there has been tentative evidence (Arend & Zimmer, 2011) implying that when distractors and targets are presented in different locations, a mechanism similar to the one suggested by perceptual-load theory exists in VWM.

Finally, the present study did not find clear evidence supporting that the ensemble representation played a role in the irrelevant-change distracting effect. However, the different patterns of distracting effects between Experiments 1 and 2 leave this possibility open. In particular, although we replicated Jiang et al.’s (2000) finding that the response criterion is more stringent in the partial-probe than in the whole-probe method, and though this finding has been related to the change of spatial configuration, it could also be explained (at least partially) in terms of ensemble representation, which was lost in the test array in the partial-probe method. Similarly, the different result patterns between our two experiments (e.g., the distracting effect exhibited on d' in Exp. 1, but on criterion in Exp. 2) may also be related to the ensemble representation. Future work is needed to address this issue.