Stored color–form knowledge modulates perceptual sensitivity in search

Abstract

We used signal detection analysis to examine the effects of stored color–form knowledge on visual search. Across four experiments, we showed robust effects of stored color–form knowledge on perceptual sensitivity, whereas the effects on response criteria varied. The effects on perceptual sensitivity were stronger when multiple items were present and when the color fell on the surface of the object. The benefit was found even when the correctly colored target had a low probability of occurrence in the experiment. In addition, the benefit was present across different display sizes, and the effects of increasing the exposure duration on detecting correctly colored targets were equal across different sizes of the display. The data suggest that color–form conjunctions are detected efficiently to activate stored color–form knowledge, and that this knowledge then influences early perceptual processing in a bottom-up manner. We discuss the implications for understanding the coding of conjunctive relations.

The process of binding multiple features into a coherent representation of an object has long been assumed to be attention-demanding (Treisman, 1998). Support for this idea has come from studies contrasting efficient search for targets defined by single features with inefficient search, increasing linearly with the number of distractors present, when the target is defined by the relations between features (Treisman & Gelade, 1980). These findings suggest that the binding of features is contingent on attention being applied serially to each stimulus in the display. Converging evidence for this has come from studies of illusory conjunctions, in which feature relations are misreported when full attention to a display is prevented (Treisman & Schmidt, 1982).

However, there appear to be some exceptions to these findings. For example, search for target is efficient and apparently spatially parallel if one of the target-defining features is stereo disparity (Nakayama & Silverman, 1986; Steinman, 1987) or motion (Driver, McLeod, & Dienes, 1992; McLeod, Driver, & Crisp, 1988). More recently, it has been shown that feature relations between shapes and colors, which classically yield inefficient search, can be detected efficiently when the feature relationship is learned (Rappaport, Humphreys, & Riddoch, 2013). For example, fruit and vegetable targets can “pop out” from displays containing distractors similar in shape and color when the items carry their familiar colors—though search remains inefficient if the same items are presented in unfamiliar colors. These findings demonstrate that stored knowledge regarding color–shape bindings influences search, and as a result, objects in familiar colors do not encounter the same processing limitations as those carrying arbitrary color–shape relations.

An open question concerns the underlying changes in processing that lead to more efficient search for objects in familiar colors. Rappaport et al. (2013) proposed that the brain establishes in parallel conjunctive representations containing both shape and color information. These representations, coded from the bottom up, draw attention to themselves if they activate stored knowledge for the color–form conjunctions. Consistent with the argument for bottom-up coding of learned conjunctions, Rappaport et al. found that targets with learned conjunctive relations (e.g., yellow corn) were found efficiently, even when the targets were more likely to occur in a predictable but unfamiliar color (e.g., if purple corn was likely in the experiment). Interestingly, when these targets were absent, first saccades tended to be made to distractors carrying the predicted target color (purple). Thus, though the top-down set was to a predictable but unfamiliar color, targets in a learned color nonetheless “popped out.”

But what processes do bottom-up-coded conjunctions affect? Does the activation of stored representations change the perceptual processing of the items, generating a stronger perceptual signal? Alternatively, does the activation of stored knowledge change the response criteria that observers adopt, so that target responses are made on the basis of a lower decision threshold? These questions are difficult to address by measuring reaction times (RTs). RTs have been shown to reflect both perceptual and later decision response-related processes (e.g., Handy, Kingstone, & Mangun, 1996; Prinzmetal, McCool, & Park, 2005; Theeuwes & Van der Burg, 2007). Signal detection theory (SDT), on the other hand, allows for the separation of effects on early perceptual processing and on later stages of processing (McDonald, Teder-Sälejärvi, & Hillyard, 2000). SDT measures typically reflect early perceptual processing when stimuli are presented for a short duration and followed by masks (McDonald et al., 2000; Prinzmetal et al., 2005; Soto, Wriglesworth, Bahrami-Balani, & Humphreys, 2010; Theeuwes & Van der Burg, 2007). Therefore, instead of measuring RTs, we assessed the effects of stored representations on perceptual processing using an SDT approach. We measured perceptual sensitivity, indexed by A′, and bias, measured in b, in a series of visual search tasks with short display durations and backward masks. A′ is a nonparametric alternative to d′ as an index of sensitivity (Zhang & Mueller, 2005).

In Experiment 1, our participants were asked to detect the presence or absence of a target while the display size was manipulated and participants were presented with either one or four items at a time. All of the items were highly familiar (corn, carrot, eggplant, and lemon) and were presented in either their correct color (yellow corn, orange carrot, purple eggplant, and yellow lemon) or in an incorrect color (the colors of the items were reassigned to other items in the display, so that the same colors were always present; see Fig. 1). First, we were interested in whether the benefit of carrying a familiar color, as reported by Rappaport et al. (2013), would be reflected by changes in perceptual sensitivity and/or in response bias. Second, we assessed whether the familiar-color advantage remained even when only one item was present, when the need for binding was decreased (see Friedman-Hill, Robertson, & Treisman, 1995). In Experiment 2, we examined whether this advantage was due to the mere presence of the learned color, or if it was specifically related to the presence of the familiar conjunctive relationship. Again, we presented participants with a search display, except that here, on half of the trials, the color did not appear on the surface of the object but in the surrounding background, whereas the items themselves were gray. Was the presence of the correct color in the local background sufficient to modulate perceptual processing or the response criterion that participants adopted? In a third experiment, we pitted top-down expectancies against color knowledge and presented incorrectly colored targets on the majority of trials. We reasoned that if any advantage for correctly colored targets is driven by expectations in the search task, then the effect should disappear when the target is presented more often in a particular incorrect color and the incorrect, rather than the correct, color is expected. In a final set of experiments, we employed a variety of display sizes to examine whether changes in perceptual sensitivity and bias with target color correctness had capacity limitations.

Fig. 1

Images of stimulus pairings used in the surface (left) and background (right) conditions. Please note that color combinations are kept consistent between the surface and background conditions

Experiment 1

Method

Participants

Twenty-three students (14 female, nine male; age range 18–30 years, all right-handed) from the University of Oxford, England, took part in the experiment as paid volunteers or for course credits. In this and all following experiments, the participants were recruited using convenience sampling. Four additional participants were tested but excluded from the data analysis due to chance performance (i.e., 50 % accuracy) in one or more conditions. All participants had normal or corrected-to-normal vision and were naïve to the purpose of the experiment. Participants gave informed consent before taking part in the study and were reimbursed with money or course credit for their participation. The methods in this and subsequent experiments were approved by the Oxford Central University Research Ethics Committee (CUREC).

Materials

Stimulus presentation and response collection was controlled using E-Prime 2.0. Stimuli were displayed on a 23-in. LED with a spatial resolution of 1,920 × 1,080 and a vertical refresh rate of 60 Hz, placed at 50 cm in front of the participant. Manual responses were registered on the keyboard.

Stimuli

The stimuli were taken from Rappaport, Humphreys, and Riddoch (2013) and consisted of four pictures depicting corn, lemon, eggplant, and carrot (see Fig. 1). These items were selected because they each have a diagnostic color; that is, their shape and color are highly associated. Items were presented in three different colors: yellow (diagnostic for corn), purple (diagnostic for eggplant), or orange (diagnostic for carrot). The colors were exchanged using GIMP 2.6 image manipulation software and subtended approximately 2.36° × 2.36°. The four possible stimulus locations were equidistant from the center of the screen and from one another.

Design and procedure

The experiment consisted of a 2 (display size: one item, four items) × 3 (color: yellow, purple, orange) within-subjects design. All variables were manipulated randomly within blocks, with the only restriction being that all trial types occurred equally often. On one-item trials, the stimulus location was randomly selected. Whether the color of the items was correct or incorrect was maintained across all items for any given trial (i.e., when the target was correctly colored, so were the distractors; similarly, when the target carried an incorrect color, so did the distractors). Participants completed four blocks of 180 trials each, yielding a total of 720 trials per participant. Each trial started with a 500-ms presentation of a small black fixation cross at the center of the screen, which participants were instructed to fixate. This was followed by presentation of the stimulus display, which remained for 100 ms, after which a mask, consisting of white noise, was displayed for 200 ms at the locations where the stimuli had appeared. The mask was followed by a blank screen that remained until a response was made, and the next trial started after a 2,000-ms blank period. Please refer to Fig. 2 for the trial procedure. Participants were asked to indicate the presence or absence of the target item corn (regardless of its color) by using the “G” and “H” buttons on the keyboard. Participants completed a practice session of 20 trials to ensure that they understood the task.

Fig. 2

Trial procedures for Experiments 1 (A), 2 (B), 3 (B,–top only), 4a (C), and 4b and 4c (both D)

Data analysis

To simplify the data analysis, the incorrect color conditions, in which targets were presented in orange or purple, were collapsed. Only trials with responses occurring between 100 and 1,500 ms were included in the analysis. We calculated perceptual sensitivity, A′, and response bias, b, following the formulas provided by Zhang and Mueller (2005, pp. 207–209).

We chose A′ and b as our indices of sensitivity and bias instead of d′ and criterion c in order to avoid making distributional assumptions regarding our data that could not be confirmed (Macmillan & Creelman, 1996; Zhang & Mueller, 2005). When b is equal to 1, observers are unbiased. As observers become more biased to say “present,” b approaches 0. As the bias to say “absent” increases, b increases over 1 on an open-ended scale. Zhang and Mueller noted that estimates of A′ and b might be unreliable in some regions of receiver-operating characteristic (ROC) space (pp. 207–208 and Fig. 3)—specifically, at low to intermediate numbers of hits (H) coupled with low false alarms (F), and at H close to 1 coupled with F close to 1. To ensure that data points in the experiments reported here did not fall in these regions, and the that A′ and b estimates were reliable, we plotted F against H (averaged across conditions) for each participant separately (see Supplementary Fig. 1). Visual inspection of the graphs suggested that the data points fell in reliable regions of ROC space.

Fig. 3

Mean A′ for correctly and incorrectly colored targets for display sizes 1 and 4 in Experiment 1. Error bars reflect ±1 standard error

Results

The mean A′ and b scores are shown in Figs. 3 and 4 separately for each display size and color condition. Accuracy averaged around 84.2 %.

Fig. 4

Mean bias b for correctly and incorrectly colored targets for display sizes 1 and 4 in Experiment 1. Error bars reflect ±1 standard error

A′

For our statistical analysis, a 2 × 2 repeated measures analysis of variance (ANOVA) including the variables display size (one vs. four items) and color (correct vs. incorrect) was calculated over the mean scores for A′. The ANOVA yielded a significant main effect of display size [F(1, 23) = 70.920, p < .001, η ² = .755] and a significant main effect of target color [F(1, 23) = 11.511, p = .003, η ² = .334]. The Display Size × Target Color interaction was also significant [F(1, 23) = 7.592, p = .011, η ² = .248], reflecting that sensitivity was greater for correctly colored targets only at display size 4 (correct–incorrect display size 4, t(23) = 3.544, p = .002; correct–incorrect display size 1, t(23) = 1.263, n.s.].

Bias b

The same 2 × 2 repeated measures ANOVA was conducted over the mean b values. The ANOVA yielded a significant Display Size × Target Color interaction [F(1, 23) = 4.469, p = .046, η ² = .163], although subsequent t tests failed to establish any reliable differences between individual pairs of conditions. The mean b values to incorrectly colored targets at display size 1 were significantly different from 1 [t(23) = 2.556, p = .018].

Discussion

We found a significant modulation of perceptual sensitivity by target color only when observers searched for an item in a search display containing distractors (display size 4), but not when only one item was present at one of four possible locations. This suggests that under conditions of reduced binding demands, the color advantage effect disappears. Changes in response bias, on the other hand, were not systematically modulated by the items carrying correct or incorrect colors.

Experiment 2

If the observed effects were due to the activation of stored representations containing “bound” information for the shape with a familiar surface color, then any advantage for correctly colored targets should only be observed when the color falls on the surface of the target, and not when the color falls in the local background. This was tested in Experiment 2, in which the colors fell either on the surface of the objects or in the local background (see Fig. 2). In this background condition, the stimulus display remained on screen for 200 ms (and not 100 ms, as when the color fell on the surface of the object), because pilot data suggested that the task would become too difficult otherwise. The two presentation conditions were presented in blocks, and the order of the blocks was counterbalanced across participants. To reduce the number of conditions, only a single display size (6) was used.

Methods

Participants

Fifteen students (14 female, one male; age range 18–30 years, all right-handed) from the University of Oxford, England, took part as paid volunteers or for course credits. Seven additional participants were tested but excluded from the data analysis due to chance performance (i.e., 50 % accuracy) in one or more conditions. All of the participants had normal or corrected-to-normal vision and were naïve to the purpose of the experiment. Participants gave informed consent before taking part in the study.

Materials, stimuli, design, procedure, and data analysis

These were exactly the same as in Experiment 1, except for the above-mentioned changes. In the background condition, the stimuli subtended the same 2.36° × 2.36° rectangle as the stimuli in the surface condition, but the area not occupied by the stimulus was filled in with color. The six stimuli were presented on the outline of an imaginary circle with a radius of 4.23° of visual angle equidistant from one another and from the center of the screen. If a target was present, five distractors were randomly chosen from the selection of distractors (lemon, eggplant, and carrot). The colors of the distractors were kept constant—that is, the distractors were always presented in the same specific incorrect color for a specific incorrect target color. When no target was present, six distractors were randomly selected, with the constraint that either all or none of the distractors were presented in their correct color, and in incorrect-color trials, stimuli of the same identity were always presented in the same incorrect color (e.g., two purple lemons, but never a purple lemon with an orange lemon).

Results

The mean A′ and b scores are shown in Figs. 5 and 6 separately for each presentation and color condition. Accuracy was on average 71.4 %, and it was significantly modulated by color.

Fig. 5

Mean A′ for correctly and incorrectly colored targets in the two presentation conditions in Experiment 2. Error bars reflect ±1 standard error

Fig. 6

Mean bias b for correctly and incorrectly colored targets in the two presentation conditions in Experiment 2. Error bars reflect ±1 standard error

A′

For our statistical analysis, a 2 × 2 repeated measures ANOVA including the variables presentation condition (surface vs. background) and color (correct vs. incorrect) was calculated for A′. The ANOVA yielded a significant main effect of target color [F(1, 14) = 10.719, p = .006, η ² = .434]. Importantly, the Presentation Condition × Target Color interaction was also significant [F(1, 14) = 4.862, p = .045, η ² = .258]: Sensitivity was greater for correctly colored targets only in the surface presentation condition [correct–incorrect surface condition, t(14) = 3.021, p = .009; correct–incorrect background condition, t(14) = 0.451, p = .659].

Bias b

The same ANOVA, conducted over the mean b values, yielded a significant Presentation Condition × Target Color interaction [F(1, 14) = 13.035, p = .003, η ² = .482], reflecting that bias was significantly greater for correctly colored targets than for incorrectly colored targets in the background condition only [correct–incorrect surface condition, t(14) = –1.785, n.s.; correct–incorrect background condition, t(14) = 3.036, p = .009]. None of the mean bias values were significantly different from 1 (all ts < ±1.5).

Discussion

Perceptual sensitivity for correctly colored targets was significantly improved when the color was presented on the surface of the objects, but not when it was presented as the target’s background. This suggests that the correct color advantage is driven by bound representations of color and shape (see also Rappaport et al., 2013). Bias, on the other hand, only changed in the background condition.

Experiment 3

In this experiment, we further examined the effect of color correctness on perceptual sensitivity and manipulated top-down expectancies regarding the target color. If the color effect is driven by expectations regarding the target color, then the effect should disappear when the target is more likely, and therefore expected, to be presented in a particular incorrect color. On the other hand, if enhanced perceptual sensitivity reflects the bottom-up activation of stored conjunctive relations for stimuli, then the effect should remain even when targets are more likely to appear in an incorrect color (see Rappaport et al., 2013). To test this, we ran a study in which the target was more likely to appear in a particular incorrect color (purple, on 70 % of the trials) than in the correct or the other incorrect color (the remaining 30 % of trials). Participants were not explicitly told about the different likelihoods of target color correctness.

Method

Participants

Eleven students (nine female, two male; age range 18–30 years, one left-handed) from the University of Oxford, England, took part as paid volunteers or for course credits. Three additional participants were tested but excluded from the data analysis due to chance performance (i.e., 50 % accuracy) in one or more conditions. All participants had normal or corrected-to-normal vision and were naïve to the purpose of the experiment. Participants gave informed consent before taking part in the study.

Materials, stimuli, design, procedure, and data analysis

These were exactly the same as in Experiment 1, except that the experiment consisted of only 300 trials and there was just a single display size of six items. The six stimuli were presented on the outline of an imaginary circle with a radius of 4.23° of visual angle equidistant from one another and from the center of the screen. If a target was present, five distractors were randomly chosen from the selection of distractors (lemon, eggplant, and carrot). The colors of the distractors were kept constant—that is, distractors were always presented in the same specific incorrect color for a specific incorrect target color. When no target was present, six distractors were randomly selected, with the constraint that either all or none of the distractors were presented in their correct color, and in incorrect-color trials, stimuli of the same identity were always presented in the same incorrect color (e.g., two purple lemons, but never a purple lemon with an orange lemon).

Results

The mean A′ and b scores are shown in Figs. 7 and 8 separately for each color condition. Accuracy was on average 70.4 % correct.

Fig. 7

Mean A′ for correctly and incorrectly colored targets in Experiment 3. Error bars reflect ±1 standard error

Fig. 8

Mean A′ for correctly and incorrectly colored targets separately for first, second and third part of Experiment 3. Error bars reflect ± 1 standard error

A′

For the statistical analysis, a paired-samples t test was calculated comparing the mean values for A′ in each of the two color conditions. The difference between the two color conditions was significant [t(10) = 2.634, p = .025, d = 0.57], reflecting greater A′ for correctly colored targets.

Since participants were not explicitly instructed about the relative proportions of target colors, it may have taken some time for them to learn the probabilities and update their top-down expectancies regarding the target color. To examine whether the color advantage changed over the course of the experiment, we assessed the color effects separately for the first, second, and third blocks of the experiment in a 2 (color: correct, incorrect) × 3 (block: first, second, third) repeated measures ANOVA (see Fig. 9). We observed a main effect of block [F(2, 20) = 14.714, p = .001, η ² = 1.191] and a main effect of color [F(1, 20) = 7.471, p = .021, η ² = .272]. The Block × Color interaction was marginally significant [F(2, 20) = 3.931, p = .046, η ² = .564], reflecting that the color advantage was not present in the third block [t(10) = –0.71, p = .49, n.s.], but was present in the first and second blocks [t(10) = 2.34, p = .042, d = 0.71; and t(10) = 4.46, p = .001, d = 1.35, respectively]. Specifically, the disappearance of the color advantage was driven by increases in sensitivity for incorrectly colored targets from Block 1 to 2 [t(10) = –2.17, p = .055] and Block 2 to 3 [t(10) = –3.61, p = .005], whereas sensitivity for correctly colored targets first increased [t(10) = –2.33, p = .042] and then remained stable [t(10) = –0.66, p = .53].

Fig. 9

Mean bias b for correctly and incorrectly colored targets in Experiment 3. Error bars reflect ±1 standard error

Bias b

The same t test was conducted over the mean b values. We found no significant difference in bias between the two color conditions [t(10) = 1.910, n.s.].

Discussion

Again, we found a significant advantage for correctly colored targets on the measure of perceptual sensitivity (A′). However, there were no differences in bias between the two conditions, suggesting that our manipulation did not influence observers’ response criteria. Critically, the advantage for the correctly over the incorrectly colored target was maintained here, even though the correctly colored target appeared with a low probability. This result matches the report of Rappaport et al. (2013) in a study measuring search RTs, and it indicates that the correct color advantage was unlikely to be due to participants holding an expectation for the target to be in its familiar color in the experiment.

When we examined the results more closely, we found that the color advantage changed over the course of the experiment, and in the final block perceptual sensitivity was no longer enhanced for correctly colored relative to incorrectly colored targets. This suggests that participants updated their top-down expectancies over the course of the experiment and that the trial-proportion manipulation was successful. Importantly, however, the disappearance of the color advantage was driven by an increase in sensitivity for incorrectly colored targets, while sensitivity for correctly colored targets first increased and then remained stable. If the color advantage were driven completely by top-down expectancies, one would expect a drop in performance for correctly colored targets—which were no longer expected—along with the increase in performance for incorrectly colored targets. Thus, although top-down expectancies play a role in the color advantage, these results suggest that a strong bottom-up component drives the benefit.

Experiment 4

In Experiment 4, we investigated whether the correct-color advantage might be modulated by display size and/or display duration. In three subexperiments, we presented participants with (a) four or six items, (b) to eight or twelve items, and (c) eight or twelve items while (independently) varying the display duration between 100 and 200 ms. We asked whether capacity limitations on the binding of familiar color–form relations would emerge as the display size increased.

Experiment 4a

Method

Participants

Sixteen students (nine female, seven male; age range 18–30 years, all right-handed) from the University of Oxford, England, took part as paid volunteers or for course credits. Six additional participants were tested but excluded from the data analysis due to chance performance (i.e., 50 % accuracy) in one or more conditions. All of the participants had normal or corrected-to-normal vision and were naïve to the purpose of the experiment. Participants gave informed consent before taking part in the study.

Materials, stimuli, design, procedure, and data analysis

These were exactly the same as in Experiment 1, except that the display size was varied between four and six items and the background condition was included (as in Exp. 2). When only four items were presented, the two placeholders to the left and right of the central fixation cross were left blank, so that items appeared at the four outer locations (see Fig. 2).

Results

The mean A′ and b scores are shown in Figs. 10 and 11 separately for each display size, presentation, and color condition. The accuracy was on average 81.3 % correct.

Fig. 10

Mean A′ for correctly and incorrectly colored targets for each display size and presentation condition in Experiment 4a. Error bars reflect ±1 standard error

Fig. 11

Mean bias b for correctly and incorrectly colored targets for each display size and presentation condition in Experiment 4a. Error bars reflect ±1 standard error

A′

A 2 × 2 × 2 repeated measures ANOVA including the variables presentation condition (surface vs. background), color (correct vs. incorrect), and display size (four vs. six) was carried out. The ANOVA yielded significant main effects of presentation condition [F(1, 15) = 11.965, p = .004, η ² = .444], display size [F(1, 15) = 38.452, p < .001, η ² = .719], and target color [F(1, 15) = 25.599, p < .001, η ² = .631]. The Presentation Condition × Display Size interaction was also significant [F(1, 15) = 5.8040, p = .040, η ² = .251], reflecting that the decrease in A′ with an increase in display size was more pronounced in the surface condition. Importantly, the Presentation Condition × Target Color interaction was also significant [F(1, 15) = 4.631, p = .048, η ² = .236]: The advantage for correctly colored targets was more pronounced in the surface condition than in the background condition (mean difference surface = .065, mean difference background = .032). The Presentation Condition × Display Size × Target Color interaction was not significant [F(1, 15) = 2.303, p = .150, η ² = .133].

Bias b

The same ANOVA was conducted over the mean b values. The ANOVA yielded a significant main effect of target color [F(1, 15) = 19.747, p < .001, η ² = .586]. The Presentation Condition × Target Color interaction was also significant [F(1, 15) = 13.981, p = .002, η ² = .482]: The bias score was significantly greater for correctly colored targets than for incorrectly colored targets in the background condition only [correct–incorrect surface condition, t(15) = 1.471, n.s.; correct–incorrect background condition, t(15) = 6.584, p < .001]. Only bias values for correctly and incorrectly colored targets at display size 4 in the surface presentation condition were significantly different from 1 (t = 3.52, p = .003, and t = 3.63, p = .002, respectively); that is, observers were more biased to say “absent” in these conditions.

Discussion

There was again a color benefit on perceptual sensitivity, but it did not interact with the display size here. However, the display size manipulation was relatively weak in this experiment, and differences might emerge with the stronger manipulations that we employed in the next two experiments. Interestingly, and in contrast to Experiment 2, here we did also find a significant color advantage effect in the background condition. The mere presence of the familiar target color in the display was sufficient to enhance perceptual sensitivity, though the gain was much larger when the familiar color fell on the surface of the object. In the background condition, the presence of the correct color also modulated the response criteria.

Experiment 4b

Method

Participants

Ten students (eight female, two male; age range 18–30 years, one left-handed) from the University of Oxford, England, took part as paid volunteers or for course credits. All participants had normal or corrected-to-normal vision and were naïve to the purpose of the experiment. Participants gave informed consent before taking part in the study and were reimbursed with money or course credit for their participation.

Materials, stimuli, design, procedure, and data analysis

These were exactly the same as in Experiment 1, except that the experiment consisted of 320 trials and the display size was varied between eight and twelve items. The items were presented, equidistant from one another, along the outlines of three imaginary, central circles with radii of 3.23°, 6.23°, and 8.23° of visual angle. If a target was present, seven or eleven distractors were randomly chosen from the selection of distractors (lemon, eggplant, and carrot), with the constraint that each distractor type was represented in a given trial. The colors of the distractors were kept constant—that is, distractors were always presented in the same specific incorrect color for a specific incorrect target color. When no target was present, 12 distractors were randomly selected, with the constraint that either all or none of the distractors were presented in their correct color, and in incorrect-color trials, distractors of a specific identity were always presented in the same incorrect color (e.g., two purple lemons, but never a purple lemon with an orange lemon).

Results

The mean A′ and b scores are shown in Figs. 12 and 13 separately for each display size and color condition. Accuracy was on average 71.5 %.

Fig. 12

Mean A′ for correctly and incorrectly colored targets for each display size in Experiment 4b. Error bars reflect ±1 standard error

Fig. 13

Mean bias b for correctly and incorrectly colored targets for each display size in Experiment 4b. Error bars reflect ±1 standard error

A′

A 2 × 2 repeated measures ANOVA was conducted with the variables display size (eight vs. twelve) and color (correct vs. incorrect). The ANOVA yielded significant main effects of target color [F(1, 9) = 5.225, p = .048, η ² = .367] and display size [F(1, 9) = 32.627, p < .001, η ² = .784]. The Display Size × Target Color interaction was not significant [F(1, 9) = 0.535, p = .483, η ² = .056].

Bias b

The same ANOVA was conducted over the mean b values. This yielded a significant main effect of target color [F(1, 9) = 8.648, p = .016, η ² = .490]. The Display Size × Target Color interaction was marginally significant [F(1, 9) = 4.839, p = .055, η ² = .350]: The bias measure was increased for incorrectly colored targets, particularly at the smaller display size.

Discussion

As in Experiment 4a, an effect was visible of having the correct color on the surface of the target, but this was again not modulated by the display size. However, an influence of display size might emerge if the demands on the visual system were further increased by not only having larger display sizes, but also changing the exposure duration. To examine this question, we ran the same experiment again, but added a display duration manipulation and randomly varied the duration between 100 and 200 ms. In Experiment 4b, the correct color also had some effect on the bias measure, indicating that participants adopted a less strict response criterion when responding to the correctly colored target. In the other experiments this result did not occur and a shift in response criteria did not appear to be necessary to produce the change in perceptual sensitivity.

Experiment 4c

Method

Participants

Thirteen students (12 female, one male; age range 18–30 years, one left-handed) from the University of Oxford, England, took part as paid volunteers or for course credits. Eight additional participants were tested but excluded from the data analysis due to chance performance (i.e., 50 % accuracy) in one or more conditions. All participants had normal or corrected-to-normal vision and were naïve to the purpose of the experiment. Participants gave informed consent before taking part in the study and were reimbursed with money or course credit for their participation.

Materials, stimuli, design, procedure, and data analysis

These were exactly the same as in Experiment 4b, except that the display duration was varied between 100 and 200 ms.

Results

The mean A′ and b scores are shown in Figs. 14 and 15 separately for each display size, display duration, and target color. Accuracy was on averaging 74.3 % correct.

Fig. 14

Mean A′ for correctly and incorrectly colored targets for each display size and duration in Experiment 4c. Error bars reflect ±1 standard error

Fig. 15

Mean bias b for correctly and incorrectly colored targets for each display size and duration in Experiment 4c. Error bars reflect ±1 standard error

A′

A 2 × 2 × 2 repeated measures ANOVA was conducted including the variables display size (eight vs. twelve), display duration (100 vs. 200 ms), and color (correct vs. incorrect). This yielded significant main effects of target color [F(1, 12) = 8.283, p = .014, η ² = .408], display size [F(1, 12) = 72.332, p < .001, η ² = .858], and display duration [F(1, 12) = 12.154, p = .004, η ² = .503]. We also observed a significant three-way interaction between target color, display size, and display duration [F(1, 12) = 6.164, p = .029, η ² = .339].

To further investigate the three-way interaction, we conducted two separate 2 × 2 ANOVAS with the variables color and display duration for each display size condition. The first ANOVA, at display size 12, revealed a trending Color × Display Duration interaction [F(1, 12) = 3.276, p = .095, η ² = .215]. Importantly, a t test comparing correct versus incorrect color at a long display duration revealed a highly significant difference [t(12) = 3.3268, p = .006, d = 0.92], whereas there was no difference between correct and incorrect targets at a short display duration [t(12) = 0.01, p = .99, d < 0.01]. At display size 8, the interaction was not significant [F(1, 12) = 1.057, p = .324, η ² = .081], whereas the main effect of display duration was [F(1, 12) = 13.207, p = .003, η ² = .524], and the main effect of color approached significance [F(1, 12) = 4.109, p = .065, η ² = .255]. Thus, the three-way interaction was driven by the difference at display size 12, where correctly colored targets benefited from the longer exposure duration, whereas incorrectly colored targets did not.

Bias b

The same ANOVA was conducted over the mean b values. The ANOVA yielded a main effect of color [F(1, 12) = 35.170, p < .001, η ² = .746], reflecting that bias was greater for incorrectly colored targets. A trend toward the Display Size × Display Duration interaction also emerged [F(1, 12) = 3.930, p = .071, η ² = .247], reflecting a slight increase in bias for the long relative to the short display duration at display size 12 only. None of the other effects were significant. The mean bias values for incorrectly colored targets with the short display duration at a display size of eight items were significantly different from 1 [t(13) = 3.421, p = .005]—that is, observers were more biased to say “absent.”

Discussion

Here, when we varied not only the display size but also the exposure duration of the search display, we observed a subtle modulation of display size on the advantage for correctly colored items. Search for incorrectly colored targets at display size 12 remained poor, no matter how long the exposure duration; that is, search did not benefit from a longer time to evaluate the visual input. In contrast, the detection of correctly colored targets was reliably better at longer display durations than at shorter display durations, at both display sizes 8 and 12.

The present results suggest that enhanced perceptual sensitivity for correctly colored targets was gained as the exposure duration increased, consistent with the activation of stored conjunctive targets taking place over time. Interestingly, however, the gain from increasing the exposure duration was additive across the two display sizes, as was the advantage for correct over incorrectly colored targets. This pattern of results suggests an efficient accumulation of information from the stimuli, leading to the activation of stored knowledge about the familiar color–form relations. In contrast to this, the coding of arbitrary color–form relations (when the target was in an incorrect color) was subject to relatively severe processing limitations. In this case, increasing the exposure duration allowed the target to be detected at the smaller display size (eight items), but not when the display size was larger (12 items). To account for this, we propose that incorrectly colored targets depended on a less efficient allocation of attention to the display, and at the larger display size, there was insufficient time for attention to be guided reliably to the target, irrespective of the exposure duration.

As in Experiment 4b, we found an effect of the correct color on response bias, with the response criterion being higher for incorrectly colored targets.

General discussion

Across four experiments, we have presented evidence of enhanced perceptual sensitivity when participants search for familiar targets presented in their correct color. This result is consistent with stored color–form knowledge affecting early perceptual processing of the stimuli. The benefits of presenting correctly colored targets increase when there are demands to select the target from distractors (Exp. 1), though the evidence also suggests that the coding of familiar color–form conjunctions is not strongly modulated by display sizes of up to 12 items. For example, although performance decreased at the larger display sizes, equal benefits from increasing the exposure duration emerged at the two display sizes; this is consistent with equivalent gains in processing across the different display sizes (Exp. 4). The results with correctly colored objects contrast with those found for incorrectly colored objects, in which there was a gain from longer exposures only at the smaller display size—suggesting that the increased exposure then enabled serial selection to be applied to eight items, but not to 12. These effects from the correct color were at best much weaker when the color fell in the local background of the objects (Exps. 2 and 3), indicating that the color and form need to integrated in the appropriate spatial relation in order for the benefits of the familiar color to emerge. The mere presence of both the color and the form in the display is not sufficient.

The effects of stored color–form knowledge on perceptual sensitivity also arose even when the familiar color was unlikely to occur and the target was more likely to be an unfamiliar, but predictable, color (Exp. 3). This result matches the findings reported by Rappaport et al. (2013), who also manipulated whether the familiar color–form target was likely or unlikely to occur in the experiment. Rappaport et al. also found that, when the target was absent, the first saccade went to a distractor with the expected (in this case, unfamiliar) color when that color was likely to occur, indicating that top-down biases were set to the expected color. Both studies, then, suggest that stored color–form knowledge is contacted in a bottom-up manner. The present result extends that of Rappaport et al. by demonstrating that this bottom-up effect modulates ongoing perceptual sensitivity. A different interpretation of the results of Experiment 3 is that a Bayesian updating mechanism takes place, through which the relative weights of color–shape conjunctions are updated over the course of the experiment along with incoming information. This interpretation would not require a differentiation of processing into top-down and bottom-up mechanisms.

Our study does have some limitations. Firstly, in all experiments the search target was yellow corn, and the results might in principle not generalize beyond this one specific color–shape conjunction to other target shapes and colors. However, similar results have been reported for search tasks in which the targets consisted of other familiar color–shape targets (Rappaport et al., 2013). Thus, the results reported here reflect a general benefit for familiar color–shape conjunctions, and not some unique relationship between yellow corn and the distractor shapes used in our experiments. Furthermore, a significant proportion of participants were excluded due to chance performance, and our conclusions may only apply to people who are good at performing visual search tasks under challenging conditions. However, it was important to render the tasks perceptually challenging by using very short presentation durations and backward masks, to avoid ceiling effects and to allow for analysis of our data using signal detection approaches. Importantly, as we already mentioned above, similar conclusions regarding the effects of stored color–form knowledge and perceptual processing have been drawn using easier task procedures that allowed for the inclusion of all participants (Rappaport et al., 2013). It is conceivable, however, that only good performers might show an effect, whereas low performers—who are better than chance in easier task procedures but at chance at a challenging task such as ours—might not. On average, an effect would still be expected, and the findings of Rappaport et al. do, therefore, not unambiguously show that the color advantage effect is not specific to high performers. Future studies could address this possibility by adjusting perceptual difficulty for each participant individually in order to explore individual differences in, and the extent of the generalizability of, the effect.

Finally, as a result of convenience sampling and task difficulty, the numbers of participants differed between experiments, and Experiments 2 to 4 had much smaller sample sizes than did Experiment 1. Some null effects may have been observed because the studies lacked statistical power due to their small sample sizes. To ensure that this was not the case, we performed a post-hoc power analysis using G*Power (Faul, Erdfelder, Lang, & Buchner, 2007) to estimate the sample size needed to find a statistically significant effect, given the size of the color effect observed in Experiment 1 (η ² = .334), with α = .05 and β = .95. This analysis suggested that a sample size of around nine participants would be sufficient to observe effects of color on performance (with β = .95). Thus, whereas the numbers of participants were lower in Experiments 2–4 than in Experiment 1, a power analysis using the effect size from Experiment 1 suggests that sample sizes were adequate in all experiments to observe an effect of the size of the color benefit observed in Experiment 1. Importantly, similar sample size estimates were obtained when using effect size values from the interaction effects observed in Experiment 1. Finally, the fact that we observed significant three-way interaction effects in Experiment 4 suggests that our experimental designs had enough statistical power even for subtle effects.

Other evidence has also indicated that the brain seems capable of learning color–form conjunctions. For example, Walsh, Ashbridge, and Cowey (1998) had participants search for color–form conjunction targets while transcranial magnetic stimulation (TMS) was applied to posterior parietal cortex. Early on in practice, the TMS disrupted conjunction search, but after more extended practice the effects of parietal TMS on the conjunction task were no longer apparent. Walsh et al. suggested that learning of conjunction codes took place over trials, which subsequently reduced the demands for attention-dependent processes in posterior parietal cortex. This concurs with emerging neurophysiological evidence for the existence of conjunctive representations that are found from early cortical regions through to infero-temporal cortex (Rust & DiCarlo, 2010; Seymour, Clifford, Logothetis, & Bartels, 2009, 2010).

There is also additional evidence for rapid, bottom-up activation of such stored representations. Lu et al. (2010), using event-related potentials (ERPs), found modulation of early ERP components when objects were presented in their familiar, learned colors, as compared with when their colors were incorrect. The present results fit with these data in showing effects that are sufficiently rapid to modulate perceptual sensitivity and that are robust across a range of display sizes.

Recent studies have further addressed this question and examined the role of automatic bottom-up and top-down attentional mechanisms in search—specifically, the exact mechanisms underlying perceptual learning in conjunctive search. For example, Frank, Reavis, Tse, and Greenlee (2014) trained observers on a conjunction search task over the course of eight days in eight separate sessions. In each session, brain activity was recorded using functional magnetic resonance imaging (fMRI). Visual search became more efficient with training, and performance was comparable to that in a control pop-out task at the end of the training. Speeded RTs were correlated with an increase in neural activity in visual areas as early as V1. The amount of neural activity in attentional regions of interest, on the other hand, remained constant over training. These findings suggest that changes in the automatic, bottom-up processing of conjunctive targets contribute to better performance, possibly because of improved signal-to-noise ratios for targets relative to distractors in visual cortex. This is consistent with the interpretation of Walsh et al. (1998), in which conjunction search becomes less reliant on attention-dependent processes over the course of learning.

A somewhat contrasting view has come from Su et al. (2014). These authors examined the effect of training on conjunction search. They found that not only did search efficiency improve, but there was also transfer of learning to other targets sharing the features of the learned conjunction. Su et al. argued that transfer is based on attentional enhancement to each of the features making up the conjunction, rather than on the creation of a new conjunctive representation.

The argument from Su et al. contrasts with our findings, in which effects did not transfer even to target stimuli with an identical shape but a different presentation style. Similarly, Rappaport et al. (2013) found that the familiar color advantage disappeared when colored geometric shapes formed the search display, as opposed to the colored fruits we used—though feature-based attention should have matched in the two cases. These discrepancies can be explained by differences in the amounts of time observers had spent learning the conjunctions. In our case, the conjunctions were learned over a lifetime and stored in long-term memory. In the Frank et al. (2014) study, observers learned over the course of eight sessions over eight days. Su et al. (2014) tested observers in one experimental session lasting 40–50 min. Thus, it might be that the mechanisms underlying efficient conjunction search and the deployment of object-based attention dynamically change over the course of learning and depend on the robustness and quality of feature associations and object representations (Chen, 2012). Perceptual learning of conjunctive targets is first driven by enhanced feature selection of each of the individual features, and over time these features become integrated to form a new functional unit at early stages of visual processing that can be processed more efficiently during visual search.

An alternative explanation for our pattern of data could be that participants use the consistency of color correctness in search displays to predict target color; that is, the detection of a correctly colored distractor might allow observers to predict that the target will be yellow in color, thus facilitating search. There are two reasons to believe that this is not the case. Firstly, this strategy should also work for incorrectly colored items, since the color relations were kept constant for all target-color variations (see Fig. 1). Secondly, Rappaport et al. (2013) orthogonally manipulated color correctness for the targets and distractors in their Experiment 2a, making it impossible for participants to predict the target color on the basis of a distractor color. The color advantage was still present in these conditions, and search for correctly colored targets was fastest when distractors were presented in an incorrect color. These results make it unlikely that the color advantage reported here was due to participants using distractor color to predict the target color.

Although we have provided robust evidence for effects of learned color–form relationships on perceptual sensitivity, the effects on participants’ response criteria were less clear. In Experiments 1 and 2 here, there were no differences in the response criteria for correctly and incorrectly colored objects, at least when the color fell on the object’s surface (see Exp. 2). In Experiments 2 and 4a, the response criterion was larger for correctly than for incorrectly colored objects when the items fell on a local color background. In Experiments 4b and 4c, however, the criteria for correctly colored objects were smaller than those for incorrectly colored objects. The reduced response criterion for correctly colored stimuli could have led participants to adopt a more liberal response when all of the items in the displays were colored correctly, but this result was not consistent across the experiments. We conclude that stored color–form knowledge has effects on perceptual sensitivity that can be unrelated to any effects on the response criterion. Though there can be effects on the response criterion, these are less robust than the changes in perceptual sensitivity and can vary in direction according to whether the surface or the background color is correct. It might be of interest to assess the effects of stored knowledge on the response criterion more systematically. For example, one might predict that changes in the probability of a target being presented in its correct color would affect response criterion. We did manipulate this in the present set of experiments, and did not observe any systematic changes in response criterion. However, participants were not informed explicitly about the color probabilities, and the probability manipulation was done between experiments. A different, within-subjects design, which emphasized changes in target-color probability, might reveal an effect of the probability of color correctness on response criterion.