Competition between color and luminance in motion correspondence

Abstract

The visual system needs to solve the correspondence problem (i.e., which elements belong together across space and time) to allow stable representations of objects. It has been shown that spatiotemporal and feature information can influence this correspondence process, but it is unclear how these factors interact with each other, especially when they are more or less prominent due to changes in contrast magnitude. We investigated this question using a variation of the Ternus display, an ambiguous apparent motion display, in which three elements can either be perceived as moving together (group motion) or as one element jumping across the others (element motion). In the first experiment, we biased the percept by presenting some of the elements with the same feature (isoluminant color or luminance), such that they were either compatible with group motion or with element motion (simple feature biases). To change the strength of the feature bias, we manipulated the contrast magnitude of the feature. In three more experiments we introduced competitive displays, in which some of the elements showed a color/luminance based element bias of varying contrast magnitude, while other elements showed a luminance/color based group bias of varying contrast magnitude (competing feature bias). We found that for a simple feature bias the contrast magnitude did not affect the strength of the bias. For competing feature biases, however, the contrast magnitude did influence correspondence, as the bias strength increased with contrast. The implications of our results for current motion and feature-based theories of correspondence are discussed.

An important task of the visual system is to process the visual input to create robust representations of objects by establishing which elements belong together. This correspondence process (e.g., Dawson, 1991; Ullman, 1979) is complicated by the fact that we move our body, head, and eyes such that the position that one and the same object takes on the retina changes constantly. In addition, the objects themselves can change positions over time (and even disappear behind other objects for different periods of time), and the sampling of the visual system is not continuous. How the visual system is nevertheless able to solve this correspondence problem has been the focus of research for a long time. Candidate factors are spatiotemporal factors as well as object features. For example, stimuli in spatiotemporal proximity should be more likely to be perceived as belonging together, as objects usually do not teleport. Moreover, the identity of the objects in terms of their luminance, color, or form should also play a role, in a way that the more similar the objects are, the more likely they should be perceived as the same object.

For a long time, feature information has been thought to be less reliable and therefore secondary (spatiotemporal priority; e.g., Flombaum, Scholl, & Santos, 2009; Kahneman, Treisman, & Gibbs, 1992; Pylyshyn, 1989; Scholl 2007), because features can change easily as the viewpoint or the lightness context changes (e.g., if an object moves into the shadows). In line with this idea, numerous studies using different types of apparent motion displays (Wertheimer, 1912) have shown that spatiotemporal factors (i.e., spatial distance and the time interval between the elements [interstimulus interval; ISI]) play a crucial role in determining correspondence in apparent motion, as good apparent motion is only seen for certain spatiotemporal intervals (e.g., Dawson, 1991; Kolers, 1972; Korte, 1915). Feature information, in contrast, seemed not to play any important role in determining correspondence, as, for example, elements could easily be perceived as moving from one location to the other while simply transforming their color or shape on their apparent motion path (Burt & Sperling, 1981; Cavanagh, Arguin, & von Grünau, 1989; Green, 1986, 1989; Kolers & Pomerantz, 1971; Kolers & von Grünau, 1976; Navon, 1976, 1983; Ramachandran, Ginsburg, & Anstis, 1983; Sekuler & Bennett, 1996; Shechter & Hochstein, 1989; Shechter, Hochstein, & Hillman, 1988; Werkhoven, Sperling, & Chubb, 1994). In accordance with these findings, motion-based theories of correspondence have been developed that emphasize the importance of the (apparent) motion process (e.g., Braddick & Adlard, 1978; Breitmeyer & Ritter, 1986b; Pantle & Petersik, 1980; Pantle & Picciano, 1976; Petersik & Pantle, 1979; Nishida & Takeuchi, 1990; Werkhoven et al., 1994). According to these theories, the visual system first determines the motion energy using motion detectors (e.g., simple Reichardt-type detectors; Reichardt, 1961; Van Santen & Sperling, 1985) that are stimulated by changes in luminance at a certain location over time. Correspondence is then determined dependent on the direction of the predominant motion energy (Adelson & Bergen, 1985; Van Santen & Sperling, 1985; Werkhoven, Sperling, & Chubb, 1993). Feature information, as, for example, form or color, is only processed in a second step, with no or only weak influence on the first step.

In contrast to the idea of spatiotemporal priority, several studies, however, have shown that feature information can have an important influence on the correspondence process. Many of these studies used Ternus displays. The Ternus display (Pikler, 1917; Ternus, 1926/1950) is a type of ambiguous apparent motion display, in which two or three elements are presented next to each other, shifted by one position from one frame to the next (see Fig. 1). This display can be perceived in (at least) two very different ways depending on how correspondence between the elements has been established: Either all elements are perceived as moving together as a group (group motion), or one of the elements is perceived as jumping across the other stationary elements (element motion). Which percept is seen has been shown to depend on the ISI between the frames of elements (Breitmeyer & Ritter, 1986a; Pantle & Petersik, 1980; Petersik, 1989; Petersik & Pantle, 1979): The longer the ISI, the more likely it is to perceive group motion. In addition to these spatiotemporal factors, however, studies have shown large and robust effects of feature information (e.g., Alais & Lorenceau, 2002; Casco, 1990; Dawson, Nevin-Meadows, & Wright, 1994; Hein & Cavanagh, 2012; Hein & Moore, 2012; Kramer & Rudd, 1999; Kramer & Yantis, 1997; Petersik & Rice, 2008; Wallace & Scott-Samuel, 2007). In particular, it has been shown that correspondence is dependent on the luminance contrast (Alais & Lorenceau, 2002; Breitmeyer, May, Williams, 1988; but see Petersik & Pantle, 1979), the feature (dis)similarity of the elements within a frame (e.g., Alais & Lorenceau, 2002; Wallace & Scott-Samuel, 2007), as well as the feature similarity across frames (simple feature bias; see Fig. 1; Casco, 1990; Dawson et al., 1994; Hein & Cavanagh, 2012; Hein & Moore, 2012; Kramer & Yantis, 1997; Petersik & Rice, 2008; Ternus, 1926/1950). For example, the elements of the first frame could be green, red, and green, and the elements of the second frame red, green, and green (element bias; see Fig. 1b, d). In contrast, the elements of both frames could be green, red, and green (group bias; see Fig. 1a, c). To account for the importance of the (feature) identity of the elements, feature-based theories have been developed (e.g., Alais & Lorenceau, 2002; He & Ooi, 1999; Kramer & Yantis, 1997; Pikler, 1917). According to these theories, correspondence depends on the grouping strength between the elements, similar to the original explanation by Ternus (1926/1950). Kramer and colleagues (Kramer & Rudd, 1999; Kramer & Yantis, 1997), for example, proposed that the Ternus percept could depend on the spatial grouping within a Ternus frame as well as the temporal grouping across Ternus frames. According to this idea, the more similar the elements within a frame are, the more likely it is for them to be grouped together across space and the more group motion should be perceived. Moreover, the more similar (and closer in time) the elements are across frames, the more likely it is that the elements are grouped across time and the more element motion should be perceived. For feature-based theories, the visual system first determines how much the elements are likely to belong together or form a group, and only in a second step then attributes motion consistent with the interpretation of the feature information (Hein & Cavanagh, 2012; Hein & Moore, 2014; Petersik & Rice, 2006).¹

Fig. 1

Experimental paradigm for simple feature biases in Experiments 1 and 2. Group and element biases were created by color or luminance contrasts without competition. Since isoluminant colors can only be produced on calibrated screens, these colors are for illustration purposes only and do not match exactly the colors used in the experiment. The arrows indicate feature correspondence between both displays. a Group bias created by isoluminant red-green color. The central element in both displays is more reddish than the background, while the outer elements are more greenish than the background. All elements have the same luminance as the background. This bias should increase the proportion of group responses. b Element bias created by isoluminant red-green color. The central element in the first display and the first element in the second display are more reddish than the background, while the other elements are more greenish than the background. All elements have the same luminance as the background. This bias should increase the proportion of element responses. c Group bias created by luminance contrast. The central element in both displays is darker, while the other elements are lighter than the background. All elements have the same color. d Element bias created by luminance contrast. The central element in the first display and the first element in the second display are darker than the background, while the other elements are lighter than the background. All elements have the same color as the background. (Color figure online)

More recently, Hein and Moore (2012) suggested that the strength of the feature bias in the Ternus display affected the correspondence process. In particular, Hein and Moore (2012, Experiment 3) examined how the number of elements that were compatible with either element or group motion could influence the correspondence process. Interestingly, they found that the more elements were compatible with group or element motion (i.e., the stronger the feature information supporting a certain correspondence solution), the less correspondence was dependent on ISI. This pattern of results is not easily explained by either of the above-described feature-based or motion-based theories. It is inconsistent with pure feature-based theories, because it is unclear how displays that allow for several competing grouping solutions are solved on the basis of the similarity of the element between and across frames alone. It is also inconsistent with motion-based theories, because for motion-based theories the spatiotemporal factor ISI should always have an influence as the feature information should only play a secondary role.

Due to the theoretical impact of the potential effect of the strength of the feature information on the correspondence process, as suggested by the findings of Hein and Moore (2012), we wanted to investigate this effect more systematically in the present study. In particular, we had two aims: (1) We wanted to investigate whether increasing the strength of a feature contributing to a simple feature bias (see Fig. 1; Casco, 1990; Dawson et al., 1994; Hein & Moore, 2012; Kramer & Yantis, 1997; Petersik & Rice, 2008; Ternus, 1926/1950) would affect the correspondence process and, in particular, decrease the importance of the spatiotemporal factor of ISI, as suggested by the findings of Hein and Moore (2012). We increased the strength of the feature bias by increasing the contrast magnitude of the feature providing the bias, using either luminance or isoluminant color as features (Experiment 1). Based on Hein and Moore (2012), we expected that with increasing contrast the feature should become more important, overriding the effect of ISI on the correspondence solution. At its extreme, this result would be inconsistent with motion-based theories due to the importance of the ISI according to those theories. Motion-based theories could, however, predict contrast effects in general, as contrast could directly affect motion energy (Nishida & Takeuchi, 1990) or indirectly by changing the strength of pattern persistence (Breitmeyer & Ritter, 1986a). This result would also be consistent with feature-based theories, as the contrast could affect the (dis)similarity of the elements. (2) To test the effect of feature strength more specifically, we examined whether this effect is limited to the special case in which feature information is pitted against ISI or whether it is more general and can also be found for different types of surface features competing against each other within the same display. To this end we conducted three more experiments, in which we put luminance and color in direct competition against each other and changed the relative strength of these different surface features by modifying their contrast magnitude. In particular, we created a competitive version of the Ternus display, in which an element and a group feature bias was present within the same display in such a way that, for example, color should bias the percept in the direction of element motion, while luminance should bias the percept in the direction of group motion, and vice versa (see Fig. 2). We expected that the two surface features would interact in a similar way with each other as the strength of the feature bias and the ISI seemed to have interacted in the study by Hein and Moore (2012): The stronger the contrast of the feature, the more it should influence the correspondence solution. As detailed above, such an effect would be generally inconsistent with motion-based as well as feature-based theories. To disentangle the general feature bias from its interaction with ISI, we used a logarithmic function with a multiplicative factor for the spatiotemporal effect of the ISI and an additive factor for the general tendency for group responses. The model is explained in more detail in the methods section.

Fig. 2

Experimental paradigm for competing feature biases in Experiments 2–4. Competing group and element biases were created by luminance and color contrasts. The arrows indicate feature correspondence between both displays. a The central element in both displays is more reddish than the background, while the outer elements are more greenish than the background (as in Fig. 1a—color-based group bias). This should generate a group bias based on color. At the same time, the central element in the first display and the first element in the second display are darker than the background, while the other elements are lighter than the background (as in Fig. 1d—luminance-based element bias). This should generate an element bias based on luminance. If the observer’s perception is dominated by color, this display should lead to a group bias (i.e., observers should be more likely to perceive group motion). If the observer’s perception is dominated by luminance, however, then this same display should lead to an element bias (i.e., observers should perceive more element motion). b The central element in the first display and the first element in the second display are more reddish than the background, while the other elements are more greenish than the background (as in Fig. 1b—color-based element bias). At the same time, the central element in both displays is darker, while the other elements are lighter than the background (as in Fig. 1c—luminance-based group bias). If the observer’s perception is dominated by color, this display should therefore lead to an element bias. If the observer’s perception is dominated by luminance however, this display should lead to a group bias. (Color figure online)

Materials and methods

Subjects

In total, 12 subjects participated in these experiments. Subjects were students from Justus-Liebig-University Gießen. Experiments 1, 2, 3, and 4 were completed by six, nine, 11, and seven observers, respectively. Experiments were in accordance with the principles of the Declaration of Helsinki and approved by the local ethics committee LEK FB06 at the University Giessen (Proposal Number 2009-0008).

Equipment

Observers were seated in a dimly lit room facing a 21-inch CRT monitor (ELO Touchsystems, Fremont, CA, USA) addressed by an ASUS V8170 (Geforce 4MX 440) graphics board with a refresh rate of 100 Hz noninterlaced. At a viewing distance of 47 cm, the active screen area subtended 45° in the horizontal direction and 36° vertical on the subject’s retina. With a spatial resolution of 1280 × 1024 pixels, this results in 28 pixels/degree. The observer’s head was fixed in place using a chin rest. The monitor was gamma corrected. Stimulus display was controlled by the PsychToolbox (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997).

Visual stimuli

We used a Ternus display (Pikler, 1917; Ternus, 1926/1950). The size of the three dots was 1.6° of visual angle, and the distance between the dots was 2°. Frame duration was 200 ms. We induced element and group motion biases by L − M isoluminant color (red-green color) and/or L + M (luminance) contrast (see Fig. 1). Stimulus contrasts were determined in the DKL color space (Derrington, Krauskopf, & Lennie, 1984). Contrast polarity (i.e. red vs. green or black vs. white) was randomized across trials. Each experiment started with four practice trials that were not analyzed.

Experiment 1: Simple feature biases for red-green color and luminance

In this experiment, we investigated the strength of a simple feature bias of isoluminant color or luminance at different contrast magnitudes in order to see how the simple bias can be affected by decreasing or increasing the strength of the bias (see Fig. 1). We used a four-factorial design, with interstimulus-interval (ISI; 10, 20, 40, 80, 120, 160, 200, and 300 ms), contrast type (color and luminance), contrast magnitude (25, 50, and 100%) and bias type (group and element). We used these contrast values to cover a wide range of suprathreshold contrasts. Each condition was presented five times, leading to 480 trials (eight ISIs, two contrast types, three contrast magnitudes, two bias types, and five repetitions). In one additional condition, there was no bias at all. Here, each ISI was presented 10 times, leading to 80 trials.

Experiment 2: Competing feature biases for red-green color and luminance

In this experiment, we tested competition between color and luminance biases at different ISIs in order to investigate how the relative strength of one of the feature biases affects the correspondence process. Like in Experiment 1, we tested eight different ISIs (10, 20, 40, 80, 120, 160, 200, and 300 ms). We had four conditions where there was a group or an element bias defined by 10% luminance contrast or by 50% color contrast alone. There was no competition in these four conditions (simple feature biases; see Fig. 1). In six further conditions, we combined a 10% luminance contrast with 25%, 50%, or 100% color contrast. We used these contrast values to cover a wide range of suprathreshold color contrasts. Luminance and color contrast could either define group or element bias and were always competing against each other in these conditions (i.e., a display consisted at the same time of a luminance-based element bias and a color-based group bias or of a color-based element bias and a luminance-based group bias; see Fig. 2). In one additional control condition, there was no bias at all. In one block, each experimental condition was presented five times, leading to 400 trials (eight ISIs, 10 conditions, five repetitions). The control condition without bias was presented 10 times, leading to 80 trials. Each observer performed between one and three blocks.

Experiment 3: Contrast matches for red-green color and luminance

In the next two experiments, we varied color and luminance contrast at a finer scale to determine matching color and luminance contrasts. We used the method of constant stimuli to estimate at which contrasts luminance and color match each other separately for group or element biases. We selected contrast values to cover a range of potentially matching luminance and color contrasts. In two conditions luminance contrast was fixed at 5%, and color contrast was varied in seven steps from 0% to 100%. Luminance could define group or element bias. In two further conditions, color contrast was fixed at 50%, and luminance contrast was varied in seven steps from 0% to 15%. Luminance could define group or element bias. We tested only one ISI of 10 ms. Each condition was presented 10 times, leading to 280 trials (four conditions, seven contrast steps, and 10 repetitions). Contrast matches between luminance and color were obtained by fitting the percentage of group responses for the different contrast magnitudes with a cumulative Gaussian function. We used the Psignifit toolbox in MATLAB to fit the psychometric functions (Wichmann & Hill, 2001). For each condition, contrast ratios were calculated as the ratio of luminance/color contrast. Contrast ratios were averaged across constant luminance and constant color conditions, resulting in one contrast ratio for luminance group bias and one contrast ratio for color group bias.

Experiment 4: Contrast matches for blue-yellow color and luminance

This experiment was identical to Experiment 3, except that isoluminant S−(L+M) (blue-yellow color) contrasts were competing with luminance contrasts. In two conditions, luminance contrast was fixed at 5%, and blue-yellow color contrast was varied in seven steps from 0% to 100%. In two further conditions, blue-yellow color contrast was fixed at 50%, and luminance contrast was varied in seven steps from 0% to 15%. Each condition was presented 10 times, leading to 280 trials (four conditions, seven contrast steps, and 10 repetitions).

Data analysis

We used a logarithmic function to distinguish between the spatiotemporal effects of ISI and the general tendency for group responses in Experiments 1 and 2. The model contains a multiplicative parameter a, specifying the influence of ISI, larger values indicating more influence (see Fig. 3a) and an additive parameter b, specifying the overall tendency toward group responses, larger values indicating more group responses (see Fig. 3b):

$$ y=\mathrm{a}\ \log (ISI)+b. $$

(1)

Fig. 3

Modeling. a Variation of the multiplicative parameter a for the effect of ISI. The multiplicative parameter determines how strongly responses depend on ISI. Larger values lead to a stronger dependency on ISI and more element responses at short ISIs. Dashed and solid lines represent values for additive parameters b of 5 and 0, respectively. b Variation of the additive parameter b for the general tendency towards group responses. The additive parameter determines the proportion of group responses at infinite ISI. Larger values lead to more group responses at infinite ISI. Dashed and solid lines represent values for multiplicative parameters a of 1.5 and 0, respectively. (Color figure online)

The model responses were transformed into proportion of group responses using a logit transformation:

$$ {p}_g=\frac{e^y}{\left(1+{e}^y\right)}. $$

(2)

The model was fitted to the average data, because data for single observers were too variable to yield stable fits.

Results

Experiment 1: Simple feature biases for red-green color and luminance

In the first experiment, we added luminance or color contrast separately to induce either group or element biases (see Fig. 1). Both contrasts were varied in three steps to cover a wide range of suprathreshold contrasts (25%, 50%, and 100%) and to manipulate the strength of the bias. In the no-bias condition, proportion of group responses increased with increasing ISI as in previous studies (e.g., Pantle & Picciano, 1976). When luminance or color contrast induced a group bias, proportion of group responses were close to 100%, independently of ISI (see Fig. 4). When luminance or color contrast induced element bias, proportion of group responses were reduced compared with the no-bias condition and did not reach 100% even at the longest ISI of 300 ms.

Fig. 4

Experiment 1, contrast dependence of separate red-green color and luminance biases. a No-bias condition, where all dots had the same color or luminance contrast. b Simple luminance bias condition. c Simple color bias condition. b–c The three different colors represent different contrasts; light colors represent group bias; dark colors represent element bias. a–c Circles represent no bias; diamonds represent luminance bias; squares represent color bias. Error bars represent the standard error of the mean. Lines represent the model fit from Equations 1 and 2. (Color figure online)

A four-factorial (ISI, contrast type, bias, contrast magnitude) repeated-measures ANOVA showed significant main effects of ISI, F(7, 35) = 5.79, p < .001 , ε² = .537, and of bias, F(1, 5) = 30.22, p = .003, ε² = .858, as well as a significant two-way interaction between ISI and bias, F(7, 35) = 5.53, p < .001, ε² = .525. In addition, there was a significant interaction between ISI and contrast magnitude, F(14, 70) = 2.58, p = 0.005, ε² = .341, as found by Breitmeyer et al. (1988). They suggested that this interaction may be due to contrast influencing ambiguous apparent motion perception in two different ways—a direct effect on motion detectors and an indirect effect through pattern persistence. All other main effects and interactions were not significant (Fs ≤ 4.86, ps ≥ .079).

In addition to the statistical analysis, we fitted a logit-model that allows us to separate the effect of ISI (i.e., how strong the influence of spatiotemporal factors is) from an overall tendency for group responses (see Equations 1–2). This model therefore allows us to quantify more precisely on which level, the ISI effect or the overall tendency for group responses, the two feature biases are effective. Figures 5a–b summarizes the results of the model fits for these two types of effects. They show that both types of simple feature biases, element and group bias, were effective to reduce the effect of ISI compared with the no-bias condition, because the multiplicative model parameters were reduced in all bias conditions compared with the no-bias condition (i.e., they lie below and to the left of the horizontal and vertical lines representing the no bias condition; see Fig. 5a). Moreover, the group bias reduced the effect of ISI more than the element bias for all contrast conditions, because all multiplicative model parameters were lower for the group than for the element bias condition (i.e., they lie below the diagonal). Thus, the model fits suggest that the group bias is more effective in reducing the effect of the ISI than the element bias. Concerning the overall tendency toward group responses, as expected, the element bias was effective in reducing this tendency for all contrast conditions, because all additive model parameters were reduced compared with the no-bias condition (i.e., they lie to the left of the vertical line; see Fig. 5b). However, the group bias did not increase the tendency for group responses compared with the no-bias condition, except for the 100% luminance contrast, as all but this one condition lie below the horizontal line. Nevertheless, the tendency toward group responses was stronger with group than with element bias, because all additive model parameters lie above the diagonal.

Fig. 5

Experiment 1, model results from Equations 1 and 2. a Multiplicative parameter for the effect of ISI. Larger values indicate a stronger effect of ISI and more element responses at low ISIs. b Additive parameter for the general tendency toward group responses. Larger values indicate more group responses at infinite ISI. a–b Black horizontal and vertical lines indicate values for the no-bias condition. Diagonal lines indicate values with identical effects for group and element biases. Red, green, and blue colors represent color or luminance contrasts of 25%, 50%, and 100%, respectively. Diamonds represent luminance bias; squares represent color bias. (Color figure online)

These results replicate previous studies showing that luminance as well as isoluminant color contrasts are able to define object correspondence and to induce group or element biases in the Ternus display (Casco, 1990; Dawson et al., 1994; Hein & Cavanagh, 2012; Hein & Moore, 2012; Kramer & Yantis, 1997; Petersik & Rice, 2008; Ternus, 1926/1950). In addition and in contrast to our hypothesis, the results show that the magnitude of the biases is largely independent of the magnitude of luminance and color contrast, at least for the measured range of suprathreshold contrasts. Furthermore, for both types of features a group bias seems to be mainly effective by eliminating the influence of ISI, whereas an element bias seems to preserve the effect of ISI and reduce the overall tendency toward group responses, which is in line with the findings of other studies using simple feature biases in the Ternus display (e.g., Hein & Moore, 2012). Overall, the results of the first experiment confirm previous results that simple feature biases strongly affect the correspondence solution, but they also show that the strength of the bias does not modify this solution. Although the basis effect of the simple feature bias is in line with feature-based theories (e.g., Alais & Lorenceau, 2002; He & Ooi, 1999; Kramer & Yantis, 1997; Pikler, 1917), as it influences the grouping strength of the elements, it is unclear why the contrast did not affect our results, as one could imagine that it should affect the grouping of the elements as well: The stronger the contrast of the element, the more dissimilar it should appear from the other elements within the frame. In addition, motion-based theories would also predict an effect of contrast on correspondence, either by directly affecting motion energy (Nishida & Takeuchi, 1990) or indirectly via pattern persistence (Breitmeyer & Ritter, 1986a).

Experiment 2: Competing feature biases for red-green color and luminance

In the second experiment, we generated a competition between luminance and color contrasts by inducing antagonistic group or element biases (see Fig. 2) and manipulating the relative strength of the biases by modifying contrast magnitude.

In the no-bias condition, proportion of group responses increased with increasing ISI (see Fig. 6a). In four conditions we tested the influence of 10% luminance and 50% color contrasts in isolation, without competition (see Fig. 1; Fig. 6b–c). Like in Experiment 1, a group bias increased the proportion of group responses. For 10% luminance contrast, group responses were close to 100%, independently of ISI. For 50% color contrast, group responses were slightly lower, and there was a small increase with ISI. When luminance or color contrast induced element bias, proportion of group responses were again reduced compared with the no-bias condition and did not reach 100% even at the longest ISI of 300 ms. Group responses were less frequent with color than with luminance contrast. A three-factorial (ISI, contrast type, bias) repeated-measures ANOVA showed significant main effects of ISI, F(7, 56) = 8.86, p = .003, ε² = .525, contrast type, F(1, 8) = 8.98, p = .017, ε² = .529, and bias. F(1, 8) = 54.07, p < .001, ε² = .871, as well as a significant interaction between ISI and bias, F(7, 56) = 8.78, p < .001, ε² = .523. All other interactions were not significant (Fs ≤ 1.97, ps ≥ .198). The main effect of contrast supports the finding that group responses were less frequent for color contrasts, independently of ISI and bias type. The interaction of ISI and bias supports the finding that group biases reduced the effect of ISI more than element biases do.

Fig. 6

Experiment 2, competing red-green color and luminance biases. a No-bias condition. b 10% simple luminance bias. c 50% simple color bias. Light colors represent group bias; dark colors represent element bias. d–f Competing conditions. Color group bias is plotted in red; luminance group bias is plotted in blue. a–f Error bars represent the standard error of the mean. Lines represent the model fit. (Color figure online)

Like in the previous experiment, we fitted the logit-model to separate the spatiotemporal effect of ISI from the general tendency toward group responses. With simple color or luminance biases (red and blue dots in Fig. 7), the results replicated Experiment 1. Both types of biases reduced the effect of ISI compared with the no-bias condition, because the multiplicative model parameters were again reduced compared with the no-bias conditions indicated by the vertical and horizontal lines (see Fig. 7a). Moreover, the group bias reduced the effect of ISI more than an element bias, because the multiplicative model parameters lie below the diagonal, suggesting that the group bias was again more effective in reducing the effect of ISI than the element bias. Also similar to Experiment 1, both types of biases lead to a reduction of group responses compared with the no-bias condition, because the additive model parameters lie below and to the left of the horizontal and vertical lines (see Fig. 7b). Moreover, the tendency toward group responses was stronger with group than with element bias, because the additive model parameters were higher with the group than with the element bias (i.e., they lie above the diagonal).

Fig. 7

Experiment 2, model results. a Multiplicative parameter for the effect of ISI. Larger values indicate a stronger effect of ISI and more element responses at low ISIs. b Additive parameter for the general tendency toward group responses. Larger values indicate more group responses at infinite ISI. a–b Black horizontal and vertical lines indicate values for the no-bias condition. Diagonal lines indicate values with identical effects for group and element biases. Simple color bias is represented by red dots. Simple luminance bias is represented by blue dots. Light, medium, and dark gray dots represent competition between color and luminance with 25%, 50%, and 100% color contrast, respectively. (Color figure online)

In general, the 10% luminance contrast (blue dot in Fig. 7) seems to be more effective in biasing the response than the 50% color contrast (red dot in Fig. 7), because the effect of ISI was eliminated with luminance contrast but not with color contrast, and because the overall tendency for group responses was stronger for luminance than for color group bias (while at the same time there was no difference between luminance and color element bias).

In six further conditions, we tested competing biases between 10% luminance contrast and 25%, 50%, and 100% color contrast (see Fig. 2; Fig. 6d–f). When luminance induced a group bias, group responses reached a maximum of about 80% and were independent of ISI with 25% and 50% color contrast. Group responses increased with ISI for 100% color contrast. When color induced a group bias, group responses depended on ISI in all three contrast conditions, but this effect was weaker with 100% color contrast. A three-factorial (ISI, bias, contrast magnitude) repeated-measures ANOVA showed a significant main effect of ISI, F(7, 56) = 11.58, p = .003, ε² = .590, as well as significant interactions between ISI and bias, F(7, 56) = 21.98, p < .001, ε² = .733, between bias and contrast magnitude, F(2, 16) = 18.21, p < .001, ε² = .695, and a significant three-way interaction, F(14, 112) = 5.27, p < .001, ε² = .397. All other main effects and interactions were not significant (Fs ≤ 1.10, ps ≥ .357).

The logit-model showed that the effect of ISI was completely eliminated when the 10% luminance group bias was competing against 25% and 50% color element bias, because the multiplicative model parameters were close to zero (dark and medium gray dots in Fig. 7a). When it was competing against a 100% color element bias, ISI was still effective (light gray dot in Fig. 7a). This again suggests that the 50% (and of course also the 25%) color contrast was weaker than the 10% luminance contrast. Ten percent luminance and 100% color contrasts were more equally matched. The overall tendency toward group responses was reduced in all conditions, interestingly more so when color contrast induced an element bias (and luminance contrast induced a group bias) than when luminance contrast induced an element bias (and color contrast induced a group bias), because the additive model parameters lie below the diagonal (gray dots in Fig. 7b). Since the parameter b in our model is mainly determined by the asymptotic proportion of group responses at long ISIs, this would mean that the balance between color and luminance is shifted in favor of color at long ISIs.

While contrast magnitude did not matter in Experiment 1, these results suggest that the relative strength of the feature bias—as manipulated by changing the magnitude of luminance and color contrast—can matter for object correspondence when there is a conflict between color and luminance signals. In particular, the stronger the bias the more it affects the correspondence solution. The strongest influence of contrast was visible at low ISIs. These results cannot be easily explained within either theoretical framework, feature-based or motion-based theories, as outlined in the General Discussion.

Experiment 3: Contrast matches for red-green color and luminance

Here we measured at which contrasts competing color and luminance biases are equally effective in inducing group or element biases. We tested this only for an ISI of 10 ms because we found the strongest contrast effects in Experiment 2 for short ISIs.

In separate conditions, color or luminance contrasts were held constant, while the other contrast was varied to measure psychometric functions. Figure 8a–b shows the resulting psychometric functions for one representative observer. If color and luminance contrasts would be equally effective in biasing group and element responses, then the thresholds, defined as the contrast at 50% group responses, should be identical. Clearly, this is not the case for this observer. When luminance contrast was fixed at 5% (see Fig. 8a), a color group bias and a luminance element bias were matched at a color contrast of 50%. However, a luminance group bias and a color element bias were matched at a color contrast of 70%. This relationship was reversed when color contrast was fixed at 50% (see Fig. 8b). A luminance group bias and a color element bias were matched at a luminance contrast of 5.3%. However, a color group bias and a luminance element bias were matched at a luminance contrast of 6.5%.

Fig. 8

Experiment 3, matching of red-green color and luminance contrast. a–b Representative psychometric functions for one observer. Color group bias is plotted in red; luminance group bias is plotted in blue. Horizontal line represents the baseline response for a stimulus without any bias. Vertical lines represent the estimated contrast thresholds at 50% group responses, where luminance and color biases are equally effective. If color and luminance were equally effective for group and element biases, then the red and blue vertical lines should lie on top of each other. a Constant 5% luminance contrast. b Constant 50% color contrast. c Contrast ratio at 50% group responses. Open symbols represent individual subjects; filled circle the mean across subjects; error bars denote 95% confidence intervals. (Color figure online)

To compare group and element biases more easily, we calculated the ratio of luminance/color contrast separately for the two bias combinations (see Fig. 8c). This ratio was significantly smaller, t(10) = 3.67, p = .004, for luminance group bias (7% ± 2%) than for color group bias (12% ± 4%). This means that luminance and/or color contrasts were more effective in inducing group than element biases.

These results replicate the findings of the previous experiment that the magnitude of luminance and color contrasts matters for object correspondence when there is a conflict between luminance and color information. In addition, they show that group and element biases are differently affected by luminance and/or color contrast at 10 ms ISI.

Experiment 4: Contrast matches for blue-yellow color and luminance

In this final experiment, we used S-cone isolating stimuli and measured at which contrasts competing blue-yellow color and luminance biases are equally effective. Experimental procedures and analysis methods were identical to Experiment 3.

Figure 9a–b shows the resulting psychometric functions for one representative observer. Like in the previous experiment, the thresholds were not identical across conditions for this observer. When luminance contrast was fixed at 5% (see Fig. 9a), a color group bias and a luminance element bias were matched at a color contrast of 40%. However, a luminance group bias and a color element bias were matched at a color contrast of 86%. This relationship was reversed when color contrast was fixed at 50% (see Fig. 9b). When we analyzed contrast matches for all observers (Fig. 9c), the ratio of luminance/blue-yellow color contrast was significantly smaller, t(6) = 3.00, p = .0239, for luminance group bias (6% ± 2%) than for blue-yellow color group bias (11% ± 4%).

Fig. 9

Experiment 4, matching of blue-yellow color and luminance contrast. Conventions are the same as in Fig. 8. a–b Representative psychometric functions for one observer. c Contrast ratio at 50% group responses. (Color figure online)

The results of Experiment 4 show the same pattern as Experiment 3. The magnitude of luminance and color contrast matters for object correspondence when there is a conflict between luminance and color. Again, luminance and/or blue-yellow color contrast were more effective in driving a group bias than an element bias.

General discussion

Feature information, like luminance, color, or form, has been shown to influence how the visual system determines the identity of objects over space and time and thus solves the correspondence problem (e.g., Dawson, 1991; Ullman, 1979). Furthermore, it has been suggested that the strength of the feature information might also influence correspondence (Hein & Moore, 2012). The goal of the present study was to further investigate this effect by examining how spatiotemporal and feature information, as well as different types of features, interact with each other while changing their contrast magnitude in order to increase or decrease the strength of the corresponding feature.

We used the Ternus display (Pikler, 1917; Ternus, 1926/1950), an ambiguous apparent motion display, for which correspondence can be solved in such a way that either element or group motion is perceived, and that is known to show strong effects of feature information (e.g., Casco, 1990; Dawson et al., 1994; Hein & Moore, 2012; Kramer & Yantis, 1997; Petersik & Rice, 2008). We first showed that simple group and element biases, that is, Ternus elements from one frame to the other being either compatible with group or with element motion, using either isoluminant color or luminance, could strongly bias the Ternus percept toward group or element motion, largely independent of the contrast magnitude. In particular, the group bias reduced the effect of ISI much more than the element bias for all contrast magnitudes and both types of simple feature bias (Experiment 1). Moreover, we found that in a competitive version of the Ternus display, in which an element and a group motion bias was introduced at the same time, one using isoluminant color and the other using luminance, the contrast magnitude did matter for the correspondence solution. In particular, the higher the contrast of a feature, the more likely it was that correspondence was solved in the direction of the bias of this feature (Experiment 2). In addition, we found that luminance and/or color contrasts were more effective in driving the group bias than in driving an element bias, no matter whether the color we used was red/green or blue/yellow (Experiments 3 and 4).

As reviewed in the introduction different theories have been proposed in order to explain the influence of feature information on correspondence in the Ternus display (see Petersik & Rice, 2006, for an overview). These theories can be categorized broadly as feature-based or motion-based theories, depending on the focus of the correspondence explanation. Some feature-based theories have suggested that correspondence could depend on the grouping strength between the elements within a frame and across frames (He & Ooi, 1999; Kramer & Yantis, 1997; see also Alais & Lorenceau, 2002). These theories were developed to account for feature bias effects like those showed in Experiment 1 of the present study. The (dis)similarity of the elements affects the grouping strength between elements. According to these theories, however, one might expect that the contrast magnitude should have affected the motion percept, because the higher the contrast, the stronger the perceived dissimilarity between the elements within a frame should be. Our results, however, showed that the contrast magnitude did not matter for simple element and group biases. The contrast effects of the competing feature biases in Experiments 2 to 4, on the other hand, could be explained within the framework of these theories, but only if one additionally assumes that the higher the contrast of one of the feature biases, the more this feature might affect the corresponding grouping of the elements.

For motion-based theories, on the other hand, the visual system first determines the motion energy and feature information is secondary. Feature information can nevertheless still have an effect, as, for example, size or luminance contrast can influence motion energy (e.g., Nishida & Takeuchi, 1990). Moreover, feature information could also be effective by influencing pattern persistence (Breitmeyer & Ritter, 1986b). According to this theory, the higher the contrast, the more group motion should be perceived, as persistence decreases with increasing contrast. This would be compatible with our results for group biases, but not with element biases. Other motion-based theories have suggested that the two different motion percepts in the Ternus display were processed by two different motion systems (Pantle & Picciano, 1976; Petersik, 1989; Petersik & Pantle, 1979; but see Dodd, McAuley, & Pratt, 2005; Scott-Samuel & Hess, 2001). In particular, as dichoptic viewing of the two Ternus frames eliminated element motion, Pantle and Picciano (1976) suggested that the element motion percept depended on an early motion system (or short-range/sustained), while group motion percept depended on a higher-level motion system (or long-range/transient).²

In line with this two-process distinction, some researchers have related group motion to the magnocellular (transient) pathway and element motion to the parvocellular (sustained) pathway based on research with people with dyslexia who seem to perceive less element motion than other people (Cestnick & Coltheart, 1999; Skottun, 2001, for a review). Although debated in their details, it has been suggested that the parvocellular and magnocellular pathways are specialized in the processing of different types of visual information (e.g., Livingstone & Hubel, 1988). In particular, based on the differences between neurons in the two pathways, the parvocellular pathway is thought to be more sensitive to color information and spatial detail, while the magnocellular pathway is more sensitive to luminance contrast information (e.g., Merigan & Maunsell, 1993; Schiller & Logothetis, 1990). Along this distinction, two different motion channels have been put forward, one channel that is sensitive to color contrast and specialized in slow speeds, and another mechanism that is sensitive to luminance and specialized in fast speeds (Gegenfurtner & Hawken, 1996; Ohtani, Ejima, & Nishida, 1991; van der Smagt, Verstraten, & van de Grind, 1999). These theories are compatible with our finding that the balance between luminance and color signals is biased more to luminance at small ISIs (fast speeds) and more to color at long ISIs (slow speeds). They could also explain our asymmetries in Experiments 3 and 4 and would suggest that they are most likely caused by luminance signals driving a group bias more strongly.

Taken together, we can conclude in favor of both types of traditional theories explaining correspondence in the Ternus display. Both feature-based and motion-based theories explain some aspects of our findings, suggesting that they both play a role in the correspondence process. Neither can explain, however, why the simple feature biases, in particular the element biases, were not dependent on the contrast magnitude of the feature. Furthermore, neither offers a direct explanation for why contrast magnitude is effective in competing displays in the way it is. We therefore propose another type of feature-based theory, the object-based account of correspondence in the Ternus display (see also, Hein & Cavanagh, 2012; Hein & Moore, 2014).

According to this account, correspondence is solved on the basis of perceived identity or similarity of the individual elements in a one to one mapping (i.e., the best correspondence solution of the possible element pairs across frames in the Ternus display). As for traditional feature-based theories, object identity is determined in a first step, and motion is assigned only in a second step. In contrast to traditional feature-based theories, however, it is not the grouping strength or similarity of the group of elements within a frame and across frames, but rather the perceived similarity of the individual elements across frames that is important. Attentional pointers (Cavanagh, 1992) could connect elements that appear to have the same identity across space and time, which can easily explain simple feature biases at any contrast magnitude. In particular, as far as the mapping between elements can be easily established, as for simple feature biases, contrast does not matter. If the mapping is not clear, however, as it is the case for competing feature biases, the strength of the feature information (i.e., the contrast magnitude) could become relevant, and the information from the different feature types could be weighted according to their strength and then integrated (maybe following a maximum likelihood estimation model; Ernst & Bülthoff, 2004). Another possibility to explain competing biases within this framework would be that the higher contrast elements might attract attention and thus be prioritized by the attentional pointers, emphasizing the corresponding bias in the competitive Ternus display. The object-based account thus offers a possible explanation for why correspondence in competing displays is solved in the way we found it in the present study, connecting the objects with the most reliable and/or salient features. Moreover, according to the object-based account of correspondence, different correspondence solutions will be found depending on the best possible correspondence connections of each of the possible connection pairs. Importantly, this correspondence solution would include all individual element pairs across frames and not only the central ones as a lot of motion-based theories suggest, independently of the grouping of the elements within each frame as feature-based theories would suggest.

To summarize, we found that the correspondence solution in the Ternus display strongly depends on feature biases, no matter whether they are luminance or color based. Moreover, when put in competition, the higher the contrast of one of the feature biases, the more likely this feature bias is used to solve correspondence. The findings can be explained very well within an object-based account of correspondence (see also, Hein & Cavanagh, 2012; Hein & Moore, 2014), which is based on the perceived identity or similarity between individual elements (across frames) and tries to find an optimal correspondence solution for all element pairs.