Moving a visual object is known to lead to an update of its cognitive representation. Given that object representations have also been shown to include codes describing the actions they were accompanied by, we investigated whether these action codes “move” along with their object. We replicated earlier findings that repeating stimulus and action features enhances performance if other features are repeated, but attenuates performance if they alternate. However, moving the objects in which the stimuli appeared in between two stimulus presentations had a strong impact on the feature bindings that involved location. Taken together, our findings provide evidence that changing the location of an object leaves two memory traces, one referring to its original location (an episodic record) and another referring to the new location (a working-memory trace).
Introduction
Due to the modular, distributed organization of the primate brain, human perception relies on the integration of features coded in various cortical areas of the brain (cf. Treisman, 1996). Consider, for example, the neural correlate of perceiving a red cup placed on a green saucer. The two objects activate several brain regions, including those associated with processing locations and colours—red and green, top and bottom—creating a confusing situation where the features are easily mixed up into green cups and red saucers. To solve problems of that sort, integration processes have been postulated that bind features of the same object into episodic traces or object files (Kahneman, Treisman & Gibbs, 1992).
Evidence for object files has been provided by studies looking into the after-effects of feature binding Kahneman et al. (1992), for example, showed that a visual target letter can be identified faster if it appears as part of the same object in a task-irrelevant preview display. That is, if the preview display consisted of a number of letters appearing inside of boxes, repeating one of those letters yielded particularly good performance if it also appeared in the same box. This was the case even if all the boxes moved between the presentation of the preview letters and the eventual target, suggesting that the letters remained represented as part of the boxes and thus, in a sense, moved with them. Kahneman et al. suggested that letters and boxes were bound into common object files, which were updated when the boxes moved and retrieved as a unit when a letter reappeared.
The assumption that moving an object leads to the updating of its cognitive representation is consistent with the outcome of multiple-object tracking (MOT) studies. Pylyshyn and Storm (1988) showed that even if objects move rapidly and randomly, their constituent features, such as their identities as being either targets or distractors, remain bound to them. This triggered a debate as to whether attention is primarily object- (Yantis, 1992) or space-based (Pylyshyn, 1989). As it appeared that these two positions are not mutually exclusive—since space may not be the only pointer towards different objects that are tracked in parallel, but is most certainly particularly important for object based attention (Blaser, Pylyshyn & Holcombe, 2000)—later studies refocused research interests onto what exactly constitutes an object and how objects are selected and kept within attention or working memory (Mitroff & Alvarez, 2008; Pylyshyn & Annan, 2006; Scholl, Pylyshyn & Feldman, 2001).
At present, it is not clear how—or even whether—the ability to track multiple objects across time and space relies on the maintenance of object-files Pylyshyn and Storm (1988) argued that MOT is enabled by means of an early system that attaches indices to visual features in a display. Analogous to “sticky fingers”, these indices (“fingers of instantiation”, or FINSTs) remain bound to the objects in a MOT task, limited by their number (around four or five, according to Pylyshyn and Storm) and visual task demands such as the velocity of the objects. Kahneman et al. in turn suggested that these indices might be closely related to object files, hypothesising that they may even be the initial phase of object files. Further research, however, brought evidence that although object files are related to (cf. Oksama & Hyönä, 2004; Carey & Xu, 2001), they can be experimentally differentiated from (Horowitz et al., 2007), FINSTs.
Object files have been claimed to contain perceptual information about an object but may also include memory-derived knowledge about the object’s identity and meaning (Kahneman et al., 1992; Horowitz, Klieger, Fencsik, Yang, Alvarez & Wolfe, 2007). Indeed, increasing evidence suggests that object representations comprise pragmatic information about action affordances (Barsalou, 1999; Gibson, 1979; Hommel, Müsseler, Aschersleben & Prinz, 2001). Along these lines, Hommel (1998) provided evidence that action features are integrated and kept bound within object representations, resulting in what may be more appropriately labelled “event files”. To demonstrate the existence of stimulus-response bindings, he cued participants to respond with a left or right button-press (R1) to the mere onset of a visual stimulus (S1) presented above or below a central fixation. Shortly after that, another stimulus (S2) was presented to signal a binary choice response (R2) to its shape or colour. When one perceptual feature (such as the shape) was repeated between the two displays (S1 and S2), but another (such as the location) was not, participants responded slower than when both perceptual features were either repeated or alternated—thus replicating the observation of Kahneman et al. (1992). However, the same pattern emerged across perception and action: when a shape was first reacted to with one button-press, performance benefits only ensued if participants responded to the same shape in the same way or to a different shape in a different way. In other words, repeating a stimulus feature and alternating the response, or vice versa, created partial-repetition costs. Apparently, experiencing the co-occurrence of a stimulus and a response created an event file that was retrieved upon S2/R2 processing if at least one ingredient was repeated—thus inducing conflict between stimulus or response features if other ingredients did not match with the present features.
In the present study, we asked whether object files as investigated in object-tracking studies are comparable to event files as investigated along the lines of Hommel (1998; for an overview see Hommel, 2004). The Theory of Event Coding (TEC; Hommel et al., 2001) suggests that they are. Even though the resulting representation may well be complex, highly structured, and multilayered, this account claims that perceptual and action-related information is integrated into a network that acts like a functional unit. Hence, if perceptual features travel with the object they are a part of, actions should do so as well. We tested this prediction by combining the original previewing design (S1 → S2/R2) introduced by Kahneman et al. (1992) with Hommel’s (1998) S1/R1 → S2/R2 extension.
Experiment 1
In Experiment 1, participants were pre-cued to carry out a particular key press (R1) in response to the onset of a visual stimulus (S1), assuming that this would create a binding between the corresponding stimulus features in the response (see Fig. 1). Then, the second target stimulus (S2) appeared to signal a binary choice response (R2) to its shape. The location of the two stimuli varied randomly and could thus repeat or alternate. The crucial manipulation was that each target stimulus appeared in one of two boxes, which did or did not rotate by 180° in between S1 and S2 presentation. If stimulus features and/or responses would travel with their object, rotation should have a distinct effect: If S2 appears in the same physical location as S1, this should amount to a repetition of stimulus location with a static display but to an alternation with a rotating display. This might affect two types of interactions: first, the interaction between the repetitions versus alternations of the two varying stimulus features, shape and location. According to the object-file literature and object-tracking studies, the shape of S1 should be integrated with the box in which it appears and thus move with it. If so, rotation of the boxes should render alternations of the physical locations of the stimuli (S1 top → S2 bottom, or vice versa) location repetitions, so that performance should be better if shape repetitions come with changes of physical location and shape alternations with repetitions of physical location. The crucial question was whether a comparable effect would be obtained for interactions between location repetition and response repetition. The hypothesis that response information travels with the moving box would predict that rotation should result in better performance if response repetitions are combined with changes of physical location and response alternations with repetitions of physical location. In other words, we predicted that partial-repetition costs for stimulus-location and stimulus-shape combinations and for stimulus-location and response combinations would reverse in sign in the box-rotation condition.
Fig. 1
Sequence of events in two trials of Experiment 1
×
Method
Eight male and five female students from Leiden University voluntarily participated. Stimuli were presented on a 14.1” TFT monitor in 800 × 600 pixel resolution and a refresh-rate of 60 Hz. A Dell dual-core 1.66 GHz laptop PC running E-Prime 1.2 on Windows XP SP2 was used to control stimulus-presentation and record reactions. Cues, targets and boxes were presented in black against a silver (RGB 192, 192, 192) background. Cues consisted of three greater-than or lesser-than signs, and were centrally presented. Targets were presented in one of two black-lined, grey-filled (RGB 128, 128, 128) boxes of 60 × 60 pixels, presented 60 pixels above or below the centre of the screen. Rotation consisted of 45 frames, with each of these rotating 4° and lasting for approximately 27 ms. Targets were either black (RGB 0, 0, 0) circles or four-pointed stars.
As outlined in Fig. 1, a response-cue (<<< or >>>) was presented for 1,000 ms, during which the participant was asked to prepare the cued response and to press the corresponding key (‘Q’ for <<<, ‘P’ for >>>) upon the onset of the next screen. This next screen (S1) showed two vertically placed boxes, one of them containing a circle or star. Participants were asked and trained to ignore the shape and to merely respond according to the previously shown cue. Following this, the shape inside one of the boxes disappeared and the boxes either rotated (in the rotation condition), or remained still (the static condition) for another 1,200 ms. Then, during S2, a target was presented for 700 ms in one of the boxes, and now participants were required to respond (R2) within this time interval to the shape with either a left (‘Q’) or a right (‘P’) key-press (Q, for circles, P for stars, for example), with the stimulus-response mapping being counter-balanced across participants. This was followed by an inter-trial interval of 1,100 ms with feedback in terms of a score that reflected both accuracy (1 point was given for each correct reaction) and speed (2 points were given for each accurate and fast reaction). This system of feedback was explained during training, which consisted of the first 20 trials of the experiment. The experiment took approximately half an hour.
The experiment used a four-factor repeated measures design with the factors stimulus-shape, stimulus-location, and response repetition versus alternations, and rotation (static versus rotated boxes). Each of the 16 combinations of these factors was presented 24 times, and the direction of the rotation (clock- or counter-clockwise) was balanced across design cells.
Results
S2 reaction times were analysed only if both reactions were correct and fast (<700 ms). Overall, few errors were made for S1 (M = 3.8%, SD = 4.2%) compared to S2 (M = 15.7%, SD = 9.6%). In a repeated measures four-way ANOVA, reaction times were found to be faster in rotation than in static conditions, F(1, 12) = 13.49, MSe = 7300.36, p < 0.005, if the response alternated than repeated, F(1, 12) = 8.55, MSe = 6246.15, p < 0.02; and if location alternated, F(1, 12) = 6.18, MSe = 2276.20, p < 0.03. Rotation significantly interacted with location repetition, F(1, 12) = 6.51, MSe = 3790.65, p < 0.03, such that the alternation bias during static trials (15 ms) disappeared during rotation trials (−2 ms). The opposite pattern was observed with response-repetition, which yielded a significant interaction between rotation and response-repetition, F(1, 12) = 23.00, MSe = 2433.78, p < 0.001, the response-alternation benefit being smaller in static trials (4 ms) than in rotation trials (18 ms).
Replicating the pattern reported by Hommel (1998), partial-repetition costs were found (see Fig. 2): between location repetition and response repetition, F(1, 12) = 10.22, MSe = 5452.28, p < 0.01; and between shape repetition and response repetition, F(1, 12) = 45.84, MSe = 19743.78, p < 0.001; whereas the interaction between shape and location repetition only approached significance, F(1, 12) = 3.53, MSe = 892.75, p < 0.09. Finally, the three-way interaction between all three repetition effects was significant, F(1, 12) = 5.77, MSe = 1482.21, p < 0.04.
Fig. 2
Partial-repetition costs in Experiment 1 of location-by-shape, location-by-response and shape-by-response as a function of rotation. The upper part of the figure shows two conditions in which alternation of one of two features between S1 and S2 results in partial-repetition costs
×
More important for the present study, the two-two-way interactions that involved stimulus-location repetition were modulated by rotation: location-by-shape, F(1, 12) = 10.73, MSe = 2821.06, p < 0.01, and location-by-response, F(1, 12) = 25.95, MSe = 4459.38, p < 0.001. In contrast, neither the shape-by-response interaction nor the three-way interaction was further modified by rotation, F(1, 12) = 0.49, MSe = 214.47, p > 0.4, and F(1, 12) = 0.32, MSe = 117.79, p > 0.5, respectively. Table 1 shows the emerging pattern: while quite substantial partial-repetition costs1 were obtained for all three combinations of stimulus features and response, rotating the boxes eliminated the costs for the two combinations involving location repetitions and alternations.
Table 1
Experiment 1: mean reaction times and error percentages (in parentheses) as a function of rotation and repetitions versus alternations of shape, stimulus location and response
Shape
Location repeated
Location alternated
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Static
426 (12)
428 (6.4)
422 (7.4)
401 (9.2)
23 (−7.4)
Rotating
411 (9.8)
403 (10.7)
410 (8.9)
408 (9.2)
−6 (0.6)
Response
Location repeated
Location alternated
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Static
419 (5.8)
435 (12.6)
424 (12.4)
400 (4.2)
39 (15.0)
Rotating
415 (13.8)
398 (6.7)
418 (11.2)
399 (0)
2 (−2.7)
Shape
Response repeated
Response alternated
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Static
416 (4.7)
428 (13.6)
433 (14.8)
402 (2.0)
43 (21.8)
Rotating
410 (9.1)
423 (15.9)
410 (9.5)
387 (4.0)
35 (12.3)
For each combination of two features, the partial-repetition costs are shown. These were calculated as the interaction term between two features and show the cost in reaction time resulting from changing either the one feature or the other, as opposed to changing both or neither one of the two features (see footnote 1)
The analysis of errors was based on proportions and reflected only data from the trials where S1 was correct and sufficiently fast (<700 ms). In general, the error patterns followed those of the reaction times. The only reliable main effect indicated that repeating a response yielded more errors than alternating it, F(1, 12) = 5.60, MSe = 0.05, p < 0.04. Rotation significantly interacted with response repetition, F(1, 12) = 5.33, MSe = 0.03, p < 0.04, and with shape repetition, F(1, 12) = 5.96, MSe = 0.01, p < 0.04. Significant interactions were obtained for shape and response repetition, F(1, 12) = 55.84, MSe = 0.38, p < 0.001, location and response repetition, F(1, 12) = 8.25, MSe = 0.05, p < 0.02, and shape and location repetition, F(1, 12) = 4.85, MSe = 0.02, p < 0.05. All three interactions were further modified by rotation: F(1, 12) = 5.97, MSe = 0.01, p < 0.04, F(1, 12) = 10.76, MSe = 0.10, p < 0.01, and F(1, 12) = 5.08, MSe = 0.02, p < 0.05, respectively.
Discussion
The outcome of Experiment 1 can be considered mixed. On the one hand, it is clear that rotation had a strong effect on the expected direction. Whereas standard partial-repetition costs were obtained for location and response repetitions as well as for shape and location repetitions, rotating the empty boxes in between S1 and S2 presentation eliminated these costs. Also as expected, rotation only affected partial-repetition costs related to location repetitions but not with respect to the interaction of shape and response. On the other hand, however, the location-related partial-repetition costs were only eliminated, but they did not reverse in sign—as we would have expected if rotation led to an update of the respective object or event files. There are at least two interpretations of this observation.
First, it is possible that moving the boxes induced the creation of a new event file without overwriting the previous one. If, say, a circle appeared in the bottom box before the boxes were rotated, this could have left two shape-location bindings: one linking circle with bottom and another linking circle with top, the new location. If the circle would appear again then, it would retrieve two bindings with contradicting spatial information that may cancel out one another. The same logic can be applied to location-response bindings. Whereas this scenario would be consistent with our main hypothesis, there is a second, theoretically less interesting possibility, however. For various reasons, moving the empty boxes may flush any sort of visual working memory and thus delete any available binding. True, this possibility is ad hoc and does not seem to fit with the results from previewing studies using moving stimuli (Kahneman et al., 1992) and MOT studies (Pylyshyn & Storm, 1988). However, it would be consistent with assumptions from leading theories on the limitations of working memory capacity and executive control (Gilbert & Shallice, 2002; Logan & Gordon, 2001) and with studies on event segregation (Zacks, Speer, Swallow, Braver & Reynolds, 2007). Accordingly, we considered it important to replicate our findings and to seek for independent evidence supporting the multiple-binding interpretation.
Experiment 2
We attributed the disappearance of partial-repetition costs in the rotation condition of Experiment 1 to the existence of two types of event files: one linking shape and response information to the physical location of S1 and another linking this information to the updated location, that is, to the post-rotation location of the box in which S1 had appeared. The idea underlying Experiment 2 was to try making the transition between the two represented states—S1 appearing in the box and the empty box rotating—visually smoother by softly fading out S1 rather than letting it abruptly disappear. Zacks et al. (2007) have claimed, and provided evidence, that unpredicted visual changes are more likely to lead to the closing of the current event representation and the opening of a new one, whereas predicted changes merely induce an update of the currently open representation. Smoothing the transition between S1 and S2 may thus help linking these two events to one another or, more precisely, the event files representing them. If so, chances are that only one updated file would be maintained at least in some trials or that the updated file would dominate the previous one more strongly. This should drive the result pattern in the rotation condition more in the expected direction, that is, partial-repetition costs for location-related interactions should no longer be zero but go negative. We thus replicated Experiment 1 but added a further condition in which S1 gradually faded out.
Method
Six male and ten female students from Leiden University voluntarily participated. The method was as in Experiment 1, except that in fading conditions, the opacity of the stimulus shown in S1 decreased with each of the 45 frames by approximately 2.2% during the inter-stimulus interval. Thus, it appeared to gradually fade out, while its position remained anchored to the box of its prior appearance.
Results
Overall, few errors were made for S1 (M = 4.2%, SD = 3.5%) compared to S2 (M = 17.8%, SD = 7.4%). In a repeated measures five-way ANOVA with fading, rotation, shape-, location- and response-repetition as factors, responses were found to be slightly (8 ms) slower in fading conditions than in abrupt conditions, F(1, 15) = 11.05, MSe = 8959.92, p < 0.005, in static conditions than in rotating conditions, F(1, 15) = 24.26, MSe = 43348.93, p < 0.001, and if the response repeated than alternated, F(1, 15) = 6.68, MSe = 2918.48, p < 0.02. Neither repeating the location, p > 0.1 nor repeating the shape, p > 0.7, affected reaction time.
The partial-repetition costs of Experiment 1 were replicated (see Fig. 2): reliable interactions were obtained between location repetition and response repetition, F(1, 15) = 17.35, MSe = 12475.48, p < 0.001; and between shape repetition and response repetition, F(1, 15) = 68.66, MSe = 72317.37, p < 0.0001; but not between shape and location repetition, p > 0.8. The three-way interaction involving all three repetition effects was far from significant, F(1, 15) = 0.11, MSe = 68.50, p > 0.7.
Replicating the main findings from Experiment 1, partial-repetition costs involving location repetitions were modulated by rotation: location-by-shape, F(1, 15) = 8.04, MSe = 4192.07, p < 0.02, and location-by-response, F(1, 15) = 25.52, MSe = 17819.65, p < 0.001. Again, neither the shape-by-response interaction, F(1, 15) = 0.24, MSe = 99.77, p > 0.6, nor the three-way interaction, F(1, 15) = 0.03, MSe = 16.06, p > 0.8, was affected by rotation.
The fading condition affected this pattern in two significant ways. First, it modulated the interaction between response repetition and rotation that was observed in Experiment 1, F(1, 15) = 5.48, MSe = 1354.86, p < 0.04: in the fading condition, the response alternation bias no longer changed after rotation. Second, it modulated the interaction between location and shape, F(1, 15) = 9.07, MSe = 2321.87, p < 0.01: whereas partial-overlap costs were positive with abruptly disappearing stimuli, these costs were negative with fading stimuli (see Tables 2 and 3).
Table 2
Experiment 2: mean reaction times and error percentages (in parentheses) as a function of stimulus fading, rotation, repetitions versus alternations of shape, stimulus location and response
Shape
Location repeated
Location alternated
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Abrupt
Static
430 (12.2)
444 (10.2)
433 (8.8)
425 (9.2)
23 (−2.4)
Rotating
414 (12.5)
416 (10.0)
412 (8.8)
420 (11.5)
−7 (−5.2)
Fading
Static
444 (13.0)
442 (10.0)
442 (10.0)
441 (12.0)
−1 (−5.0)
Rotating
431 (13.9)
419 (13.9)
418 (11.4)
424 (13.2)
−17 (−1.7)
Response
Location repeated
Location alternated
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Abrupt
Static
426 (7.7)
449 (14.8)
438 (11.4)
420 (6.6)
41 (12.0)
Rotating
416 (12.4)
414 (10.1)
421 (10.3)
424 (13.2)
8 (−2.0)
Fading
Static
437 (8.4)
449 (14.7)
458 (15.1)
425 (7.0)
45 (14.4)
Rotating
432 (16.3)
419 (11.6)
420 (14.5)
422 (10.1)
−16 (−0.1)
Shape
Response repeated
Response alternated
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Abrupt
Static
419 (5.5)
445 (13.6)
444 (15.5)
424 (5.8)
46 (17.7)
Rotating
407 (7.0)
430 (15.7)
419 (14.2)
406 (5.9)
35 (17)
Fading
Static
435 (4.5)
460 (18.9)
451 (18.5)
424 (3.1)
52 (29.8)
Rotating
413 (8.0)
438 (22.8)
436 (17.4)
405 (4.3)
57 (27.9)
Table 3
Experiment 3: mean reaction times and errors (in parentheses) as a function of stimulus fading, rotated location and repetitions versus alternations of shape and response
Shape
Rotated towards
Rotated away
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Abrupt
391 (12.5)
398 (7.4)
399 (13.5)
390 (8.3)
15 (0.1)
Fading
393 (9.9)
399 (10.0)
408 (13.7)
390 (9.0)
23 (4.8)
Response
Rotated towards
Rotated away
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Abrupt
387 (9)
401 (10.9)
398 (9.0)
392 (12.8)
20 (−1.8)
Fading
390 (10)
401 (10.0)
401 (10.2)
397 (12.4)
16 (−2.2)
Shape
Response repeated
Response alternated
Partial repetition costs
Repeated
Alternated
Repeated
Alternated
Abrupt
387 (8.9)
402 (9.2)
397 (17.1)
391 (6.5)
22 (11.0)
Fading
395 (6.4)
405 (13.8)
396 (17.2)
393 (5.3)
13 (19.2)
As summarized in Fig. 3, the resulting partial-repetition costs in abrupt conditions were very similar to those found in Experiment 1, and they were similarly affected by rotating the boxes. In contrast, fading conditions produced significant negative partial-repetition costs in reaction times with rotating boxes for location × shape, t(15) = −2.41, p < 0.02, and marginally significant costs for location × response, t(15) = − 1.61, p = 0.06; no effects were reliable in the error data, p values > 0.3.
Fig. 3
Partial-repetition costs in Experiment 2 of location-by-shape, location-by-response and shape-by-response as a function of rotation and fading
×
Error patterns followed the reaction times. Fading stimuli increased the error rate, F(1, 15) = 5.24, MSe = 0.04, p < 0.04. Location interacted significantly with response, F(1, 15) = 21.11, MSe = 0.117, p < 0.0005, as did shape, F(1, 15) = 73.15, MSe = 1.71, p < 0.0001. The interaction between location and response was modulated by rotation, F(1, 15) = 18.33, MSe = 0.16, p < 0.001. Fading did not interact with any factor or interaction other than the shape × response interaction, F(1, 15) = 11.85, MSe = 0.107, p < 0.005, with a greater effect of partial repetition in fading trials (see Fig. 3).
Discussion
The outcome is clear-cut. We were able to replicate the main findings of Experiment 1 with abruptly disappearing stimuli: Substantial partial-repetition costs were obtained for all three combinations of stimulus features and responses with static boxes, while these costs were eliminated for location-related interactions by having the boxes rotate. With fading stimuli, static boxes yielded a similar outcome as in the abrupt condition, but rotating the boxes produced a very different pattern. As expected, the partial-repetition costs including location were no longer close to zero but significantly inverted. Hence, there is evidence that the fading manipulation had the expected effect of increasing the contribution from the updated (i.e., post-rotation) event files, which supports the suggestion of Zacks et al. (2007) that smoother and thus more predictable changes between two events facilitate their integration.
Even though it seems clear that improving the visual link between S1 and S2 led to the integration of the corresponding event representations and that this integration was apparently prevented in Experiment 1, there are still two possible interpretations of how that may have produced the findings we obtained there. According to one scenario, separate event representations for S1 and S2 were created and maintained in Experiment 1 because the abrupt visual change worked against their integration. The absence of reliable effects in the rotation condition would thus reflect the common impact from two event files that effectively cancel each other out. According to the other scenario, the more abrupt visual change in Experiment 1 may have simply flushed visual working memory and deleted the representation of S1. Hence, even though Experiment 2 demonstrated that event-file updating does take place under conditions that favour event integration, it does not provide sufficient evidence to determine what happens under less favourable conditions. Experiment 3 was conducted to fix that.
Experiment 3
In Experiment 3, we investigated whether an abrupt visual change between S1 and S2 (as in Experiment 1) only prevents the integration of these events (and, thus, the updating of the S1 representation towards S2) or whether it leads to the deletion of any S1-related bindings. We did so by replicating Experiment 1, but letting the stimulus configuration rotate only 90°, so that the previous ‘non-updated’ location of S1 would no longer interfere with the new location of S2 (or the corresponding representation). Under these conditions, we had different predictions for the two scenarios we considered. If the abrupt visual change would lead to the deletion of any S1-related binding, no interactions between repetition effects should occur, that is, no evidence for feature integration should be obtained. In contrast, if the null effects obtained in Experiment 1 were due to bindings that cancelled each other out, using different spatial dimensions for S1 and S2 should prevent cancellation, so that reliable evidence for integration should be obtained.
Method
Four male and 11 female students from Leiden University voluntarily participated. The method was as in Experiment 1, except that S1 varied on the vertical and S2 on the horizontal dimension, with clock- or counter-clockwise rotations of 90° in between (the static condition was dropped). To avoid confusion with use of the term “location repetition” in the previous experiments (which in Experiment 3 no longer applied), we will use the term ‘rotation towards’ for a “rotation into S2” (e.g., S1 presented at the top → top box rotated to the right → S2 presented to the right) and ‘rotating away’ for the opposite case (e.g., S1 presented at the top → top box rotated to the left → S1 presented to the right).
Results
Overall, fewer errors were made for S1 (M = 3.0%, SD = 2.3%) than for S2 (M = 19.1%, SD = 8.2%). One subject was omitted from further analysis due to disproportionally low accuracies (M = 33.0%, SD = 7.0%). In a repeated measures ANOVA with fading (fading vs. abrupt), rotated location (towards vs. away), shape- and response-repetition as factors, responses were found to be less (2%) accurate if they were repeated, F(1, 13) = 9.79, MSe = 774.80, p < 0.01, but not slower, p > 0.3. Repeating the shape likewise increased (2%) error rates, F(1, 13) = 6.23, MSe = 213.72, p < 0.03, but did not affect RTs, p > 0.9. The fading condition did not affect error rates, p > 0.7, and only marginally yielded (3 ms) slower responses in fading conditions than in abrupt conditions, F(1, 13) = 4.07, MSe = 549.69, p < 0.07. Rotated location had no significant effect on RT, p > 0.5, or error rates, p > 0.1 (see Table 3; Fig. 4).
Fig. 4
Partial-repetition costs in Experiment 3 of rotated location-by-shape, rotated location-by-response and shape-by-response as a function of fading. Note that partial-repetition costs based on rotated location can only be understood as repetitions by virtue of their integration with their object, i.e., locations were never repeated between two stimulus-displays
×
Even though the location was never repeated as was the case in Experiments 1 and 2 (since the first stimulus-display was vertical and the second horizontal), rotated location had similar effects to location-repetition in the former experiments.2 Significant interactions were obtained between rotated location and shape repetition on RTs, F(1, 13) = 24.89, MSe = 4475.90, p < 0.001, though only marginally on error rates, F(1, 13) = 3.56, MSe = 57.21, p < 0.09 and between rotated location and response repetition on RTs, F(1, 13) = 22.02, MSe = 5033.50, p < 0.001, but not on errors, p > 0.2. Again, shape × response repetitions interacted significantly on RTs, F(1, 13) = 25.58, MSe = 4240.38, p < 0.001, and errors, F(1, 13) = 40.70, MSe = 3186.10, p < 0.001.
Discussion
The outcome of Experiment 3 is consistent with our interpretation of the null effects obtained in Experiment 1 in terms of multiple bindings and mutual cancellation but inconsistent with an account in terms of flushing. The latter would have predicted no integration-related effects, which is the opposite of what the data show.
Conclusions
The present study allows for three conclusions of theoretical relevance. First, moving an object leads to an update of the respective event file, including the response. In other words, response information does not only become integrated with object information but it seems to behave just like perceptual ingredients of object or event files. This bolsters the claim that object representations can include more than just perceptual codes (Kahneman et al., 1992) and, more specifically, that stimulus and action codes are bound into sensorimotor event representations (Hommel et al., 2001).
Second, whether particular events are coded into one or more event files seems to be determined by perceptual factors, possibly among others, such as the smoothness of the transition between perceptual states. According to Zacks et al. (2007), short-term event representations are used to predict future perceptual states on a continuous basis. As long as these predictions are successful, incoming information is considered to belong to the same event and, put into our theoretical terminology, integrated into the same event file. As soon as the predictions fail, a new event file is opened and the prediction-integration cycle goes on. Applying this logic to the present study implies that abrupt disappearance of a stimulus might be more likely to violate perceptual expectations and trigger the opening of a new event file than a stimulus that is softly fading out.
Third, it is interesting to note that both the rotation and the fading manipulation affected location-related bindings only—but no impact was observed on shape-response bindings. This is consistent with a number of other dissociations between different types of bindings (e.g., Colzato, Fagioli, Erasmus & Hommel, 2005; Colzato, Warrens & Hommel, 2006; Hommel 1998, 2007). Apparently, event files are not created by putting all the available evidence about a given incident on one pile. Rather, they seem to consist of multilayered structures of separable (and separately accessible and alterable), but nevertheless connected associations between feature codes (Hommel 2004).
All in all, this study provides insight into our ability to track objects in the wider scope of perception for action. Tracking an object through time and space must come in handy in the perceptual arsenal of many an animal, yet the use of tracking remains limited for prey or predator if it could not bind various actions to an object that was tracked. A prowling cat may passively store the location of various objects, but without binding the act of hunting to some of them it could easily mistake; a sleeping sparrow for a muddy rock on the bank of the river. This is not what cats want.
Acknowledgments
The authors would like to thank Cristiano Cellini for help in the data collection.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Met BSL Psychologie Totaal blijf je als professional steeds op de hoogte van de nieuwste ontwikkelingen binnen jouw vak. Met het online abonnement heb je toegang tot een groot aantal boeken, protocollen, vaktijdschriften en e-learnings op het gebied van psychologie en psychiatrie. Zo kun je op je gemak en wanneer het jou het beste uitkomt verdiepen in jouw vakgebied.
Partial repetition costs were computed as the difference in priming effects for one feature (F1) as a function of repeating (rep) versus alternating (alt) another feature (F2); partial repetition-cost = (F1repF2alt − F1repF2rep) − (F1altF2alt − F1altF2rep). For example, partial repetition costs in the shape x response domain were calculated as the response priming-effect with shape alternated subtracted from the response priming-effect with shape repeated.
Presenting S2 to the left and right created spatially compatible and incompatible relations with the left and right responses, a condition that is known to lead to the Simon effect (Simon & Rudell, 1967), i.e., better performance if stimulus and response spatially correspond. In another repeated measures ANOVA with S2-response compatibility, rotated location, response-repetition and shape-repetition as factors, the Simon-effect was found to be significant in RTs, F(1, 13) = 122.44, MSe = 83252.72, p < 0.001, and error rates, F(1, 13) = 29.90, MSe = 8015.66, p < 0.001, indicating that in spatially incompatible stimulus-response conditions, participants were 31 ms slower and made 11% more errors. Importantly, however, the Simon effect did not interact with any other factor.
