Interacting hands: the role of attention for the joint Simon effect

Liepelt, Roman

doi:10.3389/fpsyg.2014.01462

ORIGINAL RESEARCH article

Front. Psychol., 17 December 2014

Sec. Cognition

Volume 5 - 2014 | https://doi.org/10.3389/fpsyg.2014.01462

Interacting hands: the role of attention for the joint Simon effect

$\r\nRoman Liepelt*\r\n$ Roman Liepelt^*

Institute for Psychology, University of Muenster, Muenster, Germany

Recent research in monkeys and humans has shown that the presence of the hands near an object enhances spatial processing for objects presented near the hand. This study aimed to test the effect of hand position on the joint Simon effect. In Experiment 1, two human co-actors shared a Simon task while placing their response hands either near the objects appearing on the monitor or away from the monitor. Experiment 2 varied each co-actor’s hand position independently. Experiment 3 tested whether enhanced spatial processing for objects presented near the hand is obtained when replacing one of the two co-actors by a non-human event-producing rubber hand. Experiment 1 provided evidence for a Simon effect. Hand position significantly modulated the size of the Simon effect in the joint Simon task showing an increased Simon effect when the hands of both actors were located near the objects on the monitor, than when they were located away from the monitor. Experiment 2 replicated this finding showing an increased Simon effect when the actor’s hand was located near the objects on the monitor, but only when the co-actor also produced action events in spatial reference. A similar hand position effect was observed in Experiment 3 when a non-human rubber hand replaced the human co-actor. These findings suggest that external action events that are produced in spatial reference bias the distribution of attention to the area near the hand. This strengthens the weight of the spatial response codes (referential coding) and hence increases the joint Simon effect.

Introduction

Social context has an enormous impact on individual task performance. Observing another person’s action activates corresponding motor representations in the observer (Iacoboni et al., 1999), which may help when planning the same action or produces action conflict when planning or executing a different action (Brass et al., 2000; Liepelt et al., 2008, 2009; Sebanz and Knoblich, 2009; Wurm and Schubotz, 2012). Ideomotor theory (James, 1890; Prinz, 1997) and extensions thereof [theory of event coding, (TEC; Hommel et al., 2001)] assume that action control operates on the perceptual representations that usually follow a particular action. Action selection therefore consists in the activation of the perceptual consequences (i.e., action effects) of the to be generated action. Action perception, action selection and action execution use common representational codes leading to strong bidirectional interaction effects between perception and action (Prinz, 1990, 1997; Liepelt et al., 2011a). Such interactions also occur when we share tasks with others. Fluent interaction requires the coordination of both partners’ actions in space (Sebanz et al., 2003) and time (Sebanz et al., 2006; Liepelt and Prinz, 2011). Internal action plans need to be continuously updated according to ongoing changes in the arrangement of both actors’ bodies and body parts. Another person’s actions and body part positions represent an important information source to achieve fluent movement interactions between two actors.

Task sharing allows individuals to achieve common goals that they cannot achieve alone, which has played an important evolutionary role for group survival (Tomasello et al., 2005). Joint action research may therefore shed light into the most basic mechanisms on which more complex social cognitive functions, social institutions and societies are build on. Hand position plays an important role for joint action, because it can be considered to be a non-verbal cue to alert the co-actor’s attention to a particular location in space leading to a deeper processing of items appearing in near hand space. The present study examines the influence of relative hand position belonging to two different individuals on dyadic task sharing performance.

Dyadic Task Sharing and the Joint Simon Effect

An effective way to test dyadic task sharing effects is the joint Simon task (Sebanz et al., 2003). Here, two actors sit alongside each other sharing complementary parts of a standard Simon task (Simon and Wolf, 1963; Simon and Small, 1969; Simon et al., 1970). In the standard Simon task, a single person has to discriminate between two visual (e.g., different geometric forms) or two auditory (e.g., auditor words) target stimuli that are presented on the left or the right side of a screen or through the left and right speaker of a headphone. In most versions of the task, the participant had to respond exclusively to the shape of the visual stimulus or the meaning of the auditory stimulus by pressing a left or right button placed on the table in front of the person, but had to ignore the location (i.e., the stimulus position) where it was presented. Although stimulus position was fully task irrelevant, responses were typically faster when the stimulus position spatially corresponded with the response position [Stimulus–Response (S–R) compatibility], as when they did not correspond (S–R incompatibility) showing that participants were unable to ignore the spatial position of the stimulus (Simon effect). The Simon effect is often explained by assuming that response selection is facilitated in S–R compatible trials because the irrelevant stimulus features (left or right position of the stimulus) automatically prime the corresponding response. In S–R incompatible trials a response conflict arises between the automatically primed response and the instructed response (Kornblum et al., 1990; Hommel, 2011). Dimensional overlap is established between codes representing the irrelevant stimulus location and the features representing the response location (De Jong et al., 1994) or between the current or previous focus of attention to the left or right side following an attentional shift to the laterally presented stimuli and the spatially coded response (Nicoletti and Umiltá, 1989; Nicoletti, 1994). Note, however, that this mechanism is different from the Stroop effect (Stroop, 1935) or the Flanker effect (Eriksen and Eriksen, 1974) lacking a clear spatial S–R compatibility and a semantic conflict could arise between different stimulus features or between different response features that are mapped to multiple stimulus features (Hommel, 2011).

When one person has to respond to only one of the two stimuli (being in charge of only half of the task) with one button press (individual Simon task), the Simon effect usually disappears (Hommel, 1996), which makes sense as the alternative response location is removed eliminating the response conflict. When a second person takes over the other half of the task responding to the complementary stimulus, the Simon effect re-appears [joint Simon effect (JSE; Sebanz et al., 2003)], as in the standard Simon task. Sebanz and colleagues concluded that both participants sharing the Simon task form a common task representation corepresenting also the action of their co-actor (action corepresentation). Including both actors’ action alternatives in a common task representation re-establishes dimensional overlap between spatial stimulus and response dimensions. Just as in the standard Simon task this leads to response priming in S–R compatible trials and response conflict in S–R incompatible trials with the only difference that the two responses now relate to two different persons. Action corepresentation therefore supposes the processes underlying the joint Simon task and the standard Simon task to be functionally equivalent (Sebanz et al., 2003; Vesper et al., 2010).

The JSE is assumed to represent an index of action corepresentation (Sebanz et al., 2003) being sensitive to manipulations changing cognitive control parameters that regulate the integrativeness of control states thereby narrowing or broadening the attentional focus (Colzato et al., 2012, 2013). Further, manipulating the interaction quality between two humans (Hommel et al., 2009) and between humans and robots (Stenzel et al., 2012) has been shown to modulate the size of the JSE. Sharing a task with a friendly co-actor (Hommel et al., 2009) or a humanoid robot (Stenzel et al., 2012) that acted in a biologically inspired way increased the JSE, as compared to conditions in which the co-actor was unfriendly or the robot responded in a purely deterministic manner. The JSE seems to be disturbed in patients with frontal and parietal lesions incapable of performing Theory of Mind tasks (Humphreys and Bedford, 2011) and schizophrenia patients (Liepelt et al., 2012), but seem to be intact in patients with autism who pass first-order Theory of Mind tasks (Sebanz et al., 2005).

However, as the JSE represents a spatial compatibility effect, it strongly depends on the relative spatial setting between the irrelevant spatial stimulus information and the spatial response to be given (Sebanz et al., 2003), the spatial coding of responses (Dolk et al., 2011; Dittrich et al., 2012) and spatial attention to alternative events (e.g., a rhythm producing metronome) that provide a spatial reference frame for the coding of the participant’s own action as left or right (Dolk et al., 2013). Based on the latter findings a referential coding account has been proposed for the finding of a JSE, which is based on the general assumptions of ideomotor theory (James, 1890; Stock and Stock, 2004), and TEC (Hommel et al., 2001) in particular. TEC assumes that an individual controls his/her own intentional actions by the activation of feature codes that represent bundles of the perceivable effects this action typically produces. Accordingly, the perception of alternative action (or object) events representing the same or similar feature codes produces an action selection conflict between externally triggered and internally activated actions (i.e., their activated feature codes). The action selection conflict can be resolved by emphasizing (cf. intentional weighting principle; Memelink and Hommel, 2013) on action features that discriminate best between own and others actions—referential coding (Hommel, 1993; Hommel et al., 2001; Dolk et al., 2013; for a review see Dolk et al., 2014). According to the intentional weighting principle (Memelink and Hommel, 2013), the discrimination between two action events can be achieved by attending more strongly on response location changing the weight of the spatial (left–right) location code of each individual actor’s actions (Dolk et al., 2011, 2013). Spatial coding of the participant’s own response again introduces dimensional feature overlap between lateralized stimuli and both actors’ responses (Kornblum et al., 1990) and hence might explain the re-appearance of the Simon effect in a joint-task setting without needing the assumption of task corepresentation (Liepelt et al., 2011b). In the Simon task the best discriminating features are spatial response features. However, according to the referential coding account, in principle all other features could be used for referential coding depending on the specific task that two people share and the level at which the action selection conflict arises (Sellaro et al., in press). One should note here that referential coding might also include features that are often considered to be social (e.g., intentionality, in-group vs. out-group membership, agency, etc.), which can also be used to discriminate between actions produced by different actors (Dolk et al., 2014).

The JSE is typically not observed when the spatial reference that is usually provided by a human co-actor is made less salient (Guagnano et al., 2010; but see also Welsh et al., 2013). Guagnano et al. (2010) showed that the JSE, which is observed when two actors respond in peripersonal space, is diminished when they are located in extrapersonal space (out of reach distance). A similar finding is observed when the co-actor is placed in a separate room, which is spatially not further specified (Welsh et al., 2007; Sellaro et al., 2013; but see Tsai et al., 2008).

Enhanced Processing of Stimuli Near the Hand

Recent research provided evidence for effects of action on perception by showing how hands alter visual processing (Schendel and Robertson, 2004). Schendel and Robertson (2004) tested a patient with a damage of the right primary visual cortex who had a considerable loss in the left visual hemifield in a visual detection task. They showed that the patient was able to significantly improve the detection rate in their damaged visual field when reaching with their left-hand out to the left side of the monitor as when placing the hand to his lap. These findings are consistent with the finding of bimodal (visual and tactile) neurons in the putamen (Graziano and Gross, 1995), the premotor cortex (Graziano et al., 1994) and the parietal cortex (Colby et al., 1993; Iriki et al., 1996) of monkeys coding the space around the body (i.e., the monkey’s hand). Graziano and Gross (1995) showed that receptive fields of these neurons have a specific characteristic. They move with current hand position (Graziano and Gross, 1995) providing the neural substrate for altered vision near the hands (Graziano and Gross, 1995; Maravita and Iriki, 2004).

By using functional magnetic resonance imaging, brain areas have also been found in the human intraparietal sulcus that represent multisensory information in a hand-centered space (Makin et al., 2007). Cognitive research provides evidence for the role of attention for the finding of altered vision near the hands (Reed et al., 2006, 2010; Abrams et al., 2008; Witt and Proffitt, 2008; Davoli and Abrams, 2009; Davoli et al., 2010, 2012; Tseng and Bridgeman, 2011; Davoli and Brockmole, 2012; Qian et al., 2012; Schultheis and Carlson, 2013; Abrams and Weidler, 2014; Weidler and Abrams, 2014). Taking things in our hands not only affects visual search (Abrams et al., 2008), but has also profound effects on perception, attention and memory (Brockmole et al., 2013). Most studies testing effects of hand position on task performance compared performance differences between two conditions. One in which the participant responded with two response buttons attached to the display (objects close to the hands) and the other in which the person responded with both buttons located on participant’s knees (objects away from the hands). But why are objects close to the hands represented differently from objects away from the hands? A reasonable explanation given is that objects close to the hands are possible candidates for action. Since we know from ideomotor theory (James, 1890), and TEC (Hommel et al., 2001) that action and perception are tightly linked together, it makes sense to assume that the presence of the hand near an object can change the way in which we process that object (i.e., the way the object is represented). Most of the given evidence suggests that the presence of the hands enhances spatial processing for (or biases the distribution of attention to) the area near the hands (Schendel and Robertson, 2004; Reed et al., 2006; Abrams et al., 2008; Davoli et al., 2010; for a review see Brockmole et al., 2013). However, recent studies found evidence for reduced Stroop interference (Davoli et al., 2010) and reduced task-switching costs (Weidler and Abrams, 2014) for near hand stimuli providing evidence for enhanced cognitive control for stimuli that appear near the hands.

As the joint Simon task is a tool to investigate spatial processing also involving cognitive control processes, and if we assume that the position of our hands change the salience of space by providing references for upcoming actions (Dolk et al., 2013), then hand position should have a strong impact on the JSE. The aim of the present study was to systematically test the effect of hand position on the JSE.

Experiment 1

Participants had to perform three types of Simon tasks (individual Simon task, joint Simon task, and standard Simon task). Stimuli were geometric figures (square and diamond) that randomly appeared on the left or the right side of a centrally presented fixation cross. In the standard Simon task, participants were asked to press a left key for square and a right key for diamond. In both, the individual and the joint Simon task, participants responded to only one of the shapes by making a simple discrimination response. They were asked to refrain from responding when the other shape appeared.

Additionally, hand position was manipulated (Reed et al., 2006; Abrams et al., 2008). Two co-acting participants had to respond with buttons that were either located near the stimuli (hands monitor condition) or far away from the stimuli with their response buttons placed on their knees (hands knee condition). This manipulation was applied to all three tasks (individual Simon, joint Simon and standard Simon task). While the hand location was related to both hands of an individual actor in the standard Simon task, it was related to two different actors in the joint Simon task.

If hand posture biases the distribution of attention to the space near one’s own hand, one should predict a larger JSE when both actors’ response hands are placed on the monitor as when they are located on their knees. However, if people exhibit enhanced cognitive control for stimuli that appear near the hands (Weidler and Abrams, 2014), one should predict a smaller JSE when both actors’ responses are placed on the monitor as when they are located on their knees.

Methods

Participants

A sample of 24 students (12 male; mean age, 24.6 years; SD = 2.2) participated in this experiment. All were right-handed, had normal or corrected-to-normal vision, and were naive with regard to the hypotheses of the experiment. They were paid €7 or course credit points for taking part in the study. Participants gave their informed consent to participate in the study, which was conducted in accordance with the ethical standards laid down in the 1975 Declaration of Helsinki.

Stimuli and apparatus

Participants were seated in a sound-attenuated, dimly lit room. All stimuli were displayed on a CRT computer monitor (19-inch) in white on a black background at a constant viewing distance of 60 cm. The fixation point at the center of the screen was marked by a plus sign (0.9 × 0.9°). Stimuli consisted of squares and diamonds (1.9 × 1.9°), presented to the left or right of the fixation cross with an eccentricity of 5.7° visual angle. Responses were recorded with two keys. The keys were either placed on participants’ left or right knee or at the left and right side of the computer monitor with a distance of 50 cm from each other (see Figure 1, red border). The distance between response keys in the hands monitor and the hands knee condition was approximately kept constant. This was achieved by placing the response buttons on the outer knees that were matched in distance to the response buttons in the hands monitor condition.

FIGURE 1

FIGURE 1. Experimental design of the joint Simon task of Experiment 1 (red border) showing two persons sharing a visual Simon task (own hand: right person and other hand: left person) in the hands knee condition (lower right panel) and the hands monitor condition (upper left panel). In Experiment 1, all participants also performed an individual and a standard version of the Simon task using the same hand position manipulation (not shown). Experimental design for the joint Simon task of Experiment 2 (green border) showing the own hands knee – other hands monitor condition (upper right panel) and the own hands monitor – other hands knee condition (lower left panel) in addition.

Task and procedure

In the standard Simon task, participants responded by pressing the left key with the index finger of their left-hand for the square and the right key with their right index finger for the diamond. The stimuli randomly appeared on the left or the right of the centrally presented fixation cross. Participants were seated to either the left or the right side in front of the monitor in the joint Simon task. To keep the seating position identical in all three tasks an empty chair remained in place in the individual Simon task and the standard Simon task.

In the individual and the joint Simon tasks, participants responded to one of the shapes only (e.g., squares) by making a simple discrimination response. They were asked to refrain from responding if the other shape (e.g., diamond) appeared. In the joint Simon task, they performed the identical task sitting alongside another person who responded to the other stimulus. The left-sitting person responded with the left hand and the right-sitting person with the right-hand. In the standard Simon task left-hand and right-hand responses were related to a single individual.

Each trial began with the presentation of a fixation cross for 250 ms. The target stimulus (square or diamond) appeared together with the fixation cross for 150 ms. Afterward, responses had to be given within 1800 ms. In the case of correct responses, the fixation cross was provided as feedback for 300 ms. If no response was given within 1800 ms after stimulus onset, the feedback “zu langsam” (too slow) was shown. In the case of an incorrect response, error feedback “Fehler” (error) was provided. All forms of feedback (fixation cross, too slow, or error) were displayed for 300 ms. Following feedback, there was a constant inter-trial interval of 1750 ms before the next trial started. In each task, participants completed five experimental blocks of 112 trials, separated by short breaks. Before each task, participants performed a block of 66 practice trials.

Each participant performed the experiment on two consecutive days, performing all three tasks with a different hand position on each day. Half of the participants started with the hands monitor condition, the other half with the hands knee condition. The order of hand positions and tasks were counter-balanced across pairs of participants.

Results

For statistical reaction time (RT) analysis, all trials in which responses were incorrect (2.2%), and RTs were faster than 150 ms or slower than 900 ms (1%) were excluded to avoid distortions of statistic estimates. To test effects of hand position on the Simon task a 3 × 2 × 2 repeated measures analyses of variance (ANOVA) with the within-subjects factors setting (individual Simon task, joint Simon task, standard Simon task), hand position (hands monitor, hands knee) and compatibility (compatible, incompatible) was performed separately for RTs and percent errors. Planned post hoc comparisons were performed when required.

Reaction times

The RT analyses showed a significant main effect of compatibility, F(1,23) = 61.42, p < 0.001, partial η² = 0.73, which indicated that irrespective of setting and hand position, responses were faster with S–R compatibility (420 ms) than with incompatibility (434 ms), reflecting an overall Simon effect. The analysis also revealed a significant main effect of setting, F(2,46) = 44.06, p < 0.001, partial η² = 0.66. RTs in the joint Simon task were faster (391 ms) as compared to the individual Simon task (417 ms) and the standard Simon task (473 ms). Both factors, setting and compatibility significantly interacted, F(2,46) = 9.01, p < 0.001, partial η² = 0.28, showing that the compatibility effects was significantly increased in the joint Simon task (18 ms, p < 0.001), as compared to the individual Simon task (7 ms, p = 0.001), F(1,23) = 19.26, p < 0.001, partial η² = 0.46, while the compatibility effects between the joint Simon task and the standard Simon task (19 ms, p < 0.001) were not statistically different (F < 1). Further, there was a significant three-way interaction of setting, hand position and compatibility, F(2,46) = 3.37, p = 0.043, partial η² = 0.13 (see Figure 2) showing that hand position had a stronger impact on the compatibility effect in the joint Simon task, as in the individual Simon task, F(1,23) = 6.44, p = 0.018, partial η² = 0.22, and the standard Simon task F(1,23) = 4.20, p = 0.05, partial η² = 0.15¹. The significant interaction of hand position × compatibility in the joint Simon task, F(1,23) = 9.64, p = 0.005, partial η² = 0.30, indicated a larger compatibility effect when both actors’ hands were located on the monitor (22 ms, t = 7.1, p < 0.001), as when they were located on their knee (14 ms, t = 4.8, p < 0.001). No such effect of hand position on the compatibility effect was observed in the individual and the standard Simon task (both Fs < 1). No further main effects or interactions reached statistical significance (all ps > 0.05).

FIGURE 2

FIGURE 2. Mean reaction time (RT; in milliseconds) of the joint Simon task (upper middle panel), the individual Simon task (lower left panel) and the standard Simon task (lower right panel) as a function of hand position (hands monitor, hands knee) and compatibility (compatible, incompatible) of Experiment 1 (**p < 0.01 indicating the interaction of hand position and compatibility in the joint Simon task). Error bars depict the SE of the mean.

Error rates

For errors (see Table 1), there was a significant main effect of compatibility (1.8% in compatible trials and 2.6% in incompatible trials), F(1,23) = 16.89, p < 0.001, partial η² = 0.42, and a main effect of setting, F(2,46) = 29.56, p < 0.001, partial η² = 0.56. Also the interaction of setting and compatibility was significant, F(2,46) = 3.45, p = 0.040, partial η² = 0.13. We observed significant compatibility effects in the standard Simon task (3.1% in compatible trials and 4.8% in incompatible trials, p = 0.011) and in the joint Simon task (1.6% in compatible trials and 2.3% in incompatible trials, p = 0.044), but not in the individual Simon task (F < 1). No further main effects or interactions reached statistical significance (all ps > 0.05).

TABLE 1

TABLE 1. Error rates (percentage) shown for different Simon task settings (joint, standard, and individual) as a function of other’s hand position (other hand monitor, other hand knee), participant’s hand position (own hand monitor, own hand knee), and compatibility (C, compatible; IC, incompatible) for Experiment 1.

Discussion

Experiment 1 showed a JSE in the joint Simon task and the standard Simon task, as well as a tiny effect in the individual Simon task. Hand posture modulated the size of the Simon effect only during joint-task processing. When the hands of both actors were located near the monitor, the JSE was significantly larger, as when both actors’ response hands were more distant to the monitor. These data are in line with the prediction that an attention capturing hand of the co-actor located near the monitor biases the distribution of attention to the space near one’s own hand in the joint Simon task. Using a distribution analyses, it could be ruled out that the hand posture effect found in the joint Simon task was not due to a social facilitation effect and faster RTs in the joint Simon task as compared to the individual Simon task and the standard Simon task. The error data support the RT findings but are as in most studies measuring the JSE less meaningful, which is probably due to the relatively low overall error rate.

The finding of a Simon effect in the joint Simon task and of a tiny effect in the individual Simon task is in line with recent findings of other studies (Hommel, 1996; Liepelt et al., 2011b; Dolk et al., 2013) suggesting that the Simon effect is the result of an action discrimination problem between internally and externally triggered actions and its corresponding resolution by referential coding (Dolk et al., 2013, 2014). The hand position on the monitor seems to enhance the attention to the spatial response location leading to a larger JSE. This conclusion is in line with the finding of a relatively large JSE in the present study, as compared to previous studies measuring the JSE (Sebanz et al., 2003; Liepelt et al., 2011b, 2013; Dolk et al., 2013) using the standard response key arrangement and response hands placed on the table.

Experiment 2

In the first experiment both actors always moved their hands together either to the monitor or the knee position. Experiment 2 aimed to isolate effects of own and other’s hand positions, as well as their potential interactions, using the joint Simon task of Experiment 1. This experiment applied four different hand position conditions. While again using the two hand position conditions of Experiment 1 (own hand monitor–other hand monitor, own hand knee–other hand knee) to test for a replication of the main findings of Experiment 1, two new conditions were added (own hand monitor–other hand knee, other hand monitor–own hand knee). These two new conditions allow separating effects of one’s own hand position from effects of the co-actor’s hand position. If own hand position produced the spatial attentional bias that was found in Experiment 1, one should predict a larger JSE when the actor’s own response hand is placed near the monitor, as when it is placed away from the monitor. However, when the other person’s hand position modulated attention, one should find a larger JSE when the other actor’s response is placed near the monitor, as when it is placed away from the monitor. Finally, biasing the distribution of attention to one’s own task space may interact with intentional weighting (Dolk et al., 2013) so that a stronger weighting (of discriminative action features) can compensate effects of enhanced spatial attention. In this case, enhanced spatial processing of the stimuli in near hand space should affect the JSE when the other person’s hand is also located at the monitor, but less when the other person’s hand is located on the knee.