A motor similarity effect in object memory

An increasingly popular idea is that cognitive processing is a by-product of the interaction between the perceptual and motor systems used to interact with the objects in our environment—what is known as embodied cognition (e.g., Barsalou, 1999; Glenberg, 1997; Versace, Labeye, Badard, & Rose, 2009). In the same vein, a number of researchers in the domain of immediate memory have suggested that the motor system is recruited during retention (e.g., Jones, Hughes, & Macken, 2006). For instance, there is increasing evidence that verbal retention relies on the language production architecture (e.g., Acheson & MacDonald, 2009; Page, Madge, Cumming, & Norris, 2007) and that spatial memory recruits the oculomotor system (e.g., Guérard & Tremblay, 2011; Theeuwes, Belopolsky, & Olivers, 2009). Although several studies suggest that motor affordances play an important role in object recognition (e.g., Bub & Masson, 2010; Tucker & Ellis, 1998), evidence for its role in object memory is more limited and contradictory (e.g., Mecklinger, Gruenewald, Weiskopf, & Doeller, 2004; Pecher, 2013). The objective of the present study was to investigate the role of the motor system in object memory by manipulating the similarity between the actions associated with the objects to be retained in memory.

Several studies suggest that the actions associated to an object are automatically activated by its visual presentation (e.g., Helbig, Graf, & Kiefer, 2006; Labeye, Oker, Badard, & Versace, 2008; Tucker & Ellis, 1998). For instance, when the hand used to respond to an object presented visually on the computer screen is compatible with the orientation of its handle, response times are shorter than when it is incompatible (e.g., Tucker & Ellis, 1998). Tucker and Ellis proposed that viewing an object automatically activates the motor representations specific to the hand most suited to grasp it. Subsequent studies have shown such facilitation when the hand shape —that is, a precision or a power grip (e.g., Bub, Masson, & Cree, 2008; Ellis & Tucker, 2000)—or wrist rotation (e.g., Ellis & Tucker, 2000) used during response execution is compatible with the object, as compared with when it is incompatible. These studies suggest that the actions performed to physically interact with objects are activated during object recognition. Other researchers went further to suggest that the activation of these motor representations plays a critical role in object recognition (e.g., Bub, Masson, & Bukach, 2003; Helbig, Steinwender, Graf, & Kiefer, 2010). For instance, activation of a given action representation facilitates the recognition of an object that affords this action (Helbig et al., 2006; Helbig et al., 2010).

Some studies have suggested that motor affordances also influence object retention. For instance, in an fMRI study, Mecklinger et al. (2004; see also Mecklinger, Gruenewald, Besson, Magnié, & Cramon, 2002) used a delayed matching to sample task in which participants were first presented with the picture of an object that they had to retain in memory. The stimuli were manipulable and unmanipulable objects. After a delay of 10 s, participants had to indicate whether a test-object was the same as or the mirror image of the memorized object. Mecklinger et al. (2004) showed that the retention of manipulable objects primarily activated the premotor cortex, whereas nonmanipulable objects primarily activated the Broca’s area. Such evidence points to the possibility that object memory recruits the motor system used to interact with these objects (Mecklinger et al., 2002; Mecklinger et al., 2004). However, this conclusion was not supported by a recent study by Pecher (2013; see also Pecher et al., 2013). Pecher used a paradigm similar to the one used by Mecklinger et al. (2002) and added a motor suppression task to interfere with the activation of the actions associated with the objects presented during the experiment. She observed that suppression was equally detrimental to the recall of manipulable and unmanipulable objects and suggested that motor affordances play no role in the retention of objects.

The objective of the present study was to examine the role of motor affordances in immediate memory for objects by manipulating the similarity between the actions associated to a series of objects to be retained. Manipulations of similarity have been useful in order to examine the characteristics of stimuli that are used during encoding and retrieval. For instance, the finding that items sharing similar features are more difficult to recall in their order of presentation than dissimilar items has long been interpreted as evidence that these features are encoded and used for retrieval (e.g., Brown, Neath, & Chater, 2007). In order to examine whether motor affordances play a role in retention, we manipulated the motor similarity of lists of objects in a serial recall task where participants had to recall lists of objects in their order of presentation.

We used a serial recall task known to assess order memory for two reasons. First, the detrimental effect of similarity is typically found in order memory tasks (see, e.g., Conrad, 1964; Jalbert, Saint-Aubin, & Tremblay, 2008; Jones, Macken, & Nicholls, 2004). Most important, in the domain of immediate memory, the motor system is often considered as a mean to seriate information (see, e.g., Acheson & MacDonald, 2009; Jones et al., 2006), and its involvement in the retention of different types of materials has been mostly studied using tasks tapping order memory (see, e.g., Guérard & Tremblay, 2011; Jones et al., 2004).

Because an object can activate a wide variety of actions, we manipulated the action associated with each object by pairing each object with a short video representing a grasping movement (e.g., Helbig et al., 2010). In similar lists, all objects were paired with videos representing the same grip type—for example, a power grip—whereas in dissimilar lists, objects were paired with videos representing different grip types. The grip type employed to seize an object is a motor characteristic often manipulated to investigate the role of motor representations in cognitive processing (Bub et al., 2008; Ellis & Tucker, 2000). If motor affordances play a role in object memory, series of objects associated to the same grip type should be more difficult to recall than series of objects associated to different grip types. In Experiment 2, we combined the memory task with motor suppression (Pecher, 2013) in order to confirm the role of motor affordances in the similarity effect observed in Experiment 1. In Experiment 3, we replicated Experiment 1 using unmanipulable objects.

Experiment 1

Method

Participants

Twenty-eight undergraduate students from Université de Moncton volunteered to participate in the experiment. All participants were right-handed and reported normal or corrected-to-normal vision.

Materials

The course of the experiment was controlled by a PC computer using E-Prime (Version 2.0; Psychology Software Tools, Inc.). The stimuli were presented on a Planar PJT175R touch screen with a resolution of 1,024 × 768 pixels. Participants were seated 60 cm in front of the computer screen. Four 300-ms 7.12o × 7.12o grayscale videos of a grasping movement differing about the grip type they represented (Human Grasping Database; Feix, 2011) were used: a power grip (the object is held against the palm of the hand and the fingers close toward the palm of the hand), an index–thumb grip (a delicate grip requiring small force where the object is held between the index finger and the thumb), a fingers–thumb grip (the object is in contact with most of the fingers and is held between the tip of the fingers and the thumb), and a parallel extension grip (the object is held between the thumb and the whole surface of the fingers, which are pressed tightly against each other). Each video consisted of 10 photos of a hand against a white background presented in rapid succession (30 ms on/0 ms off).

The stimuli were twelve 7.12o × 7.12o grayscale photographs of objects chosen from the Bank of Standardized Stimuli (BOSS; Brodeur, Dionne-Dostie, Montreuil, & Lepage, 2010) and normalized on the type of grip necessary to grasp them (Lagacé, Downing-Doucet, & Guérard, 2013). In the norming study by Lagacé et al., participants had to select the grip most suited to grasp the objects. We selected 12 objects that were equally associated to two out of four possible grip types. For example, in the norming study by Lagacé et al., 40% of participants identified a parallel extension grip as the grip they would employ to grasp a small box, whereas 43% identified a fingers–thumb grip. We therefore selected two objects for each of the six possible pairs of grips (power/index–thumb, power/fingers–thumb, power/parallel extension, index–thumb/fingers–thumb, index–thumb/parallel extension, and fingers–thumb/parallel extension). As a result, each of the four grips was associated to six objects.

Forty sequences of six objects were created. In the 20 similar lists, the six objects were paired with the same grip type. Each grip type was used in 5 lists. In the dissimilar lists, the same sets of objects as those for the similar lists were used, but the six objects within the same list were paired with different grip types, with the restriction that each of the four grips was used at least once in every sequence. Therefore, the objects used in the similar lists and in the dissimilar lists were the same. Importantly, in each of the similar and dissimilar conditions, all objects were paired with each of their two associated grip types five times.

Procedure

On each trial, a sequence of six to-be-remembered objects was presented in the center of a computer screen. Each object was preceded by a black fixation cross appearing in the center of the screen for 1,000 ms, followed by the video for 300 ms. After the video, each object was presented for 1,500 ms. Participants were instructed to look at the videos and to memorize the order of the objects. After list presentation, all objects were re-presented simultaneously for recall in two rows of three objects in a different random order. Participants were required to touch the objects in their order of presentation (Fig. 1). Once selected, objects were surrounded by a red frame. Participants were asked to perform articulatory suppression—that is, to repeat the letters “A-B-C-D” during list presentation and recall to avoid verbal recoding. The articulatory suppression was recorded using an audio recorder. The experiment lasted 30 min.

Fig. 1
figure 1

Illustration of similar (left) and dissimilar (right) trials used in Experiments 1 and 2

Full size image

Design

There were two repeated measures factors: similarity (two levels: similar, dissimilar) and serial position (six levels: 1–6). The objects on each trial and the lists in the two conditions were presented in a different random order for each participant. The experimental trials were preceded by four practice trials.

Results and discussion

Responses were scored according to a strict serial recall criterion: To be scored as correct, the object had to be recall in the same order as it was presented in the sequence. In all analyses, the .05 level of significance was adopted, and the Greenhouse–Geisser correction was applied when the sphericity criterion was not met. Figure 2 suggests that dissimilar lists were better recalled than similar lists. A 2 (similarity) × 6 (serial position) repeated measures analysis of variance (ANOVA) performed on the proportion correct confirmed that the main effects of similarity, F(1, 27) = 4.36, MSE = .02, η 2 p = .14, and of serial position, F(5, 135) = 48.81, MSE = .03, η 2 p = .64, were significant. The interaction between similarity and serial position was not significant, F < 1. Therefore, Experiment 1 showed that when the objects to be retained are associated to the same grip type, they are more difficult to recall than when they are associated to different grip types.

Fig. 2
figure 2

Proportion of correct responses as a function of serial position and similarity in Experiment 1. Errors bars represent 95% confidence intervals

Full size image

Experiment 2

The objective of Experiment 2 was to verify whether the detrimental effect of similarity observed in Experiment 1 was due to the activation of similar actions. The task used in Experiment 1 was therefore combined with motor suppression (e.g., Pecher, 2013) in order to interfere with the activation of actions that could be recruited during object retention. Motor suppression consisted of opening and closing both hands and should, therefore, interfere with the activation of hand movements necessary to grasp the objects. If the effect observed in Experiment 1 was due to motor similarity, motor suppression should abolish the similarity effect.

Method

Participants

Twenty-eight undergraduate students from Université de Moncton volunteered to participate in the experiment. All participants were right-handed and reported normal or corrected-to-normal vision.

Materials

The materials were the same as those used in Experiment 1.

Design and procedure

The design and procedure were the same as in Experiment 1, except that participants were required to perform motor suppression during the presentation of the sequence. Motor suppression consisted of opening and closing both hands simultaneously by sequentially lifting up each finger—starting with the thumb and proceeding to the pinky—and then forming fists again (Pecher, 2013). Participants started the motor suppression task at the beginning of sequence presentation, until the time of recall. The experimenter remained in the room during the experiment to ensure compliance with the instructions.

Results and discussion

As is shown in Fig. 3, performance in the similar and dissimilar lists did not seem to differ. A 2 (similarity) × 6 (serial position) repeated measures ANOVA performed on the proportion correct showed that the main effect of serial position was significant, F(5, 135) = 60.99, MSE = .01, η 2 p = .69. The main effect of similarity and the interaction between similarity and serial position were not significant (Fs < 1). Therefore, Experiment 2 shows that the similarity effect observed in Experiment 1 is abolished when participants perform motor suppression, suggesting that the activation of similar actions was responsible for the detrimental effect of similarity in Experiment 1.

Fig. 3
figure 3

Proportion of correct responses as a function of serial position and similarity in Experiment 2. Errors bars represent 95% confidence intervals

Full size image

Experiment 3

In the two previous experiments, motor similarity was manipulated by presenting a grasping movement that preceded the objects to be retained. The underlying assumption was that the video activated a grasping movement that would be associated to the objects and that objects in the similar lists would be more difficult to recall because they are associated to the same action. One alternative explanation, however, is that the videos did not activate actions associated to the objects but that videos in the dissimilar condition provided more efficient retrieval cues than did those in the similar condition because each video is associated only to one or two objects. In order to test this hypothesis, Experiment 3 replicated Experiment 1 using unmanipulable objects. If the grasping movements in the dissimilar condition were used as more effective retrieval cues independently of the objects to which they were paired, the similarity effect observed in Experiment 1 should be replicated with unmanipulable objects.

Method

Participants

Twenty undergraduate students from Université de Moncton volunteered to participate in the experiment. All participants were right-handed and reported normal or corrected-to-normal vision.

Materials

The materials were the same as those used in Experiment 1, except for the objects to be retained, which were twelve 7.12o × 7.12o grayscale photographs of unmanipulable objects (elephant, bath, beacon buoy, cloud, drain, fire hydrant, inukshuk, lamppost, road, satellite dish, tree, and fence).

Design and procedure

The design and procedure were the same as those in Experiment 1.

Results and discussion

As is shown in Fig. 4, performance in the similar and dissimilar lists did not seem to differ. As in Experiment 2, a 2 (similarity) × 6 (serial position) repeated measures ANOVA performed on the proportion correct showed that the main effect of serial position was significant, F(5, 95) = 28.42, MSE = .03, η 2 p = .60. The main effect of similarity and the interaction between similarity and serial position were not significant (Fs < 1). Therefore, Experiment 3 shows that dissimilar videos are not sufficient to produce an advantage over similar lists. In order to be effective, dissimilar videos need to be associated to objects that afford the depicted action, as is the case in Experiment 1.

Fig. 4
figure 4

Proportion of correct responses as a function of serial position and similarity in Experiment 3. Errors bars represent 95% confidence intervals

Full size image

General discussion

The objective of the present study was to investigate the role of motor affordances in object memory. In Experiment 1, we showed that lists of objects associated to different grip types were better recalled than lists of objects associated to the same grip type. This effect was abolished by motor suppression in Experiment 2 and by using unmanipulable objects in Experiment 3, suggesting that it occurred due to the activation of motor affordances.

In line with several models of immediate memory (e.g., Brown et al., 2007), the finding that objects associated to similar actions are more difficult to recall than objects associated to dissimilar actions suggests that the activation of actions plays a key role in immediate memory for objects. One possibility is that actions represent one of several characteristics of objects that can be used during retention. For instance, according to SIMPLE (Brown et al., 2007), the characteristics of the items to be retained are represented along several dimensions in memory. When a characteristic allows differentiating the objects to be retained, the items are more discriminable along this dimension, which can then be used as an effective retrieval cue during recall. Therefore, the actions associated to the objects to be retained could be represented along one dimension and used during retrieval when they allow differentiating the different objects of a list. The lists of objects associated to dissimilar actions would be easier to recall than lists associated to similar actions because the action dimension could be used more efficiently in the former case. Importantly, the results of Experiment 3 suggest that in order to be useful during retrieval, the actions need to be compatible to the objects to be retained.

In line with theories of embodied cognition (e.g., Barsalou, 1999; Glenberg, 1997; Versace et al., 2009), another possibility is that the similarity effect we observed is due to the recruitment of the motor system during retention. For instance, Jones et al. (2006) have formulated the perceptual–gestural view of immediate memory, stating that immediate retention is a by-product of the interaction between our perceptual and motor systems. In line with this idea, several studies have suggested that the retention of verbal information recruits the language production architecture (Acheson & MacDonald, 2009). More precisely, Jones et al. (2006) suggested that the planning of speech is used to seriate information, by creating transitions between verbal items. Verbal rehearsal would therefore allow translating a list of to-be-remembered items into an articulated sequence of motor actions. The classic phonological – or acoustic – similarity effect showing that lists of similar sounding items are more difficult to recall than lists of dissimilar sounding items (Conrad, 1964) has recently been interpreted in light of the embodied cognition view (Acheson & MacDonald, 2009; Jones et al., 2006; Page et al., 2007). Jones et al. (2006) attributed the phonological similarity effect—at least partly—to elocution errors caused by stimuli that are pronounced similarly. The phonological similarity would therefore occur during the planning of speech, rather than being due to a confusion in some sort of phonological store (e.g., Baddeley & Hitch, 1974).

We therefore suggest that the motor similarity effect observed in the present study might also be explained in terms of the motor skills that are recruited for the retention of objects. Indeed, several studies have shown that the visual presentation of an object activates its motor affordances (e.g., Tuker & Ellis, 1998) and that they play a role in object recognition (e.g., Bub et al., 2003; Helbig et al., 2010). One possibility is that such affordances also play a role in the immediate retention of objects (e.g., Mecklinger et al., 2002; Mecklinger et al., 2004) through the creation of a chain of motor actions associated to the objects. For instance, the presentation of each object would activate the action it affords—in the present experiment, the grip necessary to seize it. The actions associated to the successive objects would then be assembled into a motor program, just as linguistic information associated to letters or words is assembled by the language production architecture in order to support their ordered retention (see, e.g., Acheson & MacDonald, 2009; Jones et al., 2006). Recall could then proceed by reinstating the motor program elaborated during the encoding of the sequence, with the emulation of each action used as a cue for the recall of the associated object. When the objects to be retained are associated to the same grip type, objects could be swapped during the planning of movements that is recruited to retain the sequence. The effect of motor suppression on the similarity effect in Experiment 2 replicates the effect of articulatory suppression on the phonological similarity effect (Murray, 1968): Since motor suppression prevents the recruitment of the motor system during the task, the detrimental effect of motor similarity is abolished.

The views of memory described above have been put forward to account for order retention and suggest that the motor features activated for each item are used to retain them in a particular order. However, although we used an order reconstruction task, errors might well occur because of a loss of item information (Neath, 1997), and our results do not allow determining whether the observed effects are due to order or item retention. The idea that motor affordances could be useful for item memory is in line with previous studies showing that the activation of motor features through enactment was most beneficial for item than for order recall (e.g., Engelkamp & Dehn, 2000; Olofsson, 1996) and that retention of a single manipulable object was associated to activation of the brain areas involved in object manipulation (e.g., Mecklinger et al., 2002; Mecklinger et al., 2004). Further research is therefore necessary in order to determine whether motor affordances play a role at the level of item retention, order retention, or both.

The present results therefore suggest that object retention recruits the motor system we use to physically interact with objects (Mecklinger et al., 2002; Mecklinger et al., 2004), which appears to be in contradiction to Pecher’s (2013) suggestion that motor affordances have no role in short-term retention of objects. The difference between our results and those of Pecher could be explained by differences in task requirements. In Pecher’s study, participants were required to retain only one object in memory, whereas our experiment required order retention, which might involve different processes for seriation. However, in a recent study, Pecher et al. (2013) showed no role for motor affordances using the N-back task, a task requiring the retention of the last N items in their order of presentation. This suggests that the order requirement of the task used in the present study is not responsible for the implication of the motor system.

Pecher’s conclusion (2013; Pecher et al., 2013) rests on the finding that motor suppression is no more detrimental to the retention of manipulable objects than to the retention of unmanipulable objects. Another possibility is that the retention of manipulable objects relies on the motor system but that, when the motor system is not available for retention—that is, when participants perform motor suppression—manipulable objects are retained using the same processes as those recruited during the retention of unmanipulable objects (e.g., verbal, semantic, or sensory processing) with no additional costs. This hypothesis is in line with Jones et al.’s (2006) idea that different skills can be recruited to support immediate retention.

In sum, the present results suggest a role for motor affordances in object retention. The nature of the underlying processes remains to be elucidated however. For instance, it is unknown whether the effect observed in the present study occurred because of the activation of the motor system recruited to interact with objects in the environment or through the activation of semantic knowledge. The effect of motor suppression on the similarity effect nevertheless suggests that the occurrence of this effect is dependent on the availability of the motor system. This study is therefore consistent with theories of embodied cognition, by suggesting that object retention is a by-product of the interaction between the perceptual and motor systems we use to interact with objects in our environment (e.g., Jones et al., 2006).