Imagine you are redecorating your office and wondering what the room would look like if you rearranged the furniture. When you imagine moving the couch and the small reading lamp to the far wall, do you feel like you have to put in more effort to imagine moving the heavy couch? That is, do the properties of the couch that constrain your physical interactions with it, such as its weight and bulk, also constrain your imagined interactions? Here we ask this very question: are objects that are more difficult to physically manipulate also more difficult to mentally manipulate?

To explore this issue, we created a novel adaptation of the classic Shepard and Metzler (1971) mental rotation task (see also Shepard & Cooper, 1982). In the original experiment, participants were presented with two images of three-dimensional block objects and had to indicate whether the objects were identical or were mirror-images of one another. Response times were linearly proportional to the angular disparity between the objects, suggesting that participants were mentally rotating the objects in order to solve the task. These results, along with related findings have been used to argue that there is a close relationship between perceptual processing and mental imagery (Kosslyn, 1980; Kosslyn, Ganis, & Thompson, 2001; Kosslyn, Thompson, & Ganis, 2006; Shepard & Cooper, 1982).

Surprisingly, however, much less attention has been paid to the relationship between our actual physical experiences with objects and our subsequent ability to represent and manipulate those objects in our minds. On the one hand, there appear to be tight links among perception, action and mental imagery, both in behavior and in the underlying neural circuitry (Decety, 1996; Jeannerod & Frak, 1999; O’Craven & Kanwisher, 2000; Parsons, 1987; Porro et al., 1996; Schwartz, 1999; Thomas, 1999; Washburn, 1916; Wexler, Kosslyn, & Berthoz, 1998; Winawer, Huk, & Boroditsky, 2010). On the other hand, what often seems to separate our imagination from the everyday world is that it appears free from the constraints imposed by the physical environment. Indeed, people often imagine actions or events that they have never seen or never could experience themselves. We can imagine ourselves riding on a beam of light, leaping tall buildings in a single bound, or making a hook shot from half court. This begs the question, to what extent do our experiences with objects in the real world impinge on our ability to represent and manipulate those objects in mental imagery?

In the present study, participants first physically rotated wooden Shepard and Metzler-like objects. Crucially, one pair of objects was easy to rotate while the other pair was more difficult. Participants then completed a mental rotation task consisting of images of these objects. We were interested in whether motor imagery would be affected by the participants’ physical motor experience with the objects. Indeed, participants were slower to mentally rotate objects that were more difficult to physically rotate. To confirm that these effects were specific to motor imagery, we tested another group of participants with instructions to use a visual rotation strategy instead of motor imagery. In this case, participants’ motor experience with the objects had no effect on mental rotation speed, confirming that the effects were specific to motor imagery. These results reveal that motor mental imagery is constrained by our real-world motor experiences, but these constraints can be avoided by using an alternative mental imagery strategy.

Experiment

The experiment consisted of three parts: (1) physical rotation training with wooden Shepard and Metzler objects, (2) a mental rotation task consisting of images of the wooden objects and (3) a brief exit questionnaire.

Methods

Participants

Seventy Stanford students (34 females, 36 males) participated in exchange for payment or course credit. Sixty-four of the participants were right-handed while the remaining six participants were left-handed.

Physical rotation apparatus

We constructed four Shepard and Metzler-style block objects from small wooden beams (Fig. 1). The objects were approximately 15 cm tall and had small holes drilled into the center beam to allow them to be mounted to the rotation platforms. The objects came in two distinct shapes with two instances of each shape. One instance of each shape was painted blue while the other instance was painted red.

Fig. 1
figure 1

The four wooden Shepard and Metzler objects used in physical rotation training

Full size image

We also built two rotation platforms to control the rotational affordance of the objects (Fig. 2). Each rotation platform consisted of a coffee can with an axle installed in the bottom and a freely rotating wooden paddle inside. The objects could be mounted on top via protruding metal rods. One device was filled with sand, making it more difficult to rotate when an object was mounted onto it. The other platform was left empty, making it easier to rotate. Both devices were painted black, and their inner workings were visually obscured from participants by wooden covers.

Fig. 2
figure 2

An image and schematic of the rotation platform used for physical rotation training. The wooden Shepard and Metzler objects were mounted on the rods protruding from the top

Full size image

Physical rotation training procedure

Participants were trained on all four of the objects. One object color was assigned to the rotation platform filled with sand (hard to rotate) while the other color was assigned to the empty platform (easy to rotate). Thus, for a given participant, both red objects might be hard to rotate while both blue objects would be easy to rotate, and this was counterbalanced across participants.

When the participant arrived in the laboratory, one object was already mounted on the appropriate platform. Half of the participants rotated the hard-to-rotate objects first while the remaining participants rotated the easy-to-rotate objects first. Participants were instructed to stand at arm’s length from the object, grasp the object by the center beam with their right hand, and rotate it as far as they could clockwise and then counterclockwise. This step was repeated for the left hand. The object was then removed from its mounting by the experimenter, rotated 90° and remounted in a new orientation. Participants then performed the right- and left-handed physical rotations in this position. This process was repeated twice more until the object had been manipulated from four different orientations. At this point the experimenter replaced the object with the other object of the same color and participants replicated the physical rotation regimen described above for this second object. This whole procedure was then repeated for the other pair of objects on the other rotation platform. Finally, participants went through this entire process again from the beginning, so that by the end they had manipulated each object twice. The training session lasted approximately 8 min.

Mental rotation stimuli and procedure

The mental rotation task was conducted on an Apple iMac computer using Matlab software and the Psychophysics toolbox (Brainard, 1997). Stimuli consisted of full color digital photographs of the physical objects described above. Each object was photographed in eight different orientations. These orientations were chosen to allow for the unambiguous identification of each object from the digital image. Objects that were the same shape but different colors were photographed from the same angles so that the only way to distinguish them from the photograph was by color.

On every trial, one of these 32 images appeared in the left half of the display, subtending 12-15 cm by 12-15 cm depending on the orientation, angle and shape. Either the same image or a mirror-reflected version of the image appeared in the right half of the display. The image on the right could be rotated 0°, 60°, 120° or 180° clockwise in the picture plane. Every possible trial based on the combination of these images was used exactly once in this experiment, yielding 256 unique trials. The order of trials was randomized for each participant.

Participants were instructed to press the “F” key if the left and right images showed the exact same object and the “J” key if the objects were mirror-reversed images of one another. Feedback was given after each trial in the form of a pleasant ding or buzzer sound for correct and incorrect responses, respectively.

We were interested in the effect of physical motor experience with the objects on participants’ ability to imagine manipulating the objects. Further, we wanted to test whether any effects would be specific to motor imagery. To this end, participants were tested in one of two minimally different conditions: motor imagery instructions or visual imagery instructions (following Kosslyn, Thompson, Wraga, & Alpert, 2001). Instructions in the Motor Imagery condition were: Imagine that you are grasping one of the objects with your hand and turning it until it aligns with the other object. In the Visual Imagery condition they were: Imagine that one of the objects is rotating by itself and turning until it aligns with the other object.

Exit questionnaire

At the end of the experiment, participants rated how difficult it was to engage in the type of imagery they were instructed to use (1, extremely easy to 10, extremely difficult), how often they used this type of imagery during the task (1, never used it to 10, used it on every trial) and how difficult it was for them to physically rotate the objects from each color category (1, extremely easy to 10, extremely difficult). This last question measured the effectiveness of the physical rotation manipulation for each participant. Finally, participants were asked: “What do you think was the purpose of this experiment?”

Results and discussion

Data from six participants (3 females) were removed prior to analysis due to error rates greater than 30% on the mental rotation task (N = 4), failure to complete the exit questionnaire (N = 1), or because the mental rotation computer program crashed before the participant completed all trials (N = 1). These participants were replaced to make a counterbalanced set of 32 participants in each imagery condition. The resulting final set of 64 participants had a mean accuracy of 87%.

Manipulation checks

Participants rated the objects that were mounted on the sand-filled rotation platform (mean = 6.2, SD = 1.8) as significantly harder to physically rotate than the objects mounted on the empty rotation platform (mean = 2.2, SD = 1.4), t(63) = 17.4, p < 0.0001. They also reported using their assigned type of imagery on most trials (mean = 7.83, SD = 1.53), and further reported that using this imagery was moderately easy (mean = 3.98, SD = 2.05). There was no difference in reported imagery difficulty (t(62) = 0.91, p = N.S.) or amount of assigned imagery use (t(62) = 1.23, p = N.S.) between the two conditions.

Mental rotation results

Mirror-reversed response trials were treated as distractors and therefore not included in the analysis. Only correct trials were analyzed. Trials where the RT was more than 5 SD slower than the mean RT across all participants for a given angle of rotation were excluded, leading to the removal of less than 1% of trials.

Our first analysis consisted of a mixed-design, 2 (physical rotation difficulty: Easy vs. Hard) × 4 (angle of rotation: 0°, 60°, 120°, 180°) repeated measures ANOVA with imagery type (Motor vs. Visual) included as a between-subjects factor. In line with previous studies of mental rotation, the linear effect of angle of rotation was significant, with RTs increasing linearly with increased angle of rotation, F(1, 62) = 394.66, p < 0.0001 (Fig. 3). This effect did not vary by imagery type, F(1, 62) = 0.09, p = N.S. There was no main effect of physical rotation difficulty, F(1, 62) = 1.25, p = N.S. However, there was a significant physical rotation difficulty by imagery type interaction, such that participants who used motor imagery (but not visual imagery) were overall slower to mentally rotate objects that were more difficult to physically rotate, F(1, 62) = 6.31, p < 0.02 (Fig. 4). In accuracy patterns, there was no main effect of difficulty of rotation (F(1, 62) = 1.28, p = N.S.), and no interaction by imagery type (F(1, 62) = 0.55, p = N.S.), ruling out a speed-accuracy trade-off.

Fig. 3
figure 3

Reaction time by angle of rotation, with separate lines for how difficult the objects had been to physically rotate during training, and separate plots for each imagery condition

Full size image
Fig. 4
figure 4

Overall cost of mentally rotating an image of a more difficult to physically rotate object for each imagery condition. The difference scores were obtained by taking the average difference in RT between images of the hard-to-physically rotate objects and images of the easy-to-physically-rotate objects, collapsed across trials with a rotation angle of 60˚, 120˚, and 180˚. Error bars represent standard error

Full size image

To further explore the relationship between physical rotation, mental rotation and imagery type, we ran a separate 2 (physical rotation difficulty) × 4 (angle of rotation) repeated measures ANOVA for participants in each of the two imagery conditions. For participants who used visual imagery, there was a significant linear effect of angle of rotation (F(1, 31) = 177, p < 0.001), but no main effect of physical rotation difficulty (F(1, 31) = 1.31, p = N.S.) and no interaction between physical rotation difficulty and the linear effect of angle of rotation (F(1, 31) = 0.81, p = N.S.).

For participants who used motor imagery, however, in addition to a significant linear effect of angle of rotation (F(1, 31) = 222, p < 0.001) there was also a main effect of physical rotation difficulty (F(1, 31) = 5.23, p < 0.03); RTs increased linearly with increased angle of rotation, and participants were overall slower to mentally rotate the objects that were more difficult to physically rotate. We were further interested in whether mentally rotating an object that is difficult to physically rotate imposes a fixed cost, or whether the cost accrues over angles of rotation (as it would if one were physically rotating the object). For participants who used motor imagery, the interaction between physical rotation difficulty and the linear effect of angle of rotation did not quite reach significance, F(1, 31) = 2.99, p = 0.094. One intriguing possibility is that participants who did not perceive there to be much of a difference between the objects during physical rotation would not show much of a difference during mental rotation. In the exit questionnaire, participants rated how difficult it was to rotate objects of each color during physical rotation training. From this we obtained a physical rotation difference score, and we repeated the above analysis excluding those participants whose difference score was smaller than one standard deviation away from the mean (N = 4). The average difference score for the remaining 28 participants was 5.03. This analysis revealed a significant interaction between physical rotation difficulty and the linear effect of angle of rotation, F(1, 27) = 5.41, p < 0.03. This result demonstrates that the cost of mentally rotating an object that you perceive as difficult to physically rotate accrues with increasing angles of rotation (just as it would if one were physically rotating the object).

The fact that physical experience with the objects had an effect only in the motor imagery condition and not in the visual imagery condition helps confirm that these effects are specific to motor imagery (and not simply a function of demand characteristics or more general response biases). Participants in both imagery conditions had the same physical experience with the objects and in the exit questionnaire were equally likely to predict that hard-to-rotate objects would be slower to mentally rotate (6 and 5 out of 32 participants in the motor and visual imagery conditions, respectively). Participants using motor imagery were in fact slightly more likely than those using visual imagery to make the opposite prediction that hard-to-rotate objects would be faster to mentally rotate (5 and 1 out of 32, respectively). The disconnect between people’s explicit predictions and the actual RT patterns observed reflects the fact that RT in the context of mental rotation is an implicit measure, one that participants are unlikely to intentionally modulate. All participants were motivated to complete the task as quickly as possible, and it is improbable that participants explicitly chose to slow a subset of their responses, and only in one of the conditions.

General discussion

We find that things that are more difficult to move in the real world are also more difficult to imagine moving, but this is only true when we engage in motor imagery and does not hold for purely visual imagery. That is, our previous motor experience with an object matters when we imagine moving it, but not when we imagine the object moving on its own. These results suggest that while we may automatically activate the motor representations associated with objects in some tasks (e.g., Chao & Martin, 2000; Tucker & Ellis, 1998), motor information is not necessarily activated in all tasks. This is consistent with previous work showing that our present motor state (or recent motor experience) can impact judgments of things like walking distance or the steepness of a hill in some but not all task situations (e.g., Proffitt, 2006; Proffitt, Stefanucci, Banton, & Epstein, 2003; Schwartz & Holton, 2000). One common thread across these studies may be that when people make a judgment by imagining themselves acting in the world (e.g., physically rotating the objects, walking across a room, climbing a hill), their judgments are affected by their previous or immediate motor experience (even if this leads to errors in judgment or costs in reaction time; see also Schwartz, 1999).

What are the mechanisms underlying motor and visual imagery that may give rise to this pattern of results? There appears to be a tight link between real and imagined actions. In recent years, many researchers have converged on a view of mental imagery that proposes that the same cognitive and neural systems that underlie perception and action also underlie imagined perception and action (Decety, 1996; Jeannerod & Frak, 1999; O’Craven & Kanwisher, 2000; Thomas, 1999; Winawer, Huk, & Boroditsky, 2010). Support for this view comes from a variety of sources, including behavioral and neuroimaging studies. For example, the time it takes to imagine an action is tightly correlated with the time it takes to execute that action (Jeannerod & Frak, 1999; Kunz, Creem-Regehr, & Thompson, 2009). Further, many of the same brain regions are activated for both imagined and executed perceptual and motor actions (Decety, 1996; Kunz et al., 2009; Porro et al., 1996). For instance, imagining faces and places selectively activates visual brain regions responsible for processing these stimuli (O’Craven & Kanwisher, 2000). Indeed, imagining a particular visual scene (e.g., of downward motion) can even adapt direction-selective neurons and cause a measurable motion after-effect illusion, in the same way that adapting to real downward motion would (Winawer, Huk, & Boroditsky, 2010). On this view, then, mental imagery is constituted by the (covert, partial) reinstantiation of the same set of perceptual-motor processes that would be engaged during the actual performance of a given action or perception of a given object (Decety, 1996; Thomas, 1999).

Adopting this framework helps situate the findings from the present study. During physical rotation training, participants learned about the motor affordances of the objects and stored away these experiences in memory. Previous research suggests that people can rapidly calibrate their perceptual-motor systems in this way and integrate new information with respect to their goals in order to plan and execute actions (Rieser, Pick, Ashmead, & Garing, 1995). The dissociation between patterns in motor and visual imagery in our mental rotation results suggests that the motor information participants stored away was re-instantiated only when participants were trying imagine the motor actions, and not when they imagined the objects moving on their own. This study offers behavioral evidence that motor and visual imagery draw on different cognitive processes, supplementing previous neural evidence (e.g., Kosslyn et al., 2001). It appears that people recruit different types of information into their mental imagery in a principled fashion: when they imagine physically interacting with objects, they include motor information, but when they do not imagine themselves acting on the world, they do not.

These findings raise some interesting questions about the way experience interacts with mental imagery. For example, what kinds of previous experiences can have an impact on motor imagery? If instead of performing a motor action yourself you simply observed another person performing the action, would the ease or difficulty with which they performed it influence your ability to imagine performing it yourself? More generally, what are the kinds of information (e.g., visual, linguistic, etc.) that can impinge on, constrain, or even enhance our motor imagery?

Conclusions

Are objects that are more difficult to physically manipulate also more difficult to mentally manipulate? Participants who engaged in motor imagery were slower to mentally rotate objects that were more difficult to physically rotate. Further, the cost of mentally rotating such objects accrued with increasing angles of rotation (just as it would if one were physically rotating the objects). Strikingly, this was true even though participants only had a limited amount of physical experience with the objects, the easy- and hard-to-rotate objects were identical in shape (they differed only in color), and the effect was observed in an implicit RT task where participants were motivated to work as quickly as possible. Finally, we verified that these effects were specific to motor imagery by showing that the effects can be eliminated by using strictly visual imagery (imagining the objects moving on their own). These results reveal that motor mental imagery is constrained by our real-world motor experiences (and that these constraints can be avoided by using an alternative mental imagery strategy). So, the next time you imagine redecorating your office, you can save some time by imagining the couch moving itself.