Response mode does not modulate the space–time congruency effect: Evidence for a space–time mapping at a conceptual level
Introduction
According to a prominent view in linguistics, philosophy, and cognitive psychology, people rely on spatial concepts when they talk and think about time (e.g., Casasanto et al., 2010, Evans, 2004, Fraser, 1966, Haspelmath, 1997, Klein, 2009, Tversky et al., 1991). Specifically, it is assumed that spatial concepts are necessary to structure our thinking about time, because time is an abstract concept and is thus not directly accessible via our senses (e.g., Grondin, 2001). In contrast, spatial concepts are structured by sensory and motor experiences and it is this structure that is inherited by the domain of time. This metaphoric mapping of space onto time has been called the spatial metaphor of time (e.g., Clark, 1973a).
The spatial metaphor of time is consistent with the notion of grounded cognition. Grounded cognition refutes the classical sandwich view (Hurley, 1998, p. 406) of cognition, which proceeds from the assumption of a functionally encapsulated system of cognition that receives input from perception and outputs its information to the motor system. Although there are diverse views of grounded cognition, they all share the notion that cognition is closely interwoven with sensorimotor experiences (see Barsalou, 2008, Wilson, 2002). The grounded cognition framework is supported by findings of neurophysiological studies that indicate the involvement of motor circuits during the processing of linguistic material (for a review, see Gallese & Lakoff, 2005; however, see Kranjec and Chatterjee, 2010, Miller and Brookie, 2012, Walsh, 2003). Furthermore, reading words like salt automatically activates brain regions that are involved in the processing of gustatory stimuli (Barrós-Loscertales et al., 2012). At a behavioral level, Glenberg and Kaschak (2002) demonstrated that the processing of sentence meaning interacts with the execution of manual responses. Similar findings were reported for the processing of word meaning (e.g., Dudschig, Lachmair, de la Vega, De Filippis, & Kaup, 2012). It is difficult to imagine how such results would emerge within the classical sandwich view of cognition as they clearly indicate a linkage of cognition and sensorimotor processes. According to the metaphoric mapping account of grounded cognition, however, even abstract concepts, like justice or love, are thought to be linked to sensorimotor experience (Lakoff & Johnson, 1980). Thus the mapping of space onto time is just one example of this general principle.
The idea that spatial concepts underlie our thinking about time has received strong support from reaction time (RT) studies (e.g., Santiago et al., 2007, Torralbo et al., 2006, Ulrich and Maienborn, 2010, Weger and Pratt, 2008). For example, in the study of Torralbo et al. (2006), participants responded to time-related information with a keypress on the left or right side. In the congruent condition, they responded to future-related information with a keypress on the right side with the right hand and to past-related information with a keypress on the left side with the left hand. In the incongruent condition, this stimulus–response (S–R) mapping was reversed. A space–time congruency effect was found; that is, RT was shorter in the congruent than in the incongruent condition. This congruency effect has been interpreted in terms of a mental timeline that runs from left to right.
Studies following Torralbo et al. (2006) have predominantly examined this left–right space–time congruency effect (e.g., Santiago et al., 2007, Ulrich and Maienborn, 2010, Weger and Pratt, 2008) and replicated it for different types of linguistic material. In addition, further studies have reported a space–time congruency effect on the front–back axis (Sell and Kaschak, 2011, Torralbo et al., 2006, Ulrich et al., 2012). Specifically, faster responses were observed when participants responded to future-related linguistic information with a forward movement and to past-related information with a backward movement, compared to when this mapping was reversed. This supports the existence of an additional mental timeline that runs from back to front.
As yet, the basis of these space–time congruency effects has not been addressed. At present, two alternatives based on a distinction between a weak and a strong view of metaphoric mapping seem to be possible. Both views share the assumption that bodily experiences are obligatory to enable mental representations. For the case of time this means that sensorimotor processes involved in the perception of and interaction with space (e.g., visual depth perception, self-motion in space) are essential building blocks of our mental representation of time. The strong view of metaphoric mapping presupposes that sensorimotor mechanisms are not only necessary to establish a mental representation but are also functionally involved whenever this mental representation is activated. In other words, thinking about abstract entities would be impossible without activating these low-level mechanisms. In particular, sensorimotor mechanisms that originate from our experience with space should be involved in the processing of temporal information and consequently in the emergence of the space–time congruency effect (e.g., Boroditsky, 2000, Gallese and Lakoff, 2005).
If the mental representation of time depends on low-level sensorimotor processes, as this strong view proposes, one would expect that the nature of the response modulates the size of the congruency effect because particular low-level sensorimotor processes are associated with particular response modes (e.g., vocal versus manual responses). For instance, manual response alternatives can differ in their spatial location whereas vocal responses cannot. Therefore manual response alternatives are usually defined by their spatial features (e.g., response location, or movement direction; see Rosenbaum, 1980) and consequently comprise sensorimotor processes related to space (e.g., response with the left vs. right hand, or forward vs. backward movements). On the other hand, vocal responses (e.g., the spoken words “left” vs. “right” or “forward” vs. “backward”) do not differ in their external spatial features, that is, the spatial location of vocal responses is always the same. Nevertheless, vocal responses referring to space (e.g., “left”, “right”, etc.) can convey abstract spatial information, as the words themselves contain spatial meaning. It is known that the size of S–R congruency effects increases with the number of features that are shared by responses and stimuli (Kornblum, Hasbroucq, & Osman, 1990, p. 259). Therefore one would expect a larger space–time congruency effect if both temporal stimuli and spatial responses share the same low-level sensorimotor features than when they do not. Consequently, the strong view suggests a larger space–time congruency effect for manual than for vocal responses.1
In contrast to the strong view, the weaker view of metaphoric mapping assumes that sensorimotor experiences are necessary only to establish the mental representation of time (Boroditsky, 2000). Once this representation is established, it is no longer functionally linked to those low-level sensorimotor processes. Therefore, this view postulates that the space–time congruency effect only reflects conceptual associations between time and space at higher cognitive levels. According to this view, spatial features of the response should not modulate the space–time congruency effect.
As far as we know, evidence for either of these two views is scarce. The strong view is in line with findings reported by Sell and Kaschak (2011). They observed a space–time congruency effect when participants were required to make an arm movement but no congruency effect when participants were required to make discrete keypresses (i.e. single keypresses, without preceding arm or hand movements toward the key), which indicates that response mode can modulate the congruency effect. Furthermore, Weger and Pratt (2008) report a spatial priming effect of time-related words which depends on response type. They used time-related words to prime the location of a subsequent target on the screen. A priming effect was observed only when participants responded with a left or right keypress to the location of the target but not when the target had to be detected in a simple RT task without spatially defined response alternatives. These results suggest that response mode even plays a major role in the occurrence of space–time congruency. Evidence consistent with the weak view has been reported by Ulrich et al. (2012). They found that motor features of manual responses, such as movement duration, are insensitive to a manipulation of space–time congruency, which suggests that the peripheral motor system is not involved in the emergence of the space–time congruency effect. Therefore the present evidence concerning the basis of the space–time congruency effect is inconclusive.
It should be stressed, however, that previous experiments did not aim to address this issue directly. Although a space–time congruency effect has been reported for vocal responses (Eikmeier et al., 2013, Torralbo et al., 2006), discrete keypresses (e.g., Santiago et al., 2007, Weger and Pratt, 2008) and arm movements (e.g., Sell and Kaschak, 2011, Ulrich and Maienborn, 2010), until now there has been no study that directly compares the size of the space–time congruency effect for different response modes.
The present experiment was designed to examine whether the spatial characteristics of the response modulate the size of the space–time congruency effect as predicted by the strong view of metaphoric mapping. To this end, we manipulated the response mode, while keeping all other aspects in our RT paradigm constant. Specifically, in one experimental condition, the responses were vocal, whereas in the other condition, the responses were manual movements. The strong view predicts a larger congruency effect for manual than for vocal responses, whereas the weak view predicts that the effect should not be modulated by response modality.
Section snippets
Experiment
Identical stimuli (words referring to past or future like yesterday or tomorrow) were employed in the manual and in the vocal condition of this experiment. All participants performed both response conditions. The mapping of the stimuli onto the responses was either congruent (i.e., yesterday → back response; tomorrow → front response) or incongruent (i.e., tomorrow → back response; yesterday → front response). All stimuli were presented auditorily to prevent eye movements during reading which might
Discussion
Several studies have reported a space–time congruency effect on RT (e.g., Casasanto and Bottini, 2014, Santiago et al., 2010, Torralbo et al., 2006, Weger and Pratt, 2008). Empirical evidence suggests that this congruency effect is consistent with a strong conceptual overlap between time and space (Eikmeier et al., 2013). Yet, to our knowledge, no study had directly addressed the question of whether the spatial characteristics of the response modulate the size of the space–time congruency
Acknowledgments
This work was supported by grants from the Deutsche Forschungsgemeinschaft (SFB 833, Project B7). We thank Simone Alex-Ruf, Donna Bryce, and Hartmut Leuthold and three anonymous reviewers for their helpful comments.
References (41)
- et al.
Random effects structure for confirmatory hypothesis testing: Keep it maximal
Journal of Memory and Language
(2013) Metaphoric structuring: Understanding time through spatial metaphors
Cognition
(2000)Does language shape thought? Mandarin and English speakers' conceptions of time
Cognitive Psychology
(2001)Space, time, semantics and the child
The language-as-fixed-effect fallacy: A critique of language statistics in psychological research
Journal of Verbal Learning and Verbal Behavior
(1973)- et al.
More on the language-as-fixed-effect fallacy: Monte Carlo estimates of error rates for F1, F2, F′, and min F′
Journal of Verbal Learning and Verbal Behavior
(1976) - et al.
Processing time shifts affects the execution of motor responses
Brain and Language
(2011) - et al.
Cross-cultural and developmental trends in graphic productions
Cognitive Psychology
(1991) - et al.
Left–right coding of past and future in language: The mental timeline during sentence processing
Cognition
(2010) A theory of magnitude: Common cortical metrics of time, space and quantity
Trends in Cognitive Sciences
(2003)
Reading salt activates gustatory brain regions: fMRI evidence for semantic grounding in a novel sensory modality
Cerebral Cortex
Grounded cognition
Annual Review of Psychology
lme4: Linear mixed-effects models using Eigen and S4
Mirror reading can reverse the flow of time
Journal of Experimental Psychology: General
Space and time in the child's mind: Evidence for a cross-dimensional asymmetry
Cognitive Science
Do task-irrelevant direction-associated motion verbs affect action planning? Evidence from a Stroop paradigm
Memory & Cognition
Dimensional overlap between time and space
Psychonomic Bulletin & Review
The structure of time: Language, meaning and temporal cognition
The voices of time: A cooperative survey of the man's view of time as expressed by the sciences and the humanities
The brain's concepts: The role of the sensory–motor system in conceptual knowledge
Cognitive Neuropsychology
Cited by (18)
Does walking/running experience shape the sagittal mental time line?
2023, Consciousness and CognitionCan mental time lines co-exist in 3D space?
2020, Acta PsychologicaCitation Excerpt :Actually, there is substantial evidence that supported the lateral and sagittal/vertical mental time lines may have different mapping mechanisms. Eikmeier, Hoppe, and Ulrich (2015) further found that response modes did not modulate the space-time congruency effect on the sagittal axis, which supported that the front-back space-time mapping emerges at a higher conceptual level. If the activation of time on the sagittal axis depends on a low-level sensorimotor process, the temporal stimuli and manual responses should share the same low-level sensorimotor features, which will lead to a larger congruency effect for manual responses than for vocal responses.
Action consequences affect the space-time congruency effect on reaction time
2019, Acta PsychologicaCitation Excerpt :Hence, it is unclear which spatial information of the whole action event determines the space-time congruency effect. If this effect on RT resides at a high cognitive level (e.g., Eikmeier, Hoppe, & Ulrich, 2015), it might be expected that people mentally align the left-right representation of time with external space. Consequently, the location of the more general action event should be the driving force, rather than the response key location.
The spatial representation of number, time, and serial order following sensory deprivation: A systematic review
2018, Neuroscience and Biobehavioral ReviewsCitation Excerpt :Although blind and sighted individuals share the experience of past events behind the body and future events in front of it when walking forward, blind individuals do not experience visual optic flow during locomotion, an experience that would be critical in setting a sagittal MTL. It is worth adding here that the way people “spatialize” time along the sagittal space may also depend on cognitive factors beyond perceptual experience (de la Fuente et al., 2014; Eikmeier et al., 2015). Intriguingly, in gesture patterns of the Aymara population in the Andes region the future is construed as behind the body and the past as in the front, because the past is more salient in that culture (Núñez and Sweetser, 2006).
Mixed metaphors: Electrophysiological brain responses to (un)expected concrete and abstract prepositional phrases
2018, Brain ResearchCitation Excerpt :The alternative view is the optional metaphoric mapping hypothesis (the “weak” hypothesis). In this view, abstract uses of prepositions can be understood independently of their spatial meanings, so that successfully understanding the phrase in love does not require the reconceptualization of love as container, but this conceptualization can be utilized when required by or encouraged by the task (Eikmeier et al., 2015; Kemmerer, 2005). There are a few studies that support the “strong” hypothesis, whereby metaphoric mapping is requisite.