Hand-use norms for Dutch and English manual action verbs: Implicit measures from a pantomime task

How do people understand the meaning of words? According to theories of embodied cognition, people construct linguistic meaning by relying, in part, on neural systems for action and perception (Barsalou, 1999; Pulvermüller, 1999). A fruitful testbed for these theories has been the neural representation of action verb semantics: When people process action language, they show somato-topic patterns of activity in (pre)motor cortex (e.g., when processing verbs like “kick”,“pick”, and“lick”, people show activity in motor areas controlling the leg, hand, and mouth; Hauk et al., (2004) and Aziz-Zadeh et al., (2006)).

Unimanual action verbs have offered particularly strong theoretical leverage for testing embodiment claims, because of the hemispheric specialization of the motor systems controlling the hands: Whereas left hemisphere motor circuits control the right hand, right hemisphere motor circuits control the left hand. By relying on this organizational principle of the neural systems for manual control, Willems and colleagues (2010b) used unimanual action verbs to show that left- and right-handers, who perform the same manual actions in different ways, also process language about those actions in correspondingly different ways. When reading unimanual action verbs, right-handers showed stronger activity in left premotor cortex areas controlling the right hand, whereas left-handers showed relatively increased activity in right premotor cortex controlling the left hand (Willems et al., 2010a; see also Casasanto 2009, 2011). Other studies have used unimanual action verbs to demonstrate that premotor circuits play a functional role in how people understand language about unimanual actions (Willems et al., 2011; Gijssels et al., 2018), and that linguistic context can shape the manner and extent to which people rely on the motor system for processing action language (Tomasino et al., 2010; Gijssels et al., in preparation; see Aravena et al., 2012, 2014 for related behavioral results).

Most action language studies explicitly include manual action verbs in their stimulus sets, and many neuro-imaging and neuro-stimulation studies rely almost exclusively on these verbs for their conditions of interest (e.g. fMRI: Tomasino, Werner, Weiss, & Fink, 2007, 2010; Willems, Toni, Hagoort, & Casasanto, 2010a, b; Hauk and Pulvermuller 2011; Yang et al., 2011; Yang & Shu 2014; Gijssels et al., in preparation; EEG: Boulenger et al., 2008; Moreno et al., 2013, 2015; Vanhoutte et al., 2015; (r)TMS: Tomasino et al., (2008), Willems et al., (2011), Tremblay et al., (2012), Repetto et al., (2013), and Vukovic et al., (2017)). Across these studies, researchers typically identify different types of manual action verbs by relying on participants’ explicit judgments of how they use their hands to perform the actions described by the verbs. In most studies, participants rate the extent to which they associate these verbs with manual actions (e.g., Hauk et al., (2004)). A few studies asked participants to rate how much they rely on each hand to perform these actions (Willems et al., 2010a, b; Hauk and Pulvermuller 2011; Gijssels et al., 2018), but even when studies collected continuous ratings of the relative contributions of the left and right hand, researchers have used these ratings to classify manual verbs into discrete categories (e.g., unimanual vs. bimanual vs. nonmanual), for the purpose of treating the stimuli within each category as members of an equivalence class.

Yet, the assumption that all “unimanual” verbs refer to strictly unimanual actions, and that all “bimanual” verbs refer to symmetrically bimanual actions, is unlikely to hold. Instead, the degree to which people rely on one vs. two hands for performing manual actions is likely to vary continuously, rather than categorically. Although some verbs that are categorized as unimanual may describe actions that rely exclusively on the dominant hand (e.g., “to wave”), others may describe actions that also rely on the non-dominant hand (e.g., “to cut”, where the dominant hand does most of the work, but the non-dominant hand plays an essential role in stabilizing the object being cut).

If some unimanual verbs describe actions that partly rely on the non-dominant hand as well, then this category-internal variation could form a significant source of type II error for studies like those cited above—or a significant source of theoretical leverage for future studies if this category-internal variation can be quantified^{Footnote 1} (see Fischer (2008), Tschentscher et al., (2012), and Morrissey et al., (2018)). For instance, right-handers may show strongly left-lateralized premotor cortex activity when reading a fully unimanual verb like “to wave”, but they may show a more bilateral pattern of (pre)motor cortex activity when reading a verb like “to cut”. Here, we quantified the continuous variability in how people use their dominant and non-dominant hands to perform manual actions, with the goal of providing both explicit and implicit measures of hand use for creating action-verb stimulus sets.

We constructed one Dutch and one English corpus of verbs that describe manual actions and we provide norms of how people use their hands to perform those actions. First, we adopted the same explicit rating procedure that is commonly used in studies on action verb semantics. Participants read one verb at a time and explicitly rated the extent to which they typically use their left and right hand to perform each of the actions described. Second, because actions are natural to perform but notoriously difficult to put into words, we also collected an implicit measure of how people perform these actions. Before they performed the explicit rating task, participants pantomimed each of the actions in the stimulus set. We coded the relative activity of the left and right hand separately, for each of the pantomimes. These implicit measures allowed us to quantify the way in which people typically perform a large set of manual actions, and also to directly compare how people’s explicit ratings correspond to their actual motor behaviors cued by the same verbs.

Participants

For each language, we aimed to recruit 36 native speakers (i.e., three participants for each of the counterbalancing conditions in the explicit rating task; see Procedure). Due to human error, one additional participant was collected for the Dutch sample (N = 37).

Dutch sample: 37 participants performed the Dutch norming experiment, all of whom were native Dutch speakers, as indicated by a language background questionnaire. We used the Edinburgh Handedness Inventory (EHI; Oldfield (1971)) to establish participants’ handedness. Three participants were left-handed (M = -59.2; range = -75 to -40), 31 participants were right-handed (M = 91; range = 60 to 100), and three participants were ambidextrous (M = 19.5; range = 0 to 38.5). Following Vrije Universiteit Brussel IRB guidelines, all participants provided informed consent and received 10€ for their participation.

English sample: 38 participants took part in the English norming experiment. We replaced the data from one participant for not following task instructions, and from a second participant for not being a native English speaker. All remaining 36 participants were native English speakers, based on their responses to a language background questionnaire. As indicated by their EHI scores, four participants were left-handed (M = -62.5; range = -80 to -50), 28 participants were right-handed (M = 78.1; range = 41 to 100 ), and four participants were ambidextrous (M = 27.5; range = 20 to 37.5). Following University of Chicago IRB guidelines, all participants provided informed consent and received $10 for their participation.

Since we decided a priori to only collect and analyze data from right-handers (n = 31 Dutch right-handers, n = 28 English right-handers), all ambidextrous and left-handed participants were excluded from the analyses (n = 6 Dutch non-right-handers, n = 8 English non-right-handers).

Stimuli

We constructed two separate stimulus sets for the norming experiments, one in Dutch and one in English. Each of these sets consisted of manual action verbs: verbs that describe actions people typically perform using their hands (e.g., “to write” or “to untie”). Some of the verbs were selected from the stimuli of previous studies on action semantics (Willems et al., 2010a; Hauk and Pulvermuller, 2011; Akinina et al., 2015; Shao et al., 2014; Vinson & Vigliocco, 2008). Other verbs were novel additions, suggestions from thesauruses, or Dutch-English translation equivalents.

All manual verbs were included into our stimulus set, with the following constraints. First, we excluded verbs that could also be used to describe nonmanual actions (i.e., actions performed with effectors other than the hands, e.g., “to roll”). Second, since Dutch infinitives often share their word form with semantically related plural nouns, we only included those word forms that are used more frequently as verbs than as nouns (based on INL frequency from CELEX; Baayen et al., (1995)). For instance, the word form “graven” is used more often as a plural noun (English: “graves”) than as a verb (English: “to dig”), and was excluded from our list. After applying these constraints, each of the final stimulus sets consisted of 268 unique manual action verbs (Dutch word length: M = 8.2 characters; range = 5 to 13 characters; English word length: M = 5.12 characters; range = 3–10 characters).

Finally, for each language we also selected 32 nonmanual action verbs which served as stimuli for the catch trials in the explicit rating task (see below). Half of these stimuli described actions that are typically performed using the mouth (e.g., “bijten”, “to bite”), whereas the other half described actions typically performed using the legs (e.g., “wandelen”, “to walk”).

We chose to present our stimuli in the infinitive form because, following Hauk et al., (2004), many studies on action language processing have presented verbs in their infinitive forms.

Procedure

The norming experiments were identical for both languages, and consisted of two separate tasks: a pantomime task followed by an explicit rating task performed on the same verbs.

Pantomime task

In the pantomime task, participants read one verb at a time and pantomimed the action described by each verb. Participants were seated on a chair without armrests in front of a computer screen. At the start of each trial, participants placed their hands in the designated resting position (i.e., both hands were placed palms down on the upper legs). Then, the experimenter pressed a button, causing the trial to start. Each trial started with a fixation cross (1s), which was then replaced by a single verb presented in its infinitive form (e.g., “schrijven” in Dutch; “to write” in English). Participants read the verb, acted out the action it described, and then returned their hands to the resting position, at which point the experimenter pressed a button causing the verb to disappear, and the next trial to start. If participants did not know the meaning of a verb, they verbally responded “pass”, causing the experimenter to press a button that ended the current trial and started the next.

Throughout the entire task, we videotaped all of the participants’ responses. For both the Dutch and the English sample we positioned the camera so that it could accurately capture each participant’s face, arms, and legs. In the Dutch experiment, participants were seated in a chair that was positioned about 1.5 m away from the computer screen. The camera (Sony HDR-SR11; 1920×1080) was positioned 1.5 m to the right of the computer screen, so that the camera was about 2 m from the participants, facing them at about a 45 degree angle. The angle of the camera was chosen so that while participants were seated, their arms, face, and legs were in full view. In the English experiment, we recorded participants’ pantomimes using the built-in web camera of the computer on which the experimental stimuli were displayed (21” Apple iMac, Late 2013). The screen was positioned directly in front of participants at a distance of about 2.5 m. Again, the distance and angle of the camera were calibrated so that participants’ arms, face, and legs were in full view.

Participants received a few practice trials at the start of the implicit rating task, and received a short break halfway through the experiment. The entire stimulus set was divided into two subsets of 134 stimuli each (one presented before the break and one presented after; order counterbalanced across participants). The order of the stimuli within each subset was fully randomized.

Explicit rating task

After participants completed the pantomime task, they performed an explicit rating task on the same verbs. On each trial, participants saw a single verb on the computer screen and then answered the following sequence of questions. The first question asked: “Which body part do you usually use to perform this action?” Participants answered by pressing a button that corresponded to “hands”, “mouth”, or “feet” (or pressed the space bar to pass). If participants answered “mouth” or “feet”, the trial ended and the next one started.

If participants answered “hands”, the second question appeared, which asked “Do you usually perform this action with one hand or with two hands?” Here, participants answered by pressing one of two buttons, corresponding to “one hand” or “two hands”. For the third and final question, participants provided a continuous rating indicating the relative left- to right-handedness of an action. If participants responded “one hand” on the second question, then the continuous rating prompt asked “If you use one hand, which hand do you use to perform this action?” Participants responded by selecting a value on a Likert scale ranging from 1 (“Always left hand”) to 7 (“Always right hand”), with the midpoint 4 being labeled as “No preference”. If participants responded “two hands” on the second question, then the continuous rating prompt asked “If you use two hands, which hand does most of the work to perform this action?” Participants responded by selecting a value on a Likert scale ranging from 1 (“Mainly left hand”) to 7 (“Mainly right hand”), with the midpoint 4 being labeled as “Both hands equally”. For the first and second categorical questions, the response mappings were fully counterbalanced across participants. The order of stimuli was fully randomized across participants. At the start of the task, participants received three practice trials and they received three short breaks throughout the task.

Data coding

To quantify the extent to which participants used each of their hands to pantomime the described actions, we trained two pairs of coders to analyze the video data from the Dutch and English experiments (one pair of coders assigned to each language, all four coders were right-handed by self-report).^{Footnote 2} For each trial, the coders watched the video recording and then provided three separate scores indicating the relative activity of the participant’s left hand, of the participant’s right hand, and of any of the participant’s nonmanual effectors (e.g., the face or legs). The score for each effector could range from 0 (“No voluntary movement”) to 5 (“Voluntary, ostensive movement”), and was based on the coder’s subjective impression of each effector’s force, amount of movement, and duration of use. For instance, if a participant pantomimed “to punch” by repeatedly punching forcefully with the right hand, without moving their left hand or the rest of their body, the respective scores would be: Left Hand = 0; Right Hand = 5; Nonmanual = 0. Yet, if a participant pantomimed “to unscrew” by holding a jar in the left hand, while unscrewing the lid using the right hand, the relative scores may be: Left Hand = 3; Right Hand = 5; Nonmanual = 0.

Finally, coders scored the activity levels of each effector relative to the overall activity level of all effectors combined, for a given trial. This “relative” rating was necessary in light of our observation, during pilot data coding, that the same hand action could constitute different amounts of the total movement in the context of different pantomimes. For instance, if a participant pantomimed “to pinch” by using their right hand to make a single, small pinching movement, they would receive a similar score as in the “to punch” example above (i.e., Left Hand = 0; Right Hand = 5; Nonmanual = 0), since for this trial, the right hand was the sole effector used for the entire action. By contrast, if a participant pantomimed “to hammer” by using their right hand to make a large, repeated hammering movement, while using their left hand to make a single, small pinch-and-hold movement to pantomime stabilizing a nail, then the relative scores may be: Left Hand = 3; Right Hand = 5; Nonmanual = 0, since most of the movement in this pantomime is being performed by the right hand. As such, although each of the two pinching movements may appear similar in isolation, they differ in how much they contribute to the overall movement of the action, and therefore they would receive different scores.

In addition to the relative activity scores, coders also marked actions that were not correctly performed (“error” or “pass”) or for which the participants moved their hands outside of the camera’s view (“uncodeable”).

Before coding the full data set, each pair of coders calibrated their coding system by coding a few participants, comparing those scores, and then agreeing on a coding approach for trials with divergent scores. After this calibration phase, each coder independently coded the rest of the data set. Based on the final set of ratings, our coders showed high levels of agreement between their ratings, both for the Dutch data set (inter-coder correlation for Left Hand Score: β = .93, R² = .91, χ²(1) = 233.11, p<.0001; Right Hand Score: β = .92, R² = .84, χ²(1) = 358.9 , p <.0001; and Nonmanual Score: β = .71, R² = .73, χ²(1) = 160.12, p <.0001) and the English data set (inter-coder correlation for Left Hand Score: β = .94, R² = .93, χ²(1) = 240.38, p<.0001; Right Hand Score: β = .95, R² = .90, χ²(1) = 90.947, p <.0001; and Nonmanual Score: β = .50, R² = .31, χ²(1) = 66.329, p <.0001).

Finally, for the explicit rating task, we coded trials as incorrect when participants gave a “pass” response or when they chose an incorrect or atypical body part for the action (e.g., a “feet” response for the verb “to chew”).

Analyses

To be able to directly compare the explicit and implicit continuous ratings, we transformed the explicit scale so that it covered the same -5 to + 5 range of the implicit scale. We converted the original 1 to 7 scale into a -5 (“Left Hand dominant”) to + 5 (“Right hand dominant”) scale by applying the following transformation: Transformed Value = (Original Value -4) × 5/3.

Analogously, for the implicit continuous ratings, we converted the separate hand activity scores for all of the pantomime data into a single measure that reflects the relative extent to which participants rely on the right vs. the left hand: Right Hand Dominance. For each trial, we first calculated the Right Hand Dominance for each of the two coders separately (Right Hand Score - Left Hand Score), and then averaged these two scores, arriving at a single Right Hand Dominance (RHD) value for each trial. This RHD reflects the relative activity of each of the hands for a given pantomime, and ranges from -5 (“Left Hand Only”) to + 5 (“Right hand Only”). Additionally, by taking the absolute value of this RHD, we also constructed a continuous measure of unimanuality, which reflects the extent to which participants rely on one vs. two hands to perform a specific action (labeled as Implicit Unimanuality in our reports).

Finally, we performed all statistical analyses using R (v. 3.5.3). For the subject-wise analyses comparing participants’ implicit behavior to their explicit ratings, we used the ‘lme4’ package for mixed-effect analyses with maximal slopes structures (Barr et al., 2013).

Accuracy

To identify items whose meanings were ambiguous or hard to interpret, we analyzed the overall by-item accuracy separately for each task and language. Most items were classified with high accuracy, in both the explicit tasks (Dutch: M = 93.25%, SD = 14%; English: M = 97.7%, SD = 4.77) and the implicit tasks (Dutch: M = 91.33%, SD = 15.44; English: M = 94.34%, SD = 11.3). To identify items with markedly low accuracy, we performed an outlier exclusion by removing any items with a mean accuracy 2.5 SDs below the task mean (Dutch verbs: 17 manual and 1 nonmanual excluded; English verbs: 18 manual and 1 nonmanual excluded).^{Footnote 3} The final set of items used for the analyses consisted of 251 Dutch manual stimuli, 250 English manual stimuli.

Then, we analyzed the by-participant accuracy to ensure all participants performed the task correctly. Most participants had high accuracy in the explicit tasks (Dutch: M = 95.77%, SD = 3.88; English: M = 98.4%, SD = 1.32) as well as in the implicit tasks (Dutch: M = 93.93%, SD = 6.32; English: M = 95.95%, SD = 4.26). Two participants in the Dutch experiment and two participants in the English experiment had accuracy scores 2.5 SD s below the group mean in one or both tasks, and were excluded from further analyses.

Variability in hand use across and within items

Across all types of ratings, manual actions in both languages showed wide variability in the extent to which they relied on the left and right hand. First, both Dutch and English verbs differed in how consistently they were categorized as being unimanual or bimanual. Whereas some verbs were never classified as being unimanual (i.e., M = 0% unimanual; e.g., for the Dutch verb “melken” (“to milk” in English) or the English verb “to type”), and others were always classified as unimanual (i.e. M = 100% unimanual; e.g., for the Dutch verb “aankloppen” (“to knock on someone’s door” in English) or the English verb “to point”), the average categorization of most items was somewhere in between: The average proportions of unimanual categorizations for all items show a continuous distribution from 0 to 100% (Dutch: M = 36.75%; English: M = 44.81%; see Fig. 1).

Item-wise averages of explicit manuality categorizations. Left: Dutch verbs; Right: English verbs

Full size image

Second, the items also differed in the extent to which participants pantomimed them using the right hand exclusively. The mean Right Hand Dominance (RHD) scores for each of the verbs covered a wide range, not only for verbs that were explicitly categorized as being bimanual (Dutch: M = 1.28; range = -.03 to 4; English: M = 1.03; range = -.15 to 4.1) but also for verbs that were categorized as unimanual (Dutch: M = 3.69; range = 1.23 to 4.96; English: M = 3.5; range = -.08 to 5; see Figs. 2 and 3).

Item-wise RHD scores for Dutch verbs, averaged across participants. Each verb’s explicit categorization (Bimanual vs. Unimanual) was assigned based on the option chosen by most participants. Error bars reflect SEM. Note: Plot separated into two for legibility

Full size image

Item-wise RHD scores for English verbs, averaged across participants. Each verb’s explicit categorization (Bimanual vs. Unimanual) was assigned based on the option chosen by most participants. Error bars reflect SEM. Note: Plot separated into two for legibility

Full size image

Finally, manual actions varied in how consistently participants pantomimed the same action in the same way. One source of inconsistency stems from different participants varying in how much they used their left hand to pantomime the same action, as reflected in the variation in SEMs of the item-wise RHD scores (Dutch SEM range: .06 to .66; English SEM range: .06 to 1.03; see Figs. 2 and 3; Tables 1 and 2). A second source of inconsistency is that for some items, participants used only one hand to pantomime the action, yet were equally likely to use their left or right hand to do so, as indicated by an item’s low average RHD and high Implicit Unimanuality score (e.g., “to nudge”; see Tables 1 and 2).

Correspondence between implicit and explicit ratings

The average explicit ratings for the manual items were significantly correlated with their average implicit ratings. The proportion of unimanual categorizations for an item predicted that item’s average Implicit Unimanuality score (β = 4.24; t(499) = 50.06; p <.0001; R² = .83). Moreover, an item’s average explicit continuous rating predicted its average RHD, both for bimanual verbs (β = .80 ; t(300) = 21.63; p <.0001, R² = .61) and unimanual verbs (β = .63; t(197) = 9.61; p <.0001; R²= .32).

Similarly participants’ explicit ratings for individual items also significantly predicted their corresponding implicit ratings, even though these ratings explained substantially less variance in the data. The way in which a participant explicitly categorized an action predicted their Implicit Unimanuality score for that action (β = 1.08, t(114) = 15.76, p <.0001; R² = .08), and their explicit continuous ratings predicted the RHD score of their pantomimes, both for verbs they categorized as bimanual verbs (β = .19, t(82) = 10.14, p <.0001; R² = .03), and as unimanual (β = .17, t(64.31) = 4.5, p <.0001, R² = .01).

Here, we constructed corpora of Dutch and English manual action verbs and provided continuous measures of how people use their left and right hands to perform the actions described by these verbs. We collected implicit measures of hand use by having right-handed participants pantomime the manual actions described by each of the verbs. In addition, participants explicitly rated how each of their hands contributes to performing these actions. Our results demonstrate that unimanuality (i.e., whether an action is performed with only one hand or both hands) varies continuously, rather than categorically: Participants pantomimed most actions using both their right and left hand, for so-called “unimanual” and “bimanual” actions alike, with the relative contribution of their left hand varying continuously across the different stimuli. Additionally, by comparing participants’ explicit and implicit ratings, we found only limited support for the assumption that people can explicitly report on their procedural knowledge of actions accurately. When averaging across items, the explicit ratings showed a moderate correlation with participants’ implicit behavior, yet within an individual participant, explicit reports only explained a small fraction of the variance in how people performed actions (between 1 and 3%). Together, these data show that not all unimanual and bimanual action verbs are created alike, and underscore the potential value of using implicitly measured continuous norms for studying the representation of manual action verbs.

Using manuality norms to test embodied cognition

Many studies have used manual action verbs to test theories of embodied cognition, and several have done so by leveraging the hemispheric lateralization of systems for manual action. Yet, most of these studies either do not empirically establish the manuality of their stimuli (e.g., Repetto et al., (2013)), or treat manuality in a categorical way by classifying their stimuli as being either unimanual or bimanual (e.g., Willems et al., (2010a), Niccolai et al., (2014), and Vukovic et al., (2017)).

The current results show that few actions are either exclusively unimanual or symmetrically bimanual: People pantomimed most actions by using both hands, although the contribution of the left hand varied continuously across items. This pattern suggests that treating manuality as a categorical distinction obscures the continuous variation in how people perform manual actions, and may be a significant source of type II error in experiments that posit hemispheric specialization for unimanual actions, or a right- vs. left-hand advantage in behavioral responses. That is, if actions that are assumed to be performed unimanually with the dominant hand also rely on the non-dominant hand, then the neural and behavioral responses to verbs describing these actions may show less specificity than expected, thereby decreasing a study’s power to detect the predicted results, and increasing the likelihood of drawing spurious conclusions. To illustrate this point, suppose one’s category of “unimanual stimuli” included actions like “to sew” or “to comb”; according to explicit ratings these actions are unimanual, but according to the way people pantomime these actions they are clearly bimanual. If the experimenter were hypothesizing that neural activity for “unimanual actions” should be lateralized to the hemisphere that controls the subject’s dominant hand, the results might fail to show the predicted pattern of activity for reasons that do not invalidate the experimenters’ hypothesis, but rather reflect undetected heterogeneity in their so-called “unimanual” stimuli.

By providing a set of measures that quantify the continuous variation in how people perform manual actions, these corpora offer researchers increased control over their stimulus selection, which in turn should allow them to minimize type II error (due to uncontrolled variation within a stimulus class) and to increase theoretical leverage by making more precise predictions about the neurocognitive correlates of action verbs. Our continuous measures can support two different strategies for dealing with heterogeneity in manual action stimuli. First, these norms can be used to construct stimuli that are either mostly unimanual or mostly bimanual: That is, they can be used to construct stimuli for experiments that treat manuality categorically, ensuring that the manuality categories are homogenous. For instance, by selecting items with high mean RHD and high Implicit Unimanuality scores, researchers can construct a set of verbs describing actions that mostly rely on the right hand (e.g., English: to knock, to salute, to wipe; Dutch: aankloppen, wuiven, steken). Alternatively, since our data show that manuality varies continuously rather than categorically, these norms also allow researchers to leverage this continuity as a research tool (e.g., by testing whether an action’s RHD varies continuously with the degree of laterality of neural or behavioral responses). Whereas we provide the item-wise averages for the relevant measures in Tables 1 and 2, we also provide full access to the raw data and R code through OSF, allowing researchers to construct their own measures.

Do explicit ratings reflect implicit behavior?

To select stimuli for experiments on embodied cognition, researchers typically use norms of the extent to which different words elicit specific perceptuo-motor experiences (e.g., Lynott and Connell (2009)). Often, these norms are constructed by having participants explicitly rate words on a set of theoretically relevant dimensions, based on the assumption that participants’ explicit ratings will accurately reflect their implicit knowledge. Yet, the extent to which this assumption is valid has remained unclear, especially when considering explicit ratings of procedural domains (e.g., action) that are notoriously hard to access declaratively.

Here, we show that people’s explicit ratings of hand use significantly predict some of the variance in how they enact those actions through pantomime, yet the explicit ratings also leave large amounts of the variance unexplained. At the group-level, participants’ judgments reliably reflected whether an action is performed with one or two hands, yet judgments were markedly less reliable in reflecting the continuous variation in each hand’s relative contribution to a given action. However, an individual participant’s explicit ratings captured very little of the variance in their overt behavior, suggesting that people are not able to reliably report on their own actions. Therefore, even though explicit norms may be easier to collect and may provide a moderately reliable approximation of categorical distinctions when ratings are averaged across items, our results suggest that implicit measures provide a more reliable index of continuous distinctions, especially at the level of the individual item, and the individual person.

Do manual pantomimes accurately reflect how people perform manual actions?

Although pantomimes share many behavioral and neural similarities with the actions they depict (see e.g., Weiss et al., (2000) and Grèzes et al., (2003)), they also diverge in several ways: Pantomimes and real manual actions differ in some of their kinematic properties (e.g., grip aperture or velocity; Senkfor (2008), Goodale et al., (1994), and Laimgruber et al., (2005)), as well as in the cognitive processes they rely on Frey (2008). That is, whereas manual actions are typically highly detailed and non-communicative, pantomimes are often schematic and used for communicative purposes (Senkfor, 2008; Cartmill et al., 2012).

Do these differences between how people pantomime and perform manual actions pose a problem for using the current corpora? Not necessarily. Our main goal was to provide a measure that allows researchers to predict the extent to which people show lateralized motor system activity when processing language about manual actions, and these pantomime ratings are likely to provide a reliable way of doing so, for several reasons. First, whereas explicit ratings of manual actions provide a declarative (verbal) measure of procedural action knowledge, our pantomime ratings likely offer a better approximation of manual action performance because pantomimes also rely on procedural (motor) knowledge. Second, even though the schematic nature of pantomimes may make them less similar to the manual actions they depict, this schematicity may make pantomimes more similar to language-induced motor simulations than real actions are. Just as participants’ pantomimes reflect manual actions that are decontextualized (i.e. the prompts did not specify the objects being acted upon, or other contextual constraints), so too do the linguistic stimuli that are typically used in studies of action language processing. For instance, in these studies, action verbs are often presented in their infinitive forms (e.g., “to chop”), leaving many contextual aspects of the action unspecified, which may lead participants to simulate these actions in ways that are more sparse and schematic than they would be when actually performing these actions (see Willems et al., (2010b) for discussion). Real actions are strongly constrained by the affordances of the objects being acted upon; therefore, two instances of the “same” action may be very different motorically. Pantomimes can capture schematic elements that are common to multiple different instantiations of the same action. In this way, pantomimes are similar to words, which encode actions at a level of schematicity and generality that allows the same verbs to describe many diverse instances of the actions they name.

Given that pantomimes are procedural (like motor actions) and schematic (like semantic representations), they are likely to provide reasonable approximations of the way in which people implicitly simulate actions during language understanding. Nonetheless, whereas these pantomime norms may be appropriate for constructing stimuli to test action language understanding, they should be used with caution in studies that want to construct stimuli for testing overt execution of manual actions, in which case they may need additional verification.

How do Dutch speakers’ pantomimes compare to English speakers’?

Our primary goal in the current project was to construct two separate corpora, to support studies of action language processing in two of the most popular languages in psycholinguistics research. Although the Dutch and English lists contain many “translation equivalents,” we do not assume that pairs of “equivalent” verbs have exactly the same meanings across languages, or that they refer to exactly the same actions. Furthermore, not every verb in one language corresponds to exactly one verb in the other language: During the translation process we discovered, for example, that a single verb in one language corresponds to a family of nearly synonymous verbs in the other (i.e., a one-to-many relationship). We also found, in some cases, that a verb in one language corresponds most closely to a word in the other language that has a nominal form that is used more frequently than its verbal form and therefore had to be excluded (see the Stimuli section). An attempt to quantitatively compare the manuality patterns for so-called translation equivalents, therefore, would be complicated and (we believe) ill advised.

With these caveats in mind, we note that the first author, a fluent speaker of both Dutch and English, identified several verbs that seem to have very similar meanings across languages; accordingly manuality ratings were nearly identical for such verbs, including “to shave” (scheren) and “to initial” (paraferen). For other verbs, the closest translation equivalents refer to different sets of actions. For example, the English verb “to squeeze” does not appear to have any precise translation equivalent. To squeeze, according to our English pantomimes, is a strongly bimanual action (RHD = 0.44). By contrast, one close translation equivalent, “Knijpen,” is far more unimanual (RHD = 3.40), consistent with its reference to a squeeze that English speakers might describe as a pinch. Another Dutch verb, “Uitpersen,” which refers to the kind of squeezing that an English speaker might describe as “juicing” (i.e., to squeeze an orange), has an intermediate manuality rating (RHD = 2.30) indicating that one hand typically does the juicing while the other hand stabilizes the object being juiced. Thus, some “equivalent” verbs in our Dutch and English lists correspond to different patterns of hand use, but this result may simply reflect that the notion of semantic equivalence across languages (i.e., exact synonymy) is problematic.

Dominant hand bias for manual actions

In this study, people used both hands to perform most manual actions, including those actions they explicitly classified as being unimanual. Yet, our data also show that for the majority of these actions people relied more on their dominant hand than on their non-dominant hand. This dominant hand bias was not restricted to verbs classified as being unimanual, but extended to the ones categorized as being bimanual as well (mean RHD of bimanual verbs > 0: Dutch: t(160) = 16.53; p <.0001; English: t(140) = 13.02; p <.0001). This finding confirms previous suggestions that when people perform bimanual actions, they rarely use both hands to an equal extent, but instead divide the workload asymmetrically across both hands. For instance, when hammering a nail into a wall, right-handers typically use their left hand to stabilize the nail whereas their right hand does the more effortful job of hammering it (see Guiard and Ferrand (1996) and Guiard (1987)).

This dominant hand bias may also explain why several neuro-imaging studies found left-lateralized patterns of motor system activity when participants processed manual action verbs, even though these verbs were not explicitly selected to be unimanual (e.g., Hauk et al., (2004), Aziz-Zadeh et al., (2006), and Tettamanti et al., (2005)). Since most of these previous studies exclusively tested right-handers, and since our data show that, on average, manual action verbs have a dominant hand bias, participants in these studies likely processed these verbs as describing actions that rely more strongly on the right than on the left hand. As a result, these participants were likely to show stronger activity in left hemisphere motor areas that control the right hand compared to right hemisphere motor systems that control the left hand, even for “bimanual” actions, thereby offering a possible explanation of the left-lateralized patterns of motor activation in these studies.

Do left-handers’ manual actions mirror right-handers’ actions?

Although we intended to collect data from right-handers alone, a few left-handers incidentally participated in our experiments. Given the small number of left-handed participants (Dutch: n = 3; English: n = 4), we did not have sufficient power to include them in our statistical analyses. Nonetheless, we performed an exploratory analysis that asked whether the distribution of left-handers’ pantomimes provided a mirror image of the distribution of the right-handers’ pantomimes (see supplementary Fig. 4). This analysis suggests that, compared to right-handers, the left-handers relied less on their dominant (i.e., left) hand and more on their non-dominant (i.e., right) hand when pantomiming manual actions, as indicated by the rightward shift in the peaks for both the unimanual and bimanual actions and by the long rightwards tails of the left-hander distributions.

Density plot of subject-wise average RHD, collapsed across languages. Blue = Left-handers; Red = Right-handers. Dark = Unimanual; Light = Bimanual

Full size image

This reduced dominant hand bias in left-handers matches the observations of previous studies (Willems et al., 2009) and, if it holds in a larger sample, it may offer a possible explanation for why left-handers show weaker patterns of lateralized neural activity when processing language describing unimanual actions than right-handers. If, compared to right-handers, left-handers rely more on their non-dominant hand for performing unimanual actions, then they should also show stronger activity in motor circuits controlling the non-dominant hand when processing language about those actions (see Willems et al., 2009, 2010a for a reduced neural asymmetry in left-handers; for similar findings in behavioral tasks, see: De Nooijer et al. (2013), Chrysikou et al. (2017), and Apel et al.(2012).

Why would left-handers show a weaker dominant hand bias for manual actions? On one possibility, left-handers may simply have a weaker hand preference than right-handers, overall: In several large-sample studies, strong left-handers are proportionally underrepresented compared to strong right-handers (Annett, 2004; Andersen and Siebner, 2018). On a second, non mutually-exclusive explanation, left-handers may have to use their non-dominant hand more often because they live in a right-handed world and therefore frequently have to interact with objects designed for right-handers (Chrysikou et al., 2017).

Regardless of why left-handers show a reduced dominant-hand bias, these preliminary data suggest that this bias may exist across a range of manual actions. Therefore, future work with a larger sample of left-handers may be needed to construct separate implicit norms of hand use for precisely characterizing how left-handers represent common actions in pantomime, and by inference in their minds.

The corpora as well as the data and code used to generate them are available for use and can be retrieved at https://osf.io/pd78w/?view_only=75f14cdfe2bb42c09541d712eb5246f7. The data can also be accessed in the supporting files. None of the experiments were pre-registered.