Is 9 louder than 1? Audiovisual cross-modal interactions between number magnitude and judged sound loudness

Highlights

• The cross-modal influence of numbers on perceived loudness judgments was investigated.

• Sounds paired with large digits (8, 9) were reported as louder more frequently than sounds paired with small digits (1, 2).

• The effect was nullified if the number preceded the sound.

• The effect returned if participants were tasked with holding the number in short-term memory.

Abstract

The cross-modal impact of number magnitude (i.e. Arabic digits) on perceived sound loudness was examined. Participants compared a target sound's intensity level against a previously heard reference sound (which they judged as quieter or louder). Paired with each target sound was a task irrelevant Arabic digit that varied in magnitude, being either small (1, 2, 3) or large (7, 8, 9). The degree to which the sound and the digit were synchronized was manipulated, with the digit and sound occurring simultaneously in Experiment 1, and the digit preceding the sound in Experiment 2. Firstly, when target sounds and digits occurred simultaneously, sounds paired with large digits were categorized as loud more frequently than sounds paired with small digits. Secondly, when the events were separated, number magnitude ceased to bias sound intensity judgments. In Experiment 3, the events were still separated, however the participants held the number in short-term memory. In this instance the bias returned.

Introduction

The environment continually poses demands on our basic cognitive and sensory subsystems, in which information must be organized and integrated across modalities and dimensions to form coherent, representative precepts of the world in which we live. For example, our ability to process motion depends on the integration of information across multiple sensory streams (Soto-Faraco, Kingstone, & Spence, 2003); our sense of taste is influenced by color (Spence, Levitan, Shankar, & Zampini, 2010); and speech comprehension by visually processed mouth movements (McGurk and MacDonald, 1976, Munhall et al., 1996). While examples of cross-modal influences on perception are numerous, in all cases, transfer effects have demonstrated that information from one modality can be effectively used to make fast and efficient inferences about a related – but discrepant – stimulus dimension, presented in a different modality.

In the current study we were interested in how people might adaptively use information from visually presented symbolic numbers (i.e., Arabic digits) when judging the intensity of a sound. While sound intensity may appear largely unrelated to number, it should be considered that numbers — in the form of Arabic digits are typically associated with changes in sound intensity in our environment. Whether it would be on a volume knob attached to an amplifier, or the digital read-out on your computer's internal speakers — larger numbers are typically indicative of increased volume, and small numbers decreased volume. If it is the case that through experience we form a wide array of mental short-cuts to reduce cognitive load, and improve efficiency; the presence of numerical information under regular – unconstrained – situations, is likely to be useful when judging the intensity of a sound. Therefore, large numbers should elicit biases toward reporting greater sound intensity levels.

Another possibility is that interactions witnessed between symbolic numbers and sound intensity, may be the result of both dimensions sharing a common representational neural metric, within a generalized magnitude system (see A Theory of Magnitude, or ATOM) (Bueti and Walsh, 2009, Gallistel, 2011, Holmes and Lourenco, 2011, Walsh, 2003). Some evidence in favor of this perspective has implicated the intraparietal sulcus (IPS) as the locus for a generalized magnitude system, which is commonly activated when magnitude information is processed in a variety of formats, including non-symbolic (numerosities) and symbolic (Arabic numerals, number words) number forms (Eger et al., 2003, Piazza et al., 2004, Piazza et al., 2007). Based on these findings it has been proposed that the IPS hosts a generalized, notation independent representation of number (Dehaene, 2008; but see, Lyons, Ansari, & Beilock, 2015). Additionally, this metric is thought to subserve the representations of a wide array of other continuous and discrete magnitude dimensions which include (but are not limited to) the dimensions of time and space (Heinemann et al., 2013, Vierck and Kiesel, 2010).

To understand how this abstract magnitude code works, we must first understand that magnitude appears to be coded by neurons with monotonic rate-intensity output functions (i.e., they exhibit spiking rates that increase with stimulus intensity). The outputs of these neurons are thought to feedforward onto neurons which are tuned to respond to specific distal values (e.g., numerosity detectors) (Dehaene, 2008, Nieder et al., 2002, Verguts and Fias, 2004). The tuning curves of the neurons that preferentially respond to a specific value flatten as the distal value is increased, resulting in reduced perceptual sensitivity at higher intensity levels (Allman et al., 2011, Cordes et al., 2001, Dehaene, 2003, Dehaene, 2008). One aftereffect of this organization is that one's ability to detect a perceptible disparity between two stimuli (the just noticeable difference) worsens as the intensity levels of the compared stimuli are increased (see Weber's law) (Dehaene, 2003). In support of ATOM, a wide-array of perceptual dimensions have been found to conform to Weber's law including: duration (Gibbon et al., 1984, Meck and Church, 1983), non-verbal numbers (Cordes et al., 2001, Gallistel and Gelman, 1992, Gallistel and Gelman, 2000, Whalen et al., 1999), symbolic number (Dehaene et al., 1998, Moyer and Landauer, 1967, Moyer and Landauer, 1973), and sound loudness (Knudsen, 1923, Miller, 1947, Riesz, 1928), suggesting that they may all be organized within a common representational framework.

In further support of this perspective, a variety of cross-dimensional transfer have been found between symbolic numbers and other dimensions. For example, the magnitudes of task-irrelevant symbolic numbers impact judgments about physical size and length (De Hevia, Girelli, Bricolo, & Vallar, 2008; A Henik and Tzelgov, 1982, Viarouge and de Hevia, 2013), numerosity (Naparstek & Henik, 2010), duration (Alards-Tomalin et al., 2014, Kiesel and Vierck, 2009, Oliveri et al., 2008, Vicario et al., 2008, Xuan et al., 2009, Xuan et al., 2007), and luminance (i.e., brightness) (Cohen Kadosh et al., 2008, Cohen Kadosh, Henik and Walsh, 2007; but see Pinel et al., 2004). Furthermore, they also interfere with basic spatial–motor response selection, including the performance speed of left vs. right handed responses (Dehaene et al., 1993, Nuerk et al., 2005), precision motor responses (pinch vs. whole hand grasps) responses, and grip aperture (Andres et al., 2008, Lindemann et al., 2007). Therefore, proponents of the ATOM framework, might suggest that cross-dimensional biases elicited from numbers on sound intensity judgments would demonstrate further evidence of an abstract representational framework for magnitude.

Another interpretation that may be able to account for interactions between numerical magnitude and sound amplitude comes from the heuristics and biases approach to cognition. Numerical anchoring is a cognitive phenomenon wherein task irrelevant numbers are used as referents – or starting points – for making various decisions. Assimilative anchoring is said to have occurred when an estimate is pulled in the direction of the number's magnitude. For example, when asked to estimate the number of African countries in the United Nations, participants provided greater estimates if the anchor value was initially a larger number (e.g., higher/lower than 100) versus a smaller number (e.g., higher/lower than 10) (Tversky & Kahneman, 1974). Interestingly, this occurs even when it is obvious that the anchor is unrelated to the target task. For example, numerical anchors generated through obviously random events, like a wheel-of-fortune spin (Chapman and Johnson, 1999, Tversky and Kahneman, 1974), or that are entirely incidental, like the numbers of one's social insurance number (Ariely, Loewenstein, & Prelec, 2003), or an athlete's jersey number (Critcher & Gilovich, 2008), go on to influence a variety of judgments (e.g., product valuations, judged athletic performance). In all cases, attending to large numbers facilitates a higher overall estimate. Furthermore, all that is generally required to elicit anchoring, is that sufficient attention is paid to the number. For example, participants that first made an unrelated magnitude judgment about a number (e.g., judging an ID number as lower/higher than 1920), prior to making an estimation judgment (e.g., estimate the number of physicians in the phonebook) were influenced by the numerical magnitude of the anchor despite not directly comparing their estimate against it (Wilson, Houston, Etling, & Brekke, 1996).

Researchers have now suggested that assimilative numerical anchoring is driven by people relying on the absolute value of numerical information held in short-term memory – regardless of its source – as a relevant launching point for making a wide variety of magnitude estimates (Kahneman and Knetsch, 1993, Wilson et al., 1996, Wong and Kwong, 2000). Furthermore, the Anchoring as Activation approach posits that the mere presence of the anchor will facilitate the activation of features that are held in common when the anchor,; as noted by Chapman and Johnson (1999) “…anchors have their effect because decision makers consider reasons why their value for the target item is like the anchor, but show relative neglect for reasons why their value for the item is unlike the anchor” (p. 121). We therefore propose that when participants have sufficient reason to attend to, and process the magnitude of a number presented prior to the target sound (e.g., the number and sound occur simultaneously [Experiment 1], or the participant is required to hold the number in short-term memory [Experiment 3]), then the number will function as an anchor and bias judgments in the same direction as the numerical value. However, when participants have no reason to attend to the digit (e.g., it occurs prior to the sound's presentation [Experiment 2]), people will actively discount or ignore it, thus reducing/eliminating any anchoring effects. Furthermore, the fact that the target task is perceptual in nature (judging sound intensity) is largely irrelevant, as anchors have been found to bias a wide variety of judgments ranging from estimating weights, to general/factual knowledge estimates (e.g., estimate length of Mississippi river), probability estimates, legal judgments (e.g., length of a prison sentence), purchasing decisions, and self-efficacy assessments (for a recent review see, Furnham & Boo, 2011).

Interestingly, while interference effects between numbers and other visual dimensions have been widely demonstrated, fewer studies have examined the presence of these kinds of interactions using a cross-modal paradigm, and to our knowledge, no studies have been published to date on whether sound intensity judgments are influenced by visually presented numbers. In one recent, noteworthy study it was found that participants tended to spontaneously generate a higher proportion of large magnitude numbers when listening to high intensity versus low intensity sounds (Heinemann et al., 2013). In the current study we examined the opposite interaction, whether or not visual numbers elicited biases on the perceived intensity of an otherwise unrelated sound. We predicted that visual magnitude information (in the form of symbolic numbers) would exert cross-modal biases on a basic sound intensity judgment task, in a manner consistent with anchoring, despite being task irrelevant. Furthermore, we have attempted to contrast the generalized magnitude framework against the numerical anchoring account.