Abstract
Studies of reactions to audiovisual spatial conflict (alias “ventriloquism”) are generally presented as informing on the processes of intermodal coordination. However, most of the literature has failed to isolate genuine perceptual effects from voluntary postperceptual adjustments. A new approach, based on psychophysical staircases, is applied to the case of the immediate visual bias of auditory localization. Subjects have to judge the apparent origin of stereophonically controlled sound bursts as left or right of a median reference line. Successive trials belong to one of two staircases, starting respectively at extreme left and right locations, and are moved progressively toward the median on the basis of the subjects’ responses. Response reversals occur for locations farther away from center when a central lamp is flashed in synchrony with the bursts than without flashes (Experiment 1), revealing an attraction of the sounds toward the flashes. The effect cannot originate in voluntary postperceptual decision, since the occurrence of response reversal implies that the subject is uncertain concerning the direction of the target sound. The attraction is contingent on sound-flash synchronization, for early response reversals did no longer occur when the inputs from the two modalities were desynchronized (Experiment 2). Taken together, the results show that the visual bias of auditory localization observed repeatedly in less controlled conditions is due partly at least to an automatic attraction of the apparent location of sound by spatially discordant but temporally correlated visual inputs.
Article PDF
References
Bermant, R. I., &Welch, R. B. (1976). The effect of the degree of visual-auditory stimulus separation upon the spatial interaction of vision and audition.Perceptual & Motor Skills,43, 487–493.
Bertelson, P. (1998). Starting from the ventriloquist: The perception of multimodal events. In M. Sabourin, F. I. M. Craik, & M. Robert (Eds.),Advances in psychological science: Vol. 1. Biological and cognitive aspects (pp. 419–439). Hove, U.K.: Psychology Press.
Bertelson, P., &Aschersleben, G. (1996). Visual bias of auditory location: Evidence for automaticity.Abstracts of the Psychonomic Society,1, 74.
Bertelson, P., &Radeau, M. (1976). Ventriloquism, sensory interaction, and response bias: Remarks on the paper by Choe, Welch, Gilford, and Juola.Perception & Psychophysics,19, 531–535.
Bertelson, P., &Radeau, M. (1981). Cross-modal bias and perceptual fusion with auditory-visual spatial discordance.Perception & Psychophysics,29, 578–584.
Bertelson, P., &Radeau, M. (1987).Adaptation to auditory-visual conflict: Have top-down influences been overestimated here also? Paper presented at the 2nd meeting of the European Society for Cognitive Psychology, Madrid.
Bertelson, P., Vroomen, J., Wiegeraad, G., &de Gelder, B. (1994). Exploring the relation between McGurk interference and ventriloquism. InInternational Congress on Spoken Language Processing 1994 (pp. 559–562). Yokohama: ICSLP.
Canon, L. K. (1970). Intermodality inconsistency of input and directed attention as determinants of the nature of adaptation.Journal of Experimental Psychology,84, 141–147.
Choe, C. S., Welch, R. B., Gilford, R. M., &Juola, J. F. (1975). The “ventriloquist effect”: Visual dominance or response bias?Perception & Psychophysics,18, 55–60.
Cornsweet, T. N. (1962). The staircase-method in psychophysics.American Journal of Psychology,75, 485–491.
Driver, J. (1996). Enhancement of listening by illusory mislocation of speech sounds due to lip-reading.Nature,381, 66–68.
Graziano, M. S. A., &Gross, C. G. (1996). Multiple pathways for processing visual space. In T. Inui & J. L. McClelland (Eds.)Attention and performance XVI: Information integration in perception and communication (pp. 181–207). Cambridge, MA: MIT Press.
Howard, I. P., &Templeton, W. B. (1966).Human spatial orientation. London: Wiley.
Jack, C. E., &Thurlow, W. R. (1973). Effects of degree of visual association and angle of displacement on the “ventriloquism” effect.Perceptual & Motor Skills,38, 967–979.
Jackson, C. V. (1953). Visual factors in auditory localization.Quarterly Journal of Experimental Psychology,5, 52–65.
Klemm, O. (1909). Localisation von Sinneneindrücken bei disparaten Nebenreizen.Psychologische Studien,5, 73–161.
Levitt, H. (1971). Transformed up-down methods in psychoacoustics.Journal of the Acoustical Society of America,49, 467–477.
Pick, H. L., Jr.,Warren, D. H., &Hay, J. C. (1969). Sensory conflict in judgments of spatial direction.Perception & Psychophysics,6, 203–205.
Predebon, J., &Woolley, J. S. (1994). The familiar-size cue to depth under reduced-cue viewing conditions.Perception,23, 1301–1312.
Radeau, M. (1985). Signal intensity, task context, and auditory-visual interaction.Perception,14, 571–578.
Radeau, M. (1992). Cognitive impenetrability in auditory-visual interaction. In J. Alegria, D. Holender, J. Morais, & M. Radeau (Eds.),Analytic approaches to human cognition (pp. 41–55). Amsterdam: Elsevier.
Radeau, M., &Bertelson, P. (1974). The after-effects of ventriloquism.Quarterly Journal of Experimental Psychology,26, 63–71.
Radeau, M., &Bertelson, P. (1976). The effect of a textured visual field on modality dominance in a ventriloquism situation.Perception & Psychophysics,20, 227–235.
Radeau, M., &Bertelson, P. (1977). Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations.Perception & Psychophysics,22, 137–146.
Radeau, M., &Bertelson, P. (1987). Auditory-visual interaction and the timing of inputs: Thomas (1941) revisited.Psychological Research,49, 17–22.
Stein, B. E., &Meredith, M. A. (1993).The merging of the senses. Cambridge, MA: MIT Press, Bradford Books.
Thomas, G. J. (1941). Experimental study of the influence of vision on sound localization.Journal of Experimental Psychology,28, 167–177.
Warren, D. H., Welch, R. B., &McCarthy, T. J. (1981). The role of visual-auditory “compellingness” in the ventriloquism effect: Implications for transitivity among the spatial senses.Perception & Psychophysics,30, 557–564.
Welch, R. B. (1978).Perceptual modification: Adapting to altered sensory environments. New York: Academic Press.
Witkin, H. A., Wapner, S., &Leventhal, T. (1952). Sound localization with conflicting visual and auditory cues.Journal of Experimental Psychology,43, 58–67.
Author information
Authors and Affiliations
Corresponding author
Additional information
The present work was presented at the XXVIth International Congress of Psychology, Montreal, 1996 (Bertelson, 1998), at the 1996 annual meeting of the Psychonomic Society in Chicago (Bertelson & Aschersleben, 1996), at the 50th Anniversary Meeting of the Belgian Psychological Society, and at the symposium on Varieties of Implicit Processing in Tilburg (The Netherlands), 1997. The contribution of the first author to the project was supported by the Belgian Fund for Collective Fundamental Research (FRFC), by the Ministry of Scientific Research of the Belgian French-Speaking Community (Concerted Research Actions 91/96-148 and 96/01-2037), and by Visiting Scientist Fellowships from the Max Planck Society. The computer programs were developed by Frank Miedreich, and Renate Tschakert helped with testing the subjects. Thanks are due Jim Cutting, Dom Massaro, Bob Welch, and an anonymous reviewer for their constructive examination of the manuscript. The work benefited also from discussions with Béatrice de Gelder, Jon Driver, Richard Held, Jacques Paillard, Wolfgang Prinz, and Jean Vroomen.
Rights and permissions
About this article
Cite this article
Bertelson, P., Aschersleben, G. Automatic visual bias of perceived auditory location. Psychonomic Bulletin & Review 5, 482–489 (1998). https://doi.org/10.3758/BF03208826
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03208826