Top

Gepubliceerd in:

17-05-2018 | Original Article

Cross-modal decoupling in temporal attention between audition and touch

Auteurs: Stefanie Mühlberg, Salvador Soto-Faraco

Gepubliceerd in: Psychological Research | Uitgave 8/2019

Abstract

Temporal orienting leads to well-documented behavioural benefits for sensory events occurring at the anticipated moment. However, the consequences of temporal orienting in cross-modal contexts are still unclear. On the one hand, some studies using audio-tactile paradigms suggest that attentional orienting in time and modality are a closely coupled system, in which temporal orienting dominates modality orienting, similar to what happens in cross-modal spatial attention. On the other hand, recent findings using a visuo-tactile paradigm suggest that attentional orienting in time can unfold independently in each modality, leading to cross-modal decoupling. In the present study, we investigated if cross-modal decoupling in time can be extrapolated to audio-tactile contexts. If so, decoupling might represent a general property of cross-modal attention in time. To this end, we used a speeded discrimination task in which we manipulated the probability of target presentation in time and modality. In each trial, a manipulation of time-based expectancy was used to guide participants’ attention to task-relevant events, either tactile or auditory, at different points in time. In two experiments, we found that participants generally showed enhanced behavioural performance at the most likely onset time of each modality and no evidence for coupling. This pattern supports the hypothesis that cross-modal decoupling could be a general phenomenon in temporal orienting.

vorige artikel Interaction dynamics between grouping principles in touch: phenomenological and psychophysical evidence

volgende artikel Audiomotor integration of angry and happy prosodies

In addition to mean reaction times and accuracy, we analysed the obtained median reaction times and inverse efficiency. However, since the results for the median and mean reaction times were in accordance to each other, and since no trade-offs between reaction times and accuracy could be found, we decided to only report the mean reaction times, since they have also been previously reported in Mühlberg et al. (2014), as well as in Lange and Röder (2006).

In other words, like we have a high spatial resolution for vision and therefore visual dominance in cross-modal spatial paradigms (e.g. ventriloquist effect; Jackson, 1953; Slutsky & Recanzone, 2001; Spence, 2010; Welch & Warren, 1980), the resolution for audition in the temporal domain is higher than for vision or touch (Bresciani et al., 2005; Fendrich & Corballis, 2001; Morein-Zamir et al., 2003) and therefore audition would lead in temporal attention paradigms, dominating over all possible modulations in other modalities.

Responses towards expected stimuli elicit a weaker N100 component than the unexpected stimuli (Lange, 2009) and reduced fMRI activity (Kok et al., 2012). This reduced activity is thereby caused through a suppression of the activity of neurons encoding for unexpected events, leading to a sharpening of neural activity (Kok et al., 2012).

Aufschnaiter, S., Kiesel, A., & Thomaschke, R. (2017). Transfer of time-based task expectancy across different timing environments. Psychological Research Psychologische Forschung. https://doi.org/10.1007/s00426-017-0895-1.CrossRefPubMed

Ball, F., Michels, L. E., Thiele, C., & Noesselt, T. (2018). The role of multisensory interplay in enabling temporal expectations. Cognition, 170, 130–146. https://doi.org/10.1016/j.cognition.2017.09.015.CrossRefPubMed

Bauer, M., Kennett, S., & Driver, J. (2012). Attentional selection of location and modality in vision and touch modulates low-frequency activity in associated sensory cortices. Journal of Neurophysiology, 107(9), 2342–2351. https://doi.org/10.1152/jn.00973.2011.CrossRefPubMedPubMedCentral

Bendixen, A., SanMiguel, I., & Schröger, E. (2012). Early electrophysiological indicators for predictive processing in audition: A review. International Journal of Psychophysiology, 83(2), 120–131. https://doi.org/10.1016/j.ijpsycho.2011.08.003.CrossRefPubMed

Correa, A., Lupiáñez, J., Milliken, B., & Tudela, P. (2004). Endogenous temporal orienting of attention in detection and discrimination tasks. Perception & Psychophysics, 66(2), 264–278. https://doi.org/10.3758/BF03194878.CrossRef

Correa, A., Lupiáñez, J., & Tudela, P. (2005). Attentional preparation based on temporal expectancy modulates processing at the perceptual level. Psychonomic Bulletin & Review, 12(2), 328–334. https://doi.org/10.3758/BF03196380.CrossRef

Correa, A., & Nobre, A. C. (2008). Spatial and temporal acuity of visual perception can be enhanced selectively by attentional set. Experimental Brain Research, 189(3), 339–344. https://doi.org/10.1007/s00221-008-1429-2.CrossRefPubMed

Coull, J. T., & Nobre, A. C. (1998). Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. The Journal of Neuroscience, 18(18), 7426–7435. http://www.ncbi.nlm.nih.gov/pubmed/9736662.

Cravo, A. M., Rohenkohl, G., Wyart, V., & Nobre, A. C. (2013). Temporal expectation enhances contrast sensitivity by phase entrainment of low-frequency oscillations in visual cortex. The Journal of Neuroscience, 33(9), 4002–4010. https://doi.org/10.1523/JNEUROSCI.4675-12.2013.CrossRefPubMedPubMedCentral

Davranche, K., Nazarian, B., Vidal, F., & Coull, J. T. (2011). Orienting attention in time activates left intraparietal sulcus for both perceptual and motor task goals. Journal of Cognitive Neuroscience, 23(11), 3318–3330. https://doi.org/10.1162/jocn_a_00030.CrossRefPubMed

Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Reviews of Neuroscience, 18, 193–222. http://www.annualreviews.org/doi/pdf/10.1146/annurev.ne.18.030195.001205.

Driver, J., & Spence, C. (1994). Spatial synergies between auditory and visual attention. In C. Umiltà & M. Moscovitch (Eds.), Attention and performance XV: Conscious and nonconscious information processing (pp. 311–331). Cambridge: MIT Press.

Driver, J., & Spence, C. (1998a). Attention and the crossmodal construction of space. Trends in Cognitive Sciences, 2(7), 254–262. https://doi.org/10.1016/S1364-6613(98)01188-7.CrossRefPubMed

Driver, J., & Spence, C. (1998b). Cross-modal links in spatial attention. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 353(1373), 1319–1331. https://doi.org/10.1098/rstb.1998.0286.CrossRefPubMedPubMedCentral

Eimer, M. (1999). Can attention be directed to opposite locations in different modalities? An ERP study. Clinical Neurophysiology, 110(7), 1252–1259. http://www.ncbi.nlm.nih.gov/pubmed/10423190.

Eimer, M., & Driver, J. (2000). An event-related brain potential study of cross-modal links in spatial attention between vision and touch. Psychophysiology, 37(5), 697–705. http://www.ncbi.nlm.nih.gov/pubmed/11037046.

Eimer, M., van Velzen, J., & Driver, J. (2002). Cross-modal interactions between audition, touch, and vision in endogenous spatial attention: ERP evidence on preparatory states and sensory modulations. Journal of Cognitive Neuroscience, 14(2), 254–271. https://doi.org/10.1162/089892902317236885.CrossRefPubMed

Griffin, I. C., Miniussi, C., & Nobre, A. C. (2001). Orienting Attention in Time. Frontiers in Bioscience, 6, 660–671.CrossRef

Jensen, O., & Mazaheri, A. (2010). Shaping functional architecture by oscillatory alpha activity: gating by inhibition. Frontiers in Human Neuroscience, 4, 186. https://doi.org/10.3389/fnhum.2010.00186.CrossRefPubMedPubMedCentral

Keil, J., Pomper, U., & Feuerbach, N. (2017). Temporal orienting precedes intersensory attention and has opposing effects on early evoked brain activity. NeuroImage, 148, 230–239. https://doi.org/10.1016/j.neuroimage.2017.01.039.CrossRefPubMed

Keil, J., Pomper, U., & Senkowski, D. (2016). Distinct patterns of local oscillatory activity and functional connectivity underlie intersensory attention and temporal prediction. Cortex, 74, 277–288. https://doi.org/10.1016/j.cortex.2015.10.023.CrossRefPubMed

Kennett, S., Eimer, M., Spence, C., & Driver, J. (2001). Tactile-visual links in exogenous spatial attention under different postures: convergent evidence from psychophysics and ERPs. Journal of Cognitive Neuroscience, 13(4), 462–478. http://www.ncbi.nlm.nih.gov/pubmed/11388920.

Klein, R. M. (1977). Attention and visual dominance: A chronometric analysis. Journal of Experimental Psychology: Human Perception and Performance, 3(3), 365–378.PubMed

Kok, P., Jehee, J. F. M., & de Lange, F. P. (2012). Less is more: expectation sharpens representations in the primary visual cortex. Neuron, 75(2), 265–270. https://doi.org/10.1016/j.neuron.2012.04.034.CrossRefPubMed

Kok, P., Rahnev, D., Jehee, J. F. M., Lau, H. C., & de Lange, F. P. (2012). Attention reverses the effect of prediction in silencing sensory signals. Cerebral Cortex, 22(9), 2197–2206. https://doi.org/10.1093/cercor/bhr310.CrossRefPubMed

Kubovy, M. (1988). Should we resist the seductiveness of the space is to time as vision is to audition analogy? Journal of Experimental Psychology: Human Perception and Performance, 14(2), 318–320.

Lampar, A., & Lange, K. (2011). Effects of temporal trial-by-trial cuing on early and late stages of auditory processing: Evidence from event-related potentials. Attention, Perception & Psychophysics, 73, 1916–1933. https://doi.org/10.3758/s13414-011-0149-z.CrossRef

Lange, K. (2009). Brain correlates of early auditory processing are attenuated by expectations for time and pitch. Brain and Cognition, 69(1), 127–137. https://doi.org/10.1016/j.bandc.2008.06.004.CrossRefPubMed

Lange, K. (2012). The N1 effect of temporal attention is independent of sound location and intensity: implications for possible mechanisms of temporal attention. Psychophysiology, 49(11), 1468–1480. https://doi.org/10.1111/j.1469-8986.2012.01460.x.CrossRefPubMed

Lange, K. (2013). The ups and downs of temporal orienting: a review of auditory temporal orienting studies and a model associating the heterogeneous findings on the auditory N1 with opposite effects of attention and prediction. Frontiers in Human Neuroscience, 7(June), 263. https://doi.org/10.3389/fnhum.2013.00263.CrossRefPubMedPubMedCentral

Lange, K., Krämer, U. M., & Röder, B. (2006). Attending points in time and space. Experimental Brain Research, 173(1), 130–140. https://doi.org/10.1007/s00221-006-0372-3.CrossRefPubMed

Lange, K., & Röder, B. (2006). Orienting attention to points in time improves stimulus processing both within and across modalities. Journal of Cognitive Neuroscience, 18(5), 715–729. https://doi.org/10.1162/jocn.2006.18.5.715.CrossRefPubMed

Lange, K., Rösler, F., & Röder, B. (2003). Early processing stages are modulated when auditory stimuli are presented at an attended moment in time: an event-related potential study. Psychophysiology, 40(5), 806–817. http://www.ncbi.nlm.nih.gov/pubmed/14696734.

Li, C., Chen, K., Han, H., Chui, D., & Wu, J. (2012). An FMRI study of the neural systems involved in visually cued auditory top-down spatial and temporal attention. PloS One, 7(11), e49948. https://doi.org/10.1371/journal.pone.0049948.CrossRefPubMedPubMedCentral

Liu, Y., Bengson, J., Huang, H., Mangun, G. R., & Ding, M. (2014). Top-down Modulation of Neural Activity in Anticipatory Visual Attention: Control Mechanisms Revealed by Simultaneous EEG-fMRI. Cerebral Cortex (New York, N.Y.: 1991), (February), pp. 517–529. https://doi.org/10.1093/cercor/bhu204.

Lloyd, D. M., Merat, N., McGlone, F., & Spence, C. (2003). Crossmodal links between audition and touch in covert endogenous spatial attention. Perception & Psychophysics, 65(6), 901–924.CrossRef

Los, S. A., Knol, D. L., & Boers, R. M. (2001). The foreperiod effect revisited: Conditioning as a basis for nonspecific preparation. Acta Psychologica, 106(1–2), 121–145. https://doi.org/10.1016/S0001-6918(00)00029-9.CrossRefPubMed

Macaluso, E. (2010). Orienting of spatial attention and the interplay between the senses. Cortex, 46(3), 282–297. https://doi.org/10.1016/j.cortex.2009.05.010.CrossRefPubMed

Macaluso, E., & Driver, J. (2005). Multisensory spatial interactions: a window onto functional integration in the human brain. Trends in Neurosciences, 28(5), 264–271. https://doi.org/10.1016/j.tins.2005.03.008.CrossRefPubMed

Macaluso, E., Frith, C. D., & Driver, J. (2000). Modulation of Human Visual Cortex by Crossmodal Spatial Attention. Science, 289(5482), 1206–1208. https://doi.org/10.1126/science.289.5482.1206.CrossRefPubMed

Miniussi, C., Wilding, E. L., Coull, J. T., & Nobre, A. C. (1999). Orienting attention in time. Modulation of brain potentials. Brain, 122, 1507–1518. http://www.ncbi.nlm.nih.gov/pubmed/10430834.

Mühlberg, S., Oriolo, G., & Soto-Faraco, S. (2014). Cross-modal decoupling in temporal attention. European Journal of Neuroscience, 39, 2089–2097. https://doi.org/10.1111/ejn.12563.CrossRefPubMed

Nobre, A. C., & Rohenkohl, G. (2014). Time for the fourth dimension of attention. In A. C. Nobre & S. Kastner (Eds.), The Oxford Handbook of Attention (pp. 676–724). Oxford: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199675111.013.036.CrossRef

Pomper, U., Keil, J., Foxe, J. J., & Senkowski, D. (2015). Intersensory selective attention and temporal orienting operate in parallel and are instantiated in spatially distinct sensory and motor cortices. Human Brain Mapping, 3259, 3246–3259. https://doi.org/10.1002/hbm.22845.CrossRef

Posner, M. I., Nissen, M. J., & Klein, R. M. (1976). Visual dominance: An information-processing account of its origins and significance. Psychological Review, 83(2), 157–171.CrossRef

Rohenkohl, G., Cravo, A. M., Wyart, V., & Nobre, A. C. (2012). Temporal expectation improves the quality of sensory information. The Journal of Neuroscience, 32(24), 8424–8428. https://doi.org/10.1523/JNEUROSCI.0804-12.2012.CrossRefPubMedPubMedCentral

Ruzzoli, M., & Soto-Faraco, S. (2014). Alpha stimulation of the human parietal cortex attunes tactile perception to external space. Current Biology, 24(3), 329–332. https://doi.org/10.1016/j.cub.2013.12.029.CrossRefPubMed

Sambo, C. F., & Forster, B. (2011). When far is near: ERP correlates of crossmodal spatial interactions between tactile and mirror-reflected visual stimuli. Neuroscience Letters, 500(1), 10–15. https://doi.org/10.1016/j.neulet.2011.05.233.CrossRefPubMed

Sanders, L. D., & Astheimer, L. B. (2008). Temporally selective attention modulates early perceptual processing: Event-related potential evidence. Perception & Psychophysics, 70(4), 732–742. https://doi.org/10.3758/PP.70.4.732.CrossRef

Santangelo, V., Fagioli, S., & Macaluso, E. (2010). The costs of monitoring simultaneously two sensory modalities decrease when dividing attention in space. NeuroImage, 49(3), 2717–2727. https://doi.org/10.1016/j.neuroimage.2009.10.061.CrossRefPubMed

Schroeder, C. E., & Lakatos, P. (2009). Low-frequency neuronal oscillations as instruments of sensory selection. Trends in Neurosciences, 32(1), 9–18. https://doi.org/10.1016/j.tins.2008.09.012.CrossRefPubMed

Schröter, H., Birngruber, T., Bratzke, D., Miller, J., & Ulrich, R. (2015). Task predictability influences the variable foreperiod effect: evidence of task-specific temporal preparation. Psychological Research Psychologische Forschung, 79(2), 230–237. https://doi.org/10.1007/s00426-014-0550-z.CrossRefPubMed

Schwartze, M., Farrugia, N., & Kotz, S. A. (2013). Dissociation of formal and temporal predictability in early auditory evoked potentials. Neuropsychologia, 51(2), 320–325. https://doi.org/10.1016/j.neuropsychologia.2012.09.037.CrossRefPubMed

Soto-Faraco, S., Morein-Zamir, S., & Kingstone, A. (2005). On audiovisual spatial synergy: the fragility of the phenomenon. Perception & Psychophysicssychophysics, 67(3), 444–457. http://www.ncbi.nlm.nih.gov/pubmed/16119393.

Spence, C., & Driver, J. (1996). Audiovisual Links in Endogenous Covert Spatial Attention. Journal of Experimental Psychology: Human Perception and Performance, 22(4), 1005–1031.PubMed

Spence, C., & Driver, J. (1997). On measuring selective attention to an expected sensory modality. Perception & Psychophysics, 59(3), 389–403. http://www.ncbi.nlm.nih.gov/pubmed/9136269.

Spence, C., Nicholls, M. E. R., & Driver, J. (2001). The cost of expecting events in the wrong sensory modality. Perception & Psychophysics, 63(2), 330–336. http://www.ncbi.nlm.nih.gov/pubmed/11281107.

Spence, C., Pavani, F., & Driver, J. (2000). Crossmodal Links between Vision and Touch in Covert Endogenous Spatial Attention. Journal of Experimental Psychology: Human Perception and Performance, 26(4), 1298–1319.PubMed

Spitzer, H., Desimone, R., & Moran, J. (1988). Increased Attention Enhances Both Behavioral and Neuronal Performance. Science, 240(4850), 338–340.CrossRef

Störmer, V. S., McDonald, J. J., & Hillyard, S. a. (2009). Cross-modal cueing of attention alters appearance and early cortical processing of visual stimuli. Proceedings of the National Academy of Sciences of the United States of America, 106(52), 22456–22461. https://doi.org/10.1073/pnas.0907573106.CrossRefPubMedPubMedCentral

Summerfield, C., & Egner, T. (2009). Expectation (and attention) in visual cognition. Trends in Cognitive Sciences, 13(9), 403–409. https://doi.org/10.1016/j.tics.2009.06.003.CrossRefPubMed

Tang, X., Wu, J., & Shen, Y. (2016). The interactions of multisensory integration with endogenous and exogenous attention. Neuroscience & Biobehavioral Reviews, 61, 208–224. https://doi.org/10.1016/j.neubiorev.2015.11.002.CrossRef

Teder-Sälejärvi, W. A., Münte, T. F., Sperlich, F., & Hillyard, S. A. (1999). Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study. Brain Research. Cognitive Brain Research, 8(3), 327–343.CrossRef

ten Oever, S., Romei, V., van Atteveldt, N., Soto-Faraco, S., Murray, M. M., & Matusz, P. J. (2016). The COGs (context, object, and goals) in multisensory processing. Experimental Brain Research, 234(5), 1307–1323. https://doi.org/10.1007/s00221-016-4590-z.CrossRefPubMed

Thomaschke, R., & Dreisbach, G. (2015). The time-event correlation effect is due to temporal expectancy, not to partial transition costs. Journal of Experimental Psychology: Human Perception and Performance, 41(1), 196–218. https://doi.org/10.1037/a0038328.CrossRefPubMed

Thomaschke, R., Kunchulia, M., & Dreisbach, G. (2015). Time-based event expectations employ relative, not absolute, representations of time. Psychonomic Bulletin & Review, 22, 890–895. https://doi.org/10.3758/s13423-014-0710-6.CrossRef

Thomaschke, R., Wagener, A., Kiesel, A., & Hoffmann, J. (2011). The scope and precision of specific temporal expectancy: evidence from a variable foreperiod paradigm. Attention, Perception & Psychophysics, 73, 953–964. https://doi.org/10.3758/s13414-010-0079-1.CrossRef

Todorovic, A., Schoffelen, J. M., Van Ede, F., Maris, E., & De Lange, F. P. (2015). Temporal expectation and attention jointly modulate auditory oscillatory activity in the beta band. PLoS One, 10(3), 1–16. https://doi.org/10.1371/journal.pone.0120288.CrossRef

Trenner, M. U., Heekeren, H. R., Bauer, M., Rössner, K., Wenzel, R., Villringer, A., & Fahle, M. (2008). What happens in between? Human oscillatory brain activity related to crossmodal spatial cueing. PloS One, 3(1), e1467. https://doi.org/10.1371/journal.pone.0001467.CrossRefPubMedPubMedCentral

Treue, S. (2003). Visual attention: the where, what, how and why of saliency. Current Opinion in Neurobiology, 13(4), 428–432. https://doi.org/10.1016/S0959-4388(03)00105-3.CrossRefPubMed

van Ede, F., de Lange, F., Jensen, O., & Maris, E. (2011). Orienting attention to an upcoming tactile event involves a spatially and temporally specific modulation of sensorimotor alpha- and beta-band oscillations. Journal of Neuroscience, 31(6), 2016–2024. https://doi.org/10.1523/JNEUROSCI.5630-10.2011.CrossRefPubMed

VanRullen, R., Zoefel, B., & Ilhan, B. (2014). On the cyclic nature of perception in vision versus audition. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 369, 20130214. https://doi.org/10.1098/rstb.2013.0214.CrossRefPubMedPubMedCentral

Volberg, G., & Thomaschke, R. (2017). Time-based expectations entail preparatory motor activity. Cortex, 92, 261–270. https://doi.org/10.1016/j.cortex.2017.04.019.CrossRefPubMed

Wagener, A., & Hoffmann, J. (2010). Temporal cueing of target-identity and target-location. Experimental Psychology, 57(6), 436–445. https://doi.org/10.1027/1618-3169/a000054.CrossRefPubMed

Welch, R. B., & Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88(3), 638–687. https://doi.org/10.1037/0033-2909.88.3.638.CrossRefPubMed

Yang, Z., & Mayer, A. R. (2014). An event-related FMRI study of exogenous orienting across vision and audition. Human Brain Mapping, 35(3), 964–974. https://doi.org/10.1002/hbm.22227.CrossRefPubMed

Titel: Cross-modal decoupling in temporal attention between audition and touch
Auteurs: Stefanie Mühlberg
Salvador Soto-Faraco
Publicatiedatum: 17-05-2018
Uitgeverij: Springer Berlin Heidelberg
Gepubliceerd in: Psychological Research / Uitgave 8/2019
Print ISSN: 0340-0727
Elektronisch ISSN: 1430-2772
DOI: https://doi.org/10.1007/s00426-018-1023-6

Bohn Stafleu van Loghum

Deel dit onderdeel of sectie (kopieer de link)

Abstract

Log in om toegang te krijgen

Andere artikelen Uitgave 8/2019

Deep, effortless concentration: re-examining the flow concept and exploring relations with inattention, absorption, and personality

Spatial mental representations: the influence of age on route learning from maps and navigation

From reading numbers to seeing ratios: a benefit of icons for risk comprehension

Correction to: Gestalt grouping cues can improve filtering performance in visual working memory

Audiomotor integration of angry and happy prosodies

Olfactory perception and blindness: a systematic review and meta-analysis