Crossmodal interaction in saccadic reaction time: separating multisensory from warning effects in the time window of integration model

Abstract

In a focused attention task saccadic reaction time (SRT) to a visual target stimulus (LED) was measured with an auditory (white noise burst) or tactile (vibration applied to palm) non-target presented in ipsi- or contralateral position to the target. Crossmodal facilitation of SRT was observed under all configurations and stimulus onset asynchrony (SOA) values ranging from  −500 (non-target prior to target) to 0 ms, but the effect was larger for ipsi- than for contralateral presentation within an SOA range from  −200 ms to 0. The time-window-of-integration (TWIN) model (Colonius and Diederich in J Cogn Neurosci 16:1000, 2004) is extended here to separate the effect of a spatially unspecific warning effect of the non-target from a spatially specific and genuine multisensory integration effect.

Appendices

Appendix 1

Here we derive the probability of the events I (multisensory integration) and W (warning) under the exponential distribution assumption. Specifically, the probability distributions for peripheral processing times V for a visual target and A for an auditory non-target are, respectively,

$$ \begin{aligned} f_{\rm V}(t) =\, & \lambda_{\rm V} {\rm e}^{-\lambda_{\rm V} t},\\ f_A(t) =\,& \lambda_A {\rm e}^{-\lambda_A t}\\ \end{aligned} $$

for t ≥ 0, and f _V(t) = f _A (t) ≡ 0 for t < 0. The corresponding distribution functions are referred to by F _V(t) and F _A (t).

Probability of integration: P(I)

By definition,

$$ \begin{aligned} P(I) = & Pr(A+\tau < V < \,A+\tau+\omega) \\ = & \int\limits_0^\infty f_A(a)\{F_{\rm V}(a+\tau + \omega)-F_{\rm V}(a+\tau)\} {\rm d}a, \\ \end{aligned} $$

where τ denotes the SOA value and ω is the width of the integration window. Computing the integral expression requires that we distinguish between three cases for the sign of τ + ω:

• (a) τ< τ + ω < 0

  $$ \begin{aligned} P(I) = & \int\limits_{-\tau - \omega}^{-\tau} \lambda_A {\rm e}^{-\lambda_A a} \{1-{\rm e}^{-\lambda_{\rm V}(a+\tau+\omega)}\} {\rm d}a \\ & + \int\limits_{-\tau}^\infty \lambda_A {\rm e}^{-\lambda_A a}\{{\rm e}^{-\lambda_{\rm V}(a+\tau)}-{\rm e}^{-\lambda_{\rm V}(a+\tau+\omega)}\} {\rm d}a \\ = & \frac{\lambda_{\rm V}}{\lambda_{\rm V}+\lambda_A}\; {\rm e}^{\lambda_A \tau}(-1+{\rm e}^{\lambda_A \omega}); \end{aligned} $$

• (b) τ < 0 < τ + ω

  $$ \begin{aligned} P(I) = & \int\limits_0^{-\tau} \lambda_A {\rm e}^{-\lambda_A a} \{1-{\rm e}^{-\lambda_{\rm V}(a+\tau+\omega)}\} da \\ & + \int\limits_{-\tau}^\infty \lambda_A {\rm e}^{-\lambda_A a}\{{\rm e}^{-\lambda_{\rm V}(a+\tau)}-{\rm e}^{-\lambda_{\rm V}(a+\tau+\omega)}\} da \\ = & \frac{1}{\lambda_{\rm V}+\lambda_A}\; \{\lambda_A(1-{\rm e}^{-\lambda_{\rm V}(\omega+\tau)})+\lambda_{\rm V}(1-{\rm e}^{\lambda_A \tau})\}; \end{aligned} $$

• (c) 0 < τ < τ + ω

  $$ \begin{aligned} P(I) = & \int\limits_0^\infty \lambda_A {\rm e}^{-\lambda_A a}\{{\rm e}^{-\lambda_{\rm V}(a+\tau)}-{\rm e}^{-\lambda_{\rm V}(a+\tau+\omega)}\} {\rm d}a \\ & = \frac{\lambda_A}{\lambda_{\rm V}+\lambda_A}\; \{{\rm e}^{-\lambda_{\rm V}\tau}-{\rm e}^{-\lambda_{\rm V} (\omega+\tau)}\}. \end{aligned} $$

Probability of warning: P(W)

By definition,

$$ \begin{aligned} P(W) = & {\rm Pr}(A+\tau + \gamma_A < V) \\ = & \int\limits_0^\infty f_A(a)\{1-F_{\rm V}(a+\tau + \gamma_A)\} {\rm d}a \\ & = 1-\int\limits_0^\infty f_A(a)F_{\rm V}(a+\tau + \gamma_A) {\rm d}a. \end{aligned} $$

Again, we need to consider different cases:

• (i) τ + γ _A  < 0

  $$ \begin{aligned} P(W) = & 1- \int\limits_{-\tau-\gamma_A}^\infty \lambda_A {\rm e}^{-\lambda_A a}\{1-{\rm e}^{-\lambda_{\rm V}(a+\tau+\gamma_A)}\} {\rm d}a \\ = & 1- \frac{\lambda_{\rm V}}{\lambda_{\rm V}+\lambda_A}\; {\rm e}^{\lambda_A(\tau + \gamma_A)}; \end{aligned} $$

• (ii) τ + γ _A  ≥ 0

  $$ \begin{aligned} P(W) = & 1- \int\limits_{0}^\infty \lambda_A {\rm e}^{-\lambda_A a}\{1-{\rm e}^{-\lambda_{\rm V}(a+\tau+\gamma_A)}\} {\rm d}a \\ & = \frac{\lambda_A}{\lambda_{\rm V}+\lambda_A}\; {\rm e}^{-\lambda_{\rm V}(\tau + \gamma_A)}. \end{aligned} $$

Appendix 2

Tables 5, 6, 7, 8, 9 and 10 for mean SRTs under all uni- and bimodal conditions are listed, by participant.

Table 5 Mean SRT to unimodal visual stimuli, averages across all experimental conditions

Table 6 Participant P1: Mean SRT to bimodal stimuli with auditory non-target (left four columns) and tactile non-target (right four columns) presented ipsi- or contralaterally in a blocked or mixed trial condition

Table 7 Participant P2: Mean SRT to bimodal stimuli with auditory non-target (left four columns) and tactile non-target (right four columns) presented ipsi- or contralaterally in a blocked or mixed trial condition

Table 8 Participant P3: Mean SRT to bimodal stimuli with auditory non-target (left four columns) and tactile non-target (right four columns) presented ipsi- or contralaterally in a blocked or mixed trial condition

Table 9 Participant P4: Mean SRT to bimodal stimuli with auditory non-target (left four columns) and tactile non-target (right four columns) presented ipsi- or contralaterally in a blocked or mixed trial condition

Table 10 Participant P5: Mean SRT to bimodal stimuli with auditory non-target (left four columns) and tactile non-target (right four columns) presented ipsi- or contralaterally in a blocked or mixed trial condition