Introduction

Advanced mammals such as humans have multiple internal processing steps in between perception and action that allow for flexible response implementation and execution (Hommel, Müsseler, Aschersleben, & Prinz, 2001; Mesulam, 1998). This capacity comes at a price, however, as we have an externally perceived and internally represented environment to monitor at the same time. Indeed, at any single waking moment, there is an immense amount of information available in both environments for potential further processing. As our processing capacities are limited (Lennie, 2003; Marois & Ivanoff, 2005), we need an efficient attentional mechanism to select the bits of information that are goal-related or salient within each environment. Moreover, this capacity limit implies that we cannot easily attend both environments simultaneously and competition between them thus inevitably prevails for information processing. Consequently, a prerequisite for harmonizing these two environments is the ability to flexibly switch attention between external and internal information. These observations raise the interesting question of how information processing is affected by these kinds of attentional switches, and how control may be implemented therein (Burgess, Dumontheil, & Gilbert, 2007; Myers, Stokes, & Nobre, 2017). More precisely, is a cost associated with them and, relatedly, is a single overarching control mechanism eventually involved. The present review and empirical work deriving from it (see second section below) sought to fill this gap. To ease readability, we hereafter use the terms between-domain and within-domain switch cost. Whereas the former refers to switches between internal (working memory; WM) and external attention, the latter corresponds to alternations between two tasks or representations sharing the same modality (e.g., external environment).

To address these questions, we first present the literature on task-switching as an initial framework for attentional switching. Therein, the switch cost found across a variety of experimental paradigms is often attributed to the time required for a serial control mechanism needed to reallocate a limited resource between conflicting task contexts (Liefooghe, Barrouillet, Vandierendonck, & Camos, 2008; Monsell, 2003). Then, pointing to a gap in the existing literature, we observe that there currently exists no framework or experimental paradigm suited to investigate the cost of switching attention between external and internal information, although this process is ubiquitous in daily life. In order to explore how switching between external stimuli and representations in WM likely works, we first discuss the mechanisms responsible for information selection in each environment, i.e., external and internal attention, and their relationship. We then propose that, similarly to standard task-switching, control is likely implemented serially (Burgess et al., 2007), which implies that a processing cost should emerge in case attention shifts between internal and external information. In addition, we investigate whether a single, limited resource is at play (e.g., Kiyonaga & Egner, 2013), or, alternatively, whether external and internal attention rely on partly independent mechanisms (e.g., Tas, Luck, & Hollingworth, 2016). These two models lead to the opposing predictions that either a similar cognitive cost will occur for switching between tasks requiring external and internal attention (resource sharing account), or that a reduced cost will occur (independent processes account), respectively. To test these opposing predictions, we present the results of four behavioral experiments based on a novel paradigm, which provide evidence for the resource sharing hypothesis. Finally, we discuss these findings against the notion of a unitary attention control mechanism that enables rapid shifts between internal and external information.

Attentional flexibility in task- and object-switching

Attention can be focused on external information available through the senses and on information that is internally maintained or generated (Chun, Golomb, & Turk-Browne, 2011). For example, when on an autumn day we are looking for a specific color shade among differently colored leaves in a tree, we are focusing our attention on features available in the external environment. Similarly, when retrieving a memory about a specific hike in the woods on a past holiday, we are focusing our attention on the internal environment (i.e., retrieval and activation of episodic memory traces). Attentional shifts within each environment occur when we switch attention from one color shade to another, or from one memory to another, for instance. A classical research line in cognitive psychology investigating the nature and extent of switches of attention between competing representations is the one employing task-switching paradigms (Monsell, 2003), which require participants to flexibly switch between different task-sets. In these paradigms, the stimuli – often bivalent – are presented externally to the participants, who have to respond differently depending on the current task context, which is typically conveyed by means of a specific cue. Its long history and popularity have resulted in a myriad of flavors in task-switching paradigms that can broadly be subdivided into different categories based on how the switches are actually organized (see Monsell, 2003 for an overview), for example, the list procedure (e.g., Allport et al., 1994; Jersild, 1927; Spector & Biederman, 1976), alternating runs (e.g., Rogers & Monsell, 1995), cueing (e.g., Meiran, 1996; Sudevan & Taylor, 1987), and voluntary switching (Arrington & Logan, 2004) paradigms. Each of these has its own advantages and disadvantages, and often task-parameters have been discovered unique to a specific experimental design (see Vandierendonck, Liefooghe, & Verbruggen, 2010). Most importantly, a consistent finding across these task-switching paradigms is that participants are slower and more error-prone on switch compared to repeat trials, which is known as the switch cost (or repetition benefit). This cost is presumably caused by the competition between different activated task-sets. More specifically, prominent theoretical frameworks in the existing literature explain this switch cost in terms of task-set reconfiguration (Mayr & Kliegl, 2000, 2003; Meiran, 1996; Monsell & Mizon, 2006), interference caused by previous task context (e.g., Allport et al., 1994; Waszak, Hommel, & Allport, 2003), or a combination of the twoh (e.g., Rubinstein, Meyer, & Evans, 2001; Vandierendonck et al., 2010).

Other researchers have wondered about comparable switch costs, but when utilizing mostly internally maintained objects or sets (Chambers, Lo, & Allen, 2008; De Lissnyder et al., 2012; Garavan, 1998; Janczyk, Wienrich, & Kunde, 2008). In these paradigms, participants have to keep two mental counters in WM and update them by counting specific categories of figures on the screen. Sometimes two successive trials show figures from the same category (repeat trials) and sometimes a trial with figures from a different category follows (switch trials). These experiments revealed that there are switch costs when participants have to switch their internal focus of attention between different mental counters, called a (mental) object switch cost. Conceptually, there are strong similarities with the externally-driven task-switching costs reviewed above, as both result from competition between mental representations held in WM. In standard task-switching, this competition occurs among different task-sets, whereas in mental object switching, it occurs among different mental counters. The main difference among them, however, is that conflict in the task-switching is externally generated, with the same stimuli activating different response rules that interfere with one another, whereas the conflict in mental-set shifting is primarily internally generated, as the different mental counters or sets interfere with each other. In both cases, attention needs to be divided between storage and control (Logan, 1980).

These two seemingly parallel research lines have been very fruitful in exploring cognitive processes responsible for cognitive control, attention flexibility, and, more generally, executive functions (Friedman & Miyake, 2004; Miyake et al., 2000). Nonetheless, one limitation of this literature is its almost exclusive focus and reliance on attentional processing within the same – either external or internal – environment. In daily life, however, it often occurs that we shift attention from information available in the external environment to information processed in the internal environment (i.e., WM), and vice versa. To return to our previous example, when we are looking at a specific tree and this brings up the memory of the wood in which we were walking during our past holiday, a shift between attentional domains has necessarily occurred. This kind of shift occurs regularly and is indispensable for adaptive interaction with our environment (Buckner, Andrews-Hanna, & Schacter, 2008; Carlson, Wenger, & Sullivan, 1993; Dark, 1990; Hutchison et al., 2013; Weber, Burt, & Noll, 1986). However, in the existing literature, there are surprisingly few studies investigating this kind of attentional shift. A notable exception is a group of studies performed earlier (Carlson et al., 1993; Dark, 1990; Weber et al., 1986), in which participants were required, in a list-procedure paradigm, to switch between certain perceptual or WM stimuli. These studies, however, did not reach a clear consensus on the magnitude of the switch cost, and on whether it was the same for switches from memory to perception and vice versa (Weber et al., 1986), or whether the latter was larger than the former (Dark, 1990). In addition, although a list-procedure paradigm with verbal responses is probably a useful approach at the methodological level to start exploring this complex question, it nonetheless lacks precision and is confounded with preparation effects, undermining in turn the conclusions in terms of attention switching that can be drawn based on it.

From a cognitive-neuroscience perspective, there has also been some interest in brain mechanisms responsible for shifts between stimulus-oriented and -independent cognition (Burgess et al., 2007; Gilbert, Frith, & Burgess, 2005), which can be roughly equated with shifts between perceptual attention and WM. In these paradigms, participants were required to carry out a task for a certain period with external stimuli, followed by another period during which they had to continue without them being physically presented (Gilbert et al., 2005). Whereas this research line has been valuable in unveiling the neural mechanisms responsible for these shifts, a limitation is that the experimental design used therein cannot be used easily to investigate processing costs of shifts. Thus, there currently exists, to the best of our knowledge, no explicit theoretical framework or adequate experimental paradigm that will allow investigating shifts of attention between external and internal information (see also Burgess et al., 2007; Myers et al., 2017, for similar conclusions). Consequently, improvement on the empirical level is probably desirable when specific conditions are considered and met at the methodological level, as we later outline (see section entitled Conditions to explore cross-domain shifts of attention at the behavioral level below).

In the following, we first discuss the cognitive processes that are envisioned to play a crucial role in these between-domain switches. The restrictions imposed by these processes will guide us in devising a new experimental paradigm that is suited to explore them at the behavioral level. To this end, the possible overlap and difference between regular task switching and shifting attention between domains provide a valuable framework for generating predictions and interpretations. As we argue later, even though the target of the switch is actually different in both cases (within- vs. between-domain), their corresponding architectures likely share similarities (see also Hazy, Frank, & O’Reilly, 2006).

A framework for between-domain switching

Different theoretical frameworks have already been put forward in the past to explain switch costs (see Vandierendonck et al., 2010). Most of these models, which are not mutually exclusive, emphasize the pivotal role of attentional control (Mayr & Kliegl, 2000, 2003; Meiran, 2000; Monsell & Mizon, 2006; Rubinstein et al., 2001; Vandierendonck et al., 2010; Waszak, Hommel, & Allport, 2004). Attentional control is necessary to cope with the central processing bottleneck efficiently (Pashler, 1984), which precludes processing multiple tasks simultaneously (Garavan, 1998; Liefooghe et al., 2008; Oberauer, 2009). An important model on WM capacity that is befitting for explaining switch costs assumes that there is time-based resource-sharing between processing and storage, which necessitates frequent switches of attention between them (Barrouillet, Bernardin, & Camos, 2004). From this we can infer that switch costs correspond to cognitive costs that occur as a consequence of a limited attentional resource being involved in other processes taking place in short temporal proximity (see also Liefooghe et al., 2008). This framework is compatible with theories explaining switch cost in terms of reconfiguration and with those emphasizing the role of stimulus processing, as both require attentional allocation, which is a time-consuming process. From this perspective, asking the question whether similar switch costs occur when switching between attentional domains boils down to the question whether, firstly, serial attentional control operates in this case and, secondly, whether attending information in one domain impedes attending information in the other one. To answer these questions, we first need to investigate how stimuli are generally attended and processed in each domain separately, and how these processes may relate to each other. In other words, apart from the role of attentional control acting periodically as a supervisory system, we also have to take into account the likely differences between external and internal attention, and then specify how they likely interact with each other.

In the existing literature, when describing the mechanism responsible for selecting externally perceived sensory information, one generally speaks of perceptual or selective attention.Footnote 1 Before being processed, this kind of information is modality-specific, proximal, and directly related to the external environment and the sensor through which it is conveyed. At present, we have a relatively fine-grained understanding of how selective attention biases information-processing, especially in the visual modality (e.g., Carrasco, 2011; Desimone & Duncan, 1995; Driver, 2001). Information that is attended receives additional processing resources by means of putative gain-control mechanisms, whereas unattended information is ignored or even suppressed. As is the case in the external domain, in the internal domain, selecting the currently most relevant or salient representation from among other activated internal representations is necessary as well, and of utmost importance to foster goal-adaptive behavior. The internal domain is a heuristic container term and refers to any form of internally generated information, such as inner thoughts, cognitive control processes, representations in WM, long-term memory (LTM), task rules, decisions, and responses (Chun et al., 2011). The main mechanism responsible for selection and processing in the internal domain is thought to be WM. Whereas in the traditional models WM is based on a specific storage system that is independent from perceptual processes (e.g., Baddeley, 2012; Baddeley & Hitch, 1974), more recent models state that sensory representations are maintained in the absence of sensory stimulation by biasing attention towards this specific information, with corresponding changes visible in brain regions responsible for processing it, for instance (e.g., Christophel, Klink, Spitzer, Roelfsema, & Haynes, 2017). Whereas this ongoing debate goes beyond the scope of the current paper (see, e.g., Bettencourt & Xu, 2015; Harrison & Bays, 2018; Serences, 2016; Xu, 2017), they are, crucially, both compatible with the notion that accessing information in WM is intrinsically related to internal attending (see also Souza & Oberauer, 2016, for a review on retro-cuing), a notion that we directly borrow here. Here we first expand on the likely control process at stake to switch attention between external and internal information in order to determine whether a switch cost will occur in this situation or not.

Control process underlying between-domain switching

When considering control mechanisms, prior research has mainly focused on the extent to which they are similar between perception and WM, especially in the neuroimaging literature (Esterman, Chiu, Tamber-rosenau, & Yantis, 2009; Tamber-Rosenau, Esterman, Chiu, & Yantis, 2011). Whereas univariate analyses of neuroimaging data initially indicated overlapping activations for control in both domains, more advanced analyses using multi-voxel pattern analyses (MVPA) revealed non-overlapping control effects (Esterman et al., 2009; Tamber-Rosenau et al., 2011). However, these valuable and recent research efforts mostly focused on comparing control mechanisms between the two domains, but did not investigate it at an overarching level, i.e., when it is used to switch between them. In this respect, it should be noted that there exist general models proposing a serial, unified control mechanism in cognitive processing that are potentially relevant for the matter under scrutiny here (Tamber-Rosenau, Dux, Tombu, Asplund, & Marois, 2013; Tamber-Rosenau & Marois, 2016). Moreover, earlier studies conducted by Burgess and colleagues (Burgess et al., 2007; Gilbert et al., 2005) have already addressed this question indirectly. These authors assumed the existence of central representations, which can be activated by both internal and external attention. In this framework, switching between internal and external attention, which are in competition for access to these central representations, is controlled by a supervisory attentional gateway. Such a mechanism is in line with serial, overarching control for between-domain switches. In sum, whereas differences in control mechanisms potentially exist within each domain, it is likely that a serial, overarching control mechanism is eventually responsible for switches between internal and external attention. From this, we can predict that a cost should emerge when switching attention between these two domains. However, in order to compare this cost to the within-domain switch cost directly, we additionally have to determine whether resource sharing, or alternatively independent processing, prevails.

Resource sharing in external and internal attention

When presenting current models of WM, we have seen that, functionally, it can be seen as a mechanism that partly mirrors in the internal domain selective attention operating in the external domain (see Gazzaley & Nobre, 2012; Kiyonaga & Egner, 2013; Postle, 2006). One factor that has contributed strongly to the popularity of this proposal is the use and systematic comparison of standard- versus retro-cueing to explore covert shifts of attention on physical (external) stimuli and internal representations held in WM, respectively (for a review, see Souza & Oberauer, 2016). The focus of attention is a functional state of WM that selects and elevates a single item to a separate representational state, making it more robust and easier to access for behavioral purposes (Myers et al., 2017). Retro-cuing is a way to access the focus of attention as it prioritizes an item that is maintained in WM (Griffin & Nobre, 2003; Landman, Spekreijse, & Lamme, 2003). In these paradigms, in contrast to regular cueing paradigms (e.g. Posner, 1980), the (informational) cue is shown after a display of visual stimuli during the retention interval. This results in faster responses and less errors for the prioritized item during the actual retrieval. In effect, what retro-cuing research reveals is that internal attention has similar features to external attention, because it can be voluntarily shifted towards a cued item, which then receives a privileged state among other (distracting) items (Souza & Oberauer, 2016).

Although there is a wide consensus on the functional similarity between WM and perceptual attention, there is currently an ongoing debate in the cognitive science literature regarding the nature and extent of overlap or multiplexing between them in terms of psychological processes and neural networks (Chun, 2011; Gazzaley & Nobre, 2012; Kiyonaga & Egner, 2013; Tas, Luck, & Hollingworth, 2016). On the one hand, some models tend to equate the attentional mechanism for the external and internal domains. In this framework, WM is a recycling of the cognitive and neural machinery of attention, mostly oriented towards internal representations (e.g., Awh & Jonides, 2001; Gazzaley & Nobre, 2012; Kiyonaga & Egner, 2013; Postle, 2006). For example, Kiyonaga and Egner (2013) argued that both perceptual attention and WM share a limited pool of resources and that interference occurs when both simultaneously draw on it. On the other hand, other models assume instead that perceptual attention and WM are partly independent from one another, and hence can be dissociated in some cases (Baizer, Ungerleider, & Desimone, 1991; Hollingworth & Henderson, 2002; Tas et al., 2016; Woodman, Vogel, & Luck, 2001). In addition, it is important to clearly define WM and attention when investigating their potential similarities and overlap. Tas et al. (2016) emphasized the many varieties of attention, a caveat that can be applied in both the external and internal domains, for example feature-based attention versus spatial attention (Bichot, Rossi, & Desimone, 2005; Zhou & Desimone, 2011), WM for object and surface features and spatial WM (Tresch, Sinnamon, & Seamon, 1993), or the dissociable brain systems in the prefrontal cortex (PFC) for verbal and visuospatial WM (Levy & Goldman-Rakic, 2000; Miller, Erickson, & Desimone, 1996; Miller & Cohen, 2001). Comparing these different types of attention for external and internal attention might lead to different conclusions that are not necessarily incompatible. Relatedly, a strong argument can be made for the importance of shared representational coding in accounting for the magnitude of interference (Woodman & Luck, 2004; Woodman et al., 2001). In this review, we mainly focus on visuospatial attention and visuospatial WM, and their relationship.

What is important for the argumentation here is that it has been repeatedly observed that interference occurs when participants have to simultaneously use external and internal attention. The above models on overlap between perceptual attention and WM use interference paradigms to investigate this overlap. Their underlying logic is that if interference occurs between selecting internally maintained and externally perceived information, then these mechanisms probably use a common resource or language. Although there is no consensus on the extent of overlap, most contemporary models agree that interference between WM and perceptual attention does exist, and thereby assume some convergence between them (Chun, 2011; Chun & Johnson, 2011; Gazzaley & Nobre, 2012; Kiyonaga & Egner, 2013; Tas et al., 2016). What this interference unveils is that a shared resource is taxed when both external and internal information needs to be monitored, especially when the information is behaviorally relevant. Although valuable empirical work has previously informed on how this dependency can cause interference, a neglected question is how this resource sharing affects shifts between domains. Indeed, depending on whether these attentional mechanisms share a single resource or not, we can either expect similar switch costs for task switching between and within attentional domains, or alternatively a reduced cost for the former compared to the latter, respectively. However, to the best of our knowledge, neither of these two predictions has been explicitly stated or validated at the empirical level in the existing literature.

Hypotheses

From the literature reviewed above, we can predict, firstly, that, given that some form of (serial) attentional control is necessary for between-domain switches, a processing cost should be associated with them. Secondly, as a serial control mechanism is compatible with parallel processes at a lower level (Tamber-Rosenau & Marois, 2016), we can formulate two opposing predictions regarding its magnitude compared to the within-domain one. If we assume that external and internal attention share a single resource, the between-domain switch cost should be similar to the within-domain one, as attentional reallocation of a single resource is necessary in both cases. Alternatively, if we assume relative independence among them, processing can run in parallel and a smaller between-domain than within-domain switch cost should be evidenced (for a visual representation of these two opposite predictions, see Fig. 8). As a caveat, however, we have to concede that behavioral evidence will remain only suggestive in the present case. This problem lies not only in the paradigm itself, but in general in the use of behavioral data. Arguably, a way to address this limitation would be the use of neuroimaging data or modeling, which could allow drawing more conclusive statements about resource sharing (or the lack thereof). We discuss this more extensively in the Limitations section.

Interim conclusion

In the previous sections, we briefly presented a general model of attentional control explaining switch costs in terms of attentional deployment. More specifically, this framework states that within-domain switch cost corresponds to the time taken by serial control mechanisms to reallocate a limited resource to a new task-context. In case of between-domain switch (internal to external attention or vice versa), it is likely controlled by a similar serial control mechanism responsible for reallocating attention between conflicting task demands (Burgess et al., 2007). Consequently, we predict that there should be a between-domain switch-cost, as opposed to the mere absence of any processing cost when shifting attention between internal and external information. In addition, we concluded that in some cases resource-sharing among internal and external attention can exist (e.g., Kiyonaga & Egner, 2014), implying a single, limited resource as well. In line with these models, we can expect that shifting between attentional domains should produce a between-domain switch cost that should be similar to the within-domain one. On the other hand, however, there are some models that explicitly argue against full resource sharing between internal and external attention (Tas et al., 2016). From these models, we can thus predict that the between-domain switch cost should be reduced compared to the within-domain one, as they enable parallel processing. To test these different predictions, we present a novel behavioral paradigm. Our aim is first to find evidence, using this paradigm, for the existence of the between-domain switch cost, and, secondly, to systematically compare this cost to the within-domain one to determine whether resource sharing or independence best accounts for it.

Methodological requirements to explore between-domain switching

In this section, we list some of the main prerequisites a paradigm has preferably to meet at the methodological level to investigate switch costs between different attentional domains. First, as we wish to compare the between-domain (internal to external) switch with a regular within-domain (external to external) one using a within-subject design, we need to add this latter as a control condition. Preferably, the stimuli should be exactly the same for both conditions. In addition, we need a (baseline) task that is the same for both conditions and have to specify a priori which domain will serve as the baseline, i.e., the main task to which participants return after the occasional attentional switch towards the internal or external task. Given that participants’ behavior can be better controlled in an external task, it is more straightforward to use a task requiring external attention as the baseline condition. As such, we create a within-domain (external to external) switch when they return to the baseline task from the external task and a between-domain (internal to external) one when they return from the internal task. Finally, as we wish to avoid interference caused by possible overlap in stimulus and task set, we opt for completely independent tasks for the task-switch (internal/external vs. baseline) and adopt a block design. When there is no overlap in perceptual or motor demands between the two tasks, the only cost remaining (if any) is presumably competition for central attention, translated in the central processing bottleneck (Pashler, 1984; see Souza & Oberauer, 2016).

Towards a novel paradigm

In line with the above prerequisites, we devised a novel paradigm wherein we used a task requiring external attention most of the time (Baseline Task), which was interrupted sporadically and unpredictably by a secondary task (see Fig. 1A). Participants were informed beforehand about these deviant interruptions (i.e., that they had to switch now and then to another task during the block), yet not about their actual number and occurrences within a given block. Accordingly, interruptions were anticipated by them, but they remained primarily bottom-up in the sense of being deviant and their actual onset unpredictable. We compared performance on the first trial of the Baseline Task after this interruption (i.e., “Post”) with performance on the Baseline Task just before it (i.e., “Pre”). This Pre trial was always preceded by at least one trial of the same Baseline Task. This secondary task either required internal attention (Internal Task) or external attention (External Task), which allowed us to corroborate or refute the existence of between-domain (internal to external attention) and within-domain (external to external attention) switch costs. We used a block design, i.e., external or internal blocks, to avoid multiple switches within one block. Essentially, the logic of this paradigm was comparing a shift from the Internal to the Baseline Task with a shift from the External to the same Baseline Task, using very similar conditions and task demands (see Fig. 1A). The Baseline Task was the same throughout the experiment, allowing us to compare behavioral performance in terms of accuracy and reaction time (RT) for the between- and within-domain switch costs, as created through the interruption by the Internal and External Task, respectively. As a result of this carefully controlled set-up, we could directly assess the magnitude of the between- and within-domain switch costs, and eventually compare them at the statistical level when all other dimensions were controlled.

Fig. 1
figure 1

Paradigm and main task structure. (a) The left inset shows how between-domain shift (experimental condition) is operationalized. The right inset shows how within-domain shift is conceived and used as control condition. In both cases, the same dominant task requiring external attention had to be performed by participants (with a Pre and Post trial each time), but what differed between them was the type of attention switch actually required, being either internally based (WM) or externally (perceptual attention) based. (b) Trial sequence producing Pre and Post trials. The Baseline Task, requiring external attention, was sporadically interrupted by a trial from the External or Internal (or Update) Task (block manipulation), which required external or internal attention, respectively. Returning to the Baseline Task, then, implied a Within or Between domain shift, respectively. (c) Stimuli and instructions for External, Internal (and Update) Tasks. In the left inset, the Internal Task used in Experiment 1 (a), Experiments 2 and 4 (b), and Experiment 3 (b & c). In the middle inset, the External Task used in Experiment 1, either with or without a WM load (a), Experiment 2 (b), and Experiment 3 (a and b). In the right inset, the Update Task used in Experiment 2 (a)

We used a block design. For each condition, there was a practice block (with feedback) followed by ten experimental blocks. The blocks were presented consecutively (e.g., A-B-A-B-…) and their order was counterbalanced across participants. Each block contained 24 Baseline trials and three to five probes in which the digit-related task (i.e., Internal, External, or a variant) needed to be carried out (40 in total per condition). At the beginning of each block, participants had to memorize a WM loadFootnote 2 consisting of a number of digits, which were presented in a square with four compartments. This WM load was used for the Internal Task, but was included for blocks with External Task as well, in order to control for its global interference effect. The digits were pseudo-randomly selected, excluding repetitions and incrementing or decrementing sequences (e.g., 6–5 or 5–6). At the end of each block, an empty square appeared and participants had to retrieve and insert the WM load using the numerical pad of the keyboard. The Baseline Task was implemented as a basic visual discrimination task (Janssens, De Loof, Pourtois, & Verguts, 2016) in which participants had to judge whether the largest opening in a geometric figure (a square, a diamond, or a circle) was on the top or in the bottom (see Fig. 1B). These figures were presented pseudo-randomly, with an equal number of squares, diamonds, and circles over the entire experiment. Each trial started with a cue (500–800 ms), which was identical to the target figure but contained no openings. Immediately after the cue, the target figure appeared for 250 ms, after which it was replaced by a fixation cross for 700 ms, or until response. The Internal and External Tasks were presented using probes, which appeared with intervals of three to five Baseline Task trials. These probe-trials started with the same cues as the discrimination trials (for 500–800 ms), but were followed by a square with four compartments in which digits were presented, as the one used to present the WM load digits. In the Internal Task, participants had to decide whether the digit(s) presented were in the same location as the memorized digits (see Fig. 1C). In the External Task, participants had to visually search for the digit(s) meeting certain criteria, such as being in bold and/or italic (see Fig. 1C). The probe stayed on the screen for 2,500 or 6,000 ms (Experiments 3 and 4 vs. 1 and 2, respectively), or until response.

We conducted four different experiments, in which we consistently compared the costs associated with the Internal and the External Task. Across them, slight variations of this main paradigm were introduced each time to exclude alternative interpretations (see Fig. 1C). In Experiments 1 and 2, we investigated the cost associated with an External – No Load (NL) and an Update Task, respectively. In Experiment 3, we used both a Hard and Easy variation of the External and Internal Tasks. Finally, in Experiment 4, we compared a condition with an Internal Task using long versus short cue-target intervals (CTIs).

Experiment 1

Methods

Participants

Twenty-four participants (17 women) were recruited using Experimetrix, an online platform provided by Ghent University. We did not explicitly register their age as university students aged between 18 and 27 years made up the pool of participants for this platform. Our sample size was based on the literature of task-switching experiments with a within-subject design (e.g., Liefooghe et al., 2008). The study was reviewed and approved by the local ethics committee. All participants provided signed informed consent before the start of the experiment, explaining that they were free to end participation at any time. For their participation, they received 10 euro. At the end of the experiment, participants also completed two questionnaires (Depression Anxiety and Stress Scale and Ruminative Response Scale) and carried out a WM task (O-Span). We do not discuss these results here.

Stimuli and task

We programmed this paradigm using E-Prime (version 2.0). The Baseline Task was the same in all conditions (see here above). Participants responded with the “q” key if the largest opening was on top and with the “s” key if the largest opening was below. The Internal and External Tasks were as described above. We also included an External Task without a WM load (External-NL Task) to assess and model the possible modulatory effect of WM load per se on Baseline performance, when comparing the switch cost induced by the External Task (within-domain switch) and External-NL Task (within-domain switch, without WM load). In the Internal Task (between-domain switch), the trial consisted of four digits and participants were required to respond with the numerical pad key “7” if the positions matched with the memorized WM load and “4” if the digits were in different locations. More specifically, either all digits were in the same location or two had switched places. In the External Task, again four digits were presented at the probe and participants press “7” when the two bold digits were on the left side and “4” when they were on the right side. In the External-NL Task, the instruction to memorize the WM load at the beginning of the block was replaced by the instruction to look for the digits in bold and throughout the block. In other words, the task was identical to the External Task, but there was no WM load component.

Data analysisFootnote 3

Data preprocessing and visualization were carried out in R Studio (version 1.1.383) and statistical analyses were performed in JASP (version 0.8.4). As a preprocessing plan for the data, we planned to only analyze the Baseline Task trials right before (“Pre”) and right after (“Post”) the probe, as this difference reflects the switch cost. In addition, error trials on all Tasks were removed. Moreover, when an error was made on the Internal, External, or External-NL Task, we excluded the preceding and following trial on the Baseline Task as well. We removed participants who overall were not able to successfully retrieve the WM load at the end of the block (accuracy < 60%), but for the retained participants, we did not remove the blocks in which the load was not correctly retrieved. Accuracy scores were not analyzed as a separate dependent measure, but were used to remove subjects who did not follow the instructions (see footnote 2). We report these data for completeness and to demonstrate that there was no speed-accuracy trade-off. Outliers on RTs, defined for each condition within each subject as 1.5 times lower than the .25 quantile and 1.5 times larger than the .75 quantile, were removed. Statistical analyses were only run on the Pre and Post trial. To analyze Pre-Post differences, we used a 3 x 2 Repeated-Measures Analysis of Variance (RM ANOVA) with Type (Between, Within, Within-NL) and Order (Pre, Post) as within-subject factors. We report omega squared (ω2) as an estimate of the effect size, which is less biased than eta-squared (η2) and partial eta-squared (pη2) (Albers & Lakens, 2018). Significant main or interaction effects were followed up by two-sided paired sample t-tests. We report Greenhouse-Geisser corrected values for the interaction, as the assumption of sphericity was violated (Mauchly’s W = 0.675, p = 0.035). Importantly, using an additional Bayesian factors analysis, which compared the amount of evidence gathered in favor of H0 vs. HA, we could assess if between-domain switch costs obeyed the resource sharing account (evidence in favor of H0), or alternatively, were better accounted for by the existence of relative independent processes (evidence in favor of HA).

Results

For the final sample of 19 subjects (see footnote 3), the RT pattern on the Baseline Task (Fig. 2D) showed that participants were slower on the Post trials compared to the Pre trials, for all conditions, yet with some differences between them. A 3 x 2 RM ANOVA with Type (Between, Within, Within (NL)) and Order (Pre, Post) showed a significant main effect for Order (F1,18 = 10.44, p = 0.005, ω2 = 0.08) and an interaction between Type and Order (F1.51,28.29 = 5.715, p = 0.013, ω2 = 0.02). Follow-up paired sample t-tests revealed that this was driven by a significant difference for the Post trial in the Between Type compared to the Within and Within (NL) Types (T18 = 2.22, p = 0.04, Cohen’s dz = 0.51, 95% CI = [0.02; 0.98]; T18 = 2.35, p = 0.03, Cohen’s dz = 0.54, 95% CI = [0.05; 1.01], respectively). These two latter conditions did not differ significantly from each other (T18 = -0.35, p = 0.73, Cohen’s dz = -0.08, 95% CI = [-0.52; 0.37]). Next, we carried out a Bayesian RM ANOVA to inform which model best fitted these (RT) data (see Table 1). This analysis revealed that a model with Order explained the data best, 4.5 times better than the model with Type + Order + Type x Order (BF10 = 0.22).

Fig. 2
figure 2

Results of Experiment 1. (a) Mean reaction times (RTs) and 95% confidence intervals (CIs) (in white) on the Internal (Between), External (Within), and External-NL (Within-NL) Task. Mean for each participant (black dots) and their distribution. (b) Mean accuracy on these Tasks. (c) Mean accuracy on the Baseline Task for Pre and Post Trials. (d) Mean RTs and 95% CI (in white) on Baseline Task comparing Pre and Post trials for each condition separately. Mean for each participant (black dots) and their distribution

Table 1 Model comparison for the 3 x 2 Bayesian RM ANOVA. All models were compared with the best model

As can be seen from Fig. 2A, mean RTs on the Internal, External, and External-NL Task were not similar, while accuracy (see Fig. 3B) was at a ceiling for them. A one-way RM ANOVA with Type (Between, Within, Within-NL) confirmed a significant difference between them in RT speed (F1.46,26.31 = 4.40, p = 0.032, ω2 = 0.04). Follow-up t-tests revealed that the Within-NL type led to faster RT than the other two types, i.e., Between and Within Types (T18 = 2.35, p = 0.030, Cohen’s dz= 0.53, 95% CI = [0.05; 1.02]; T18 = 3.77, p = 0.001, Cohen’s dz= 0.87, 95% CI = [0.33; 1.39], respectively). These two latter conditions did not differ significantly from each other (T18 = 0.32, p = 0.76, Cohen’s dz = 0.07, 95% CI = [-0.37; 0.52]).

Fig. 3
figure 3

Results of Experiment 1. We binned the reaction time (RT) data (ten blocks) to explore possible changes of the switch cost across time. In agreement with the notion of a larger practice effect for the within- than between-domain condition, the switch cost was comparable for these two conditions at the beginning of the experiment but became smaller for the within than between-domain condition towards the end of it

Discussion

The main result emerging from this experiment was the observation of a between-domain switch cost, besides the within-domain one. More precisely, when participants had to switch from a WM-based towards a perception-based task, RTs were longer than when this perception-based task was repeated. This is an important finding as it suggests indirectly that our new experimental paradigm was adequate to capture such a switch cost at the behavioral level. Based on our literature review, we predicted that the between-domain switch cost would be either the same or smaller than the within-domain switch cost. However, in contrast to these predictions, we actually found a larger switch cost for the between-domain compared with the within-domain transition. One potential explanation for this unexpected result is that in the Internal Task, serial shifts between perception and memory were repeatedly made when comparing numbers in each of the four squares, which was not necessary in the External Task. Accordingly, the between-domain switch cost may have been artificially augmented by unwanted internal-to-perceptual alternations. Alternatively, an imbalance in the design may have caused this inflated switch cost for the between-domain condition. In fact, participants more often encountered the External Task (because of the inclusion of the External-NL one, sharing many similarities with it, in the design) than the Internal one throughout the experiment. Hence, switching attention between domains was as a result less frequent than switching attention within the external domain. This asymmetry in the frequency of attention switches could potentially explain the presence of a larger cost in the former compared to the latter condition, where attention switches were probably better learned or automatized as a result. If true, then the difference between the two main conditions should be smaller at the beginning than at the the end of the experiment. To test this alternative interpretation, we divided the experiment into three time intervals, using a binning procedure (see Fig. 3). Results confirmed a larger practice effect for the External than for the Internal Task. In addition, and relatedly, performance on the Internal Task was slightly slower (albeit better) than the External one, making it eventually somewhat difficult to interpret right away the difference in the size of the switch cost for these two conditions observed on the Baseline Task. Interestingly, despite these differences, the Bayes factor analysis actually provided conclusive evidence in favor of the model assuming no differences between the two main switch types (i.e., the best model was Order only, see Table 1). However, to address these concerns and provide more direct evidence for this interpretation, we ran a second experiment where we sought to better control for this imbalance between the conditions. Moreover, we added a third condition that required not only WM maintenance, but also updating (Myers et al., 2017; Oberauer, 2002, 2013), to assess if a more pronounced between-domain switch cost might be observed in this condition taxing internal attention to the largest degree. This Task was designed to be sufficiently different from the Internal Task to not result in an imbalance between the tasks.

Experiment 2

Methods

Participants

Twenty-eight (22 women) participants were recruited using Experimetrix.

Stimuli and task

Whereas the procedure and Baseline Task were the same as in Experiment 1, we made some minor adjustments to the Internal and External Tasks and included an Update Task as a third condition. In all conditions, a WM load was presented at the beginning of the block to be used actively in the between-domain block versus passively in the within-domain one. In the Internal Task (between-domain switch), only one digit was presented in the square, which could either match or mismatch with the memorized load (to keep the stimuli comparable with the newly added Update Task). In the External Task (within-domain switch), four digits were presented in the square, one in normal font, one in bold, one in italic, and one in italic and bold. Participants had to decide whether this latter digit was on the left or right side of the square (i.e., one target digit as well). In the Update Task (between-domain switch and memory updating), the square appeared with an operation (+1, -1, + 0, -0) in one of the cells. Participants performed this operation on the number that was in this location for the memorized WM load and continued with the updated WM load. Each operation was equally likely to occur. Due to this condition, the WM load digits ranged from 1–8 instead of 0–9 (Experiment 1) in order to avoid negative or two-digit numbers for the load. In the Update Task, participants pressed the numerical pad key “7” when they had updated their WM load.

Data analysis

We carried out the same pre-processing steps as in Experiment 1, retaining 25 participants for the analysis (see footnote 3). We used a 3 x 2 RM ANOVA with Type (Between, Within, Update) and Order (Pre, Post). To investigate differences in RT on Internal (Between), External (Within), and Update (Update), we used a one-way RM ANOVA. For the Update Task, we only used the trials where participants had to actually perform an update (+1, -1) and discarded the trials where no update was necessary (+0, -0), as they were likely less demanding.

Results

The ANOVA (Fig. 4D) revealed a marginally significant main effect for Type (F2,50 = 3.22, p = 0.05, ω2 = 0.008), a significant main effect for Order (F1,25 = 30.14, p < 0.001, ω2 = 0.12), and significant Type x Order interaction (F2,50 = 7.73, p = 0.001, ω2 = 0.02), driven by the Post trial in the Update condition being faster. Indeed, follow-up paired sample t-tests for the Post trial revealed that there was no difference for the Between and Within Type (T25 = -1.13, p = 0.27, Cohen’s dz= -0.22, 95% CI = [-0.61; 0.17]). These two conditions were significantly different from the Update condition, which was faster (T25 = 3.22, p = 0.004, Cohen’s dz= 0.63, 95% CI = [0.21; 1.05]; T25 = 3.20, p = 0.004, Cohen’s dz= 0.63, 95% CI = [0.20; 1.04], respectively). A Bayesian RM ANOVA further showed that a model with Type + Order + Type x Order best explained these data (see Table 2), though not conclusively better than the model with Order only (BF01 = 2.49). Crucially, Bayesian follow-up paired sample t-tests showed anecdotal evidence as well for the lack of difference between the Between and Within conditions (BF01 = 2.71), which both differed from the Update condition (BF01 = 0.087 and 0.091, respectively). When considering accuracy results (Fig. 4C), Baseline Task performance was found to be high overall.

Fig. 4
figure 4

Results of Experiment 2. (a) Mean reaction times (RTs) and 95% confidence intervals (CIs) (in white) on the Internal (Between), External (Within), and Update (Within-Update) Tasks. Mean for each participant (black dots) and their distribution. (b) Mean accuracy on these tasks. (c) Mean accuracy on the Baseline Task before (Pre) and after (Post) shifting. (d) Mean RTs and 95% CIs (in white) on the Baseline Task. Mean for each participant (black dots) and their distribution

Table 2 Model comparison for the 3 x 2 Bayesian RM ANOVA. All models were compared with the best model

When analyzing RT data for the Internal, External, and Update Task (Fig. 4A), the one-way ANOVA showed a significant main effect for Type (F2,48 = 23.12, p < 0.001, ω2 = 0.28). Follow-up paired samples t-tests failed to show a significant difference between the Internal and External Task (T25 = -1.87, p = 0.073, Cohen’s dz= -0.37, 95% CI = [-0.76; 0.03]), but showed a significant difference with the Update one (T24 = -7.59, p < 0.001, Cohen’s dz = -1.52, 95% CI = [-2.09; -0.93]; T24 = -4.07, p < 0.001, Cohen’s dz = 0.81, 95% CI = [-1.26; -0.35], respectively). Accuracy on the Internal and External Task (see Fig. 4B) was high in both, but somewhat higher for the Internal Task.Footnote 4 Accuracy for the Update Task should be interpreted with caution, however, as a simple key press was required in this condition upon completion of each individual WM updating, and hence we could not infer offline accuracy based on them directly.

Discussion

The results of this experiment confirmed the presence of a reliable between-domain switch cost, i.e., when switching from a WM- towards a perception-based task, which unlike the one found in Experiment 1, had a similar size compared to the within-domain cost. This result was in line with the resource sharing account assuming that shifting between external and internal attention mostly operates based on a common, central pool of resources. The Bayes factor analysis showed that the statistical model with the two main effects and the interaction best fit the data, due to the dissimilar RT pattern in the Update condition. A follow-up test additionally showed that the Between and Within conditions were not different from each other. Moreover, despite the somewhat diverging Update condition, the complex model was not conclusively better than the model including only Order. As we surmised, the Update condition yielded the slowest RT, indicating that it was challenging and taxed internal attention the most (Myers et al., 2017; Oberauer, 2002, 2013). Against our expectations, however, a reduced switch cost was evidenced in this condition, compared to the two other ones. A plausible explanation for this unexpected effect is, that because we did not use a response deadline, participants dwelt on the Update Task excessively and pressed the key not only when the update of their WM load was achieved, but when they were already fully prepared for the next trial (Baseline Task), strongly reducing in turn the switch cost in this condition. Hence, no clear conclusion could be drawn from the results obtained for this condition (where a trade-off between the Update and Baseline Task likely took place, see Fig. 4 A and C), unfortunately. To formally rule out the possibility that task difficulty (on the Internal, External, or Update Task) accounted for these results (i.e., switch costs at the Post trial), we ran a third experiment where we directly manipulated and included this factor (i.e., task difficulty) in the experimental design.

Experiment 3

Methods

Participants

Thirty-one participants (23 women; mean age 24.8 years) were recruited through Experimetrix.

Stimuli and task

The procedure (Baseline, Internal, and External Tasks) was identical to Experiment 2, with the following notable changes. For each switch type, two difficulty levels were created by adding an easy variant for both and these were manipulated using a block design. Accordingly, we devised and included the Internal-Easy Task, which was identical in every aspect to the Internal one (Experiments 1 and 2), but made easier because either the upper or the lower two digits of the square were replaced by X’s. In the External-Easy Task, that was identical to External one (Experiments 1 and 2), it was also made easier as either the two left or the two right digits in the square were in normal font, with the other two being in bold. In all conditions, we also changed the presentation time of the square from 6,000 to 2,500 ms to discourage participants from spending excessive time on the External or Internal Task to prepare for the upcoming Baseline Task. This duration was chosen because it roughly corresponded to the average RT + 2 SDs extracted from Experiments 1 and 2. Finally, we used two different response mappings, counterbalanced across participants. For half of the participants, responses were made with key presses “q” and “s” for the largest opening being up or down, respectively, in the Baseline Task and with numerical pad presses “7” and “4” for match/left or mismatch/right respectively. For the other half, “4” and “5” in Task A and “q” and “w” for match/left and mismatch/right, respectively.

Data analysis

Twenty-two participants were retained for the analyses (see footnote 3). A 2 x 2 x 2 RM ANOVA with Type (Between, Within), Order (Pre, Post) and Difficulty (Hard, Easy) was used.

Results

The ANOVA (Fig. 5D) showed a significant main effect of Order (F1,21 = 49.73, p < 0.001, ω2 = 0.24) and significant Order x Difficulty interaction (F1,21 = 5.97, p = 0.02, ω2 = 0.01). Follow-up paired sample t-tests for the Post trials, however, did not reveal significant differences (Ts21 < 0.94, ps > 0.36, Cohen’s dz< 0.28, 95% CIs all include 0). A Bayesian RM ANOVA indeed showed that the model with Order only explained the data best, conclusively (4.6 times) better than the second-best model with Type + Order (see Table 3). In addition, this analysis showed that the model with Order explained the data 5.9 times better than the model with main effects of Order and Difficulty (BF10 = 0.17), and 6.6 times better than the one including the interaction effects for Order and Difficulty (BF10 = 0.15). Accuracy was at ceiling (i.e., around 90% correct) for all conditions (see Fig. 5C).

Fig. 5
figure 5

Results of Experiment 3. (a) Mean reaction times (RTs) and 95% confidence intervals (CIs) (in white) on the Internal-Hard (B-H), Internal-Easy (B-E), External (W-H), and External-Easy (W-E) Tasks. Mean for each participant (black dots) and their distribution. (b) Mean accuracy on these tasks. (c) Mean accuracy on the Baseline Task before (Pre) and after (Post) shifting. (d) Mean RT and 95% CIs (in white) on the Baseline Task. Mean for each participant (black dots) and their distribution

Table 3 Model comparison for the 2 x 2 x 2 Bayesian RM ANOVA. All models were compared with the best model. Other models (BF10 =< .01) not shown

As expected, performance on the Internal and External Tasks was influenced by Difficulty (Fig. 5A), with slower RTs for hard (Internal-H/External-H) than for easy conditions (Internal-E/External-E), as confirmed using a 2 x 2 RM ANOVA with Type (Between, Within) and Difficulty (Hard, Easy) and showing a significant main effect for Difficulty (F1,21 = 161.45, p < 0.001, ω2 = 0.39). In addition, we found a significant main effect for Type (F1,21 = 4.89, p = 0.03, ω2 = 0.06) and significant Type x Difficulty interaction (F1,21 = 8.04, p = 0.01, ω2 = 0.03). A follow-up paired samples t-test showed that this interaction was driven by a significant reduction in RTs for the Within-Easy Task compared to the Between-Easy one (T21 = -3.15, p = 0.005, Cohen’s dz= -0.67, 95% CI = [-1.13; -0.20]). Accuracy (Fig. 5B) was high across all conditions.Footnote 5

Discussion

Results of this experiment corroborated the existence of a reliable between-domain switch cost, being comparable in magnitude with the within-domain switch cost (see also results of Experiment 2). The Bayes factor analysis further confirmed this observation by providing strong evidence for the lack of difference between the two switch cost types. As such, these results were compatible with the resource sharing account. Another important result emerging from this experiment concerned the observation that these two types of switch costs were actually unrelated to changes in task-difficulty occurring at the probe level (Internal and External Task). As expected, participants were slower for the Internal-Hard Task than for the Internal-Easy one, and for the External-Hard than for the External-Easy one. However, and importantly, shifting away from them resulted in a similar cost, suggesting that our paradigm was suited at the methodological level to measure and capture switch costs (either within or between-domain), as opposed to variations in task difficulty only, or a mere trade-off effect between the two tasks (Baseline Task vs. Internal or Baseline Task vs. External). The Bayesian analysis revealed that the model with only order explained the data at least six times better than the ones including Difficulty. Whereas these results clearly corroborated resource sharing, one could however argue that this paradigm is not suited to reveal subtle differences between conditions because it uses a relatively long CTI (500–800 ms). In this scenario, a possible asymmetry between conditions could be obscured by strong preparation effects. We initially chose this relatively long CTI after extensive piloting to keep error rates low and mainly focus on RTs, where a clear switch cost was observed. Hence, this seemingly long CTI was actually considered short in our paradigm. To address this concern directly, we ran a fourth experiment, where we manipulated this interval, being either short (as in Experiments 13) or made longer. We reasoned that if strong preparation effects were present in Experiments 13, which potentially obscured subtle differences between the switch types, the switch cost observed using the long CTI would not be reduced compared to the one observed using the short CTI. Alternatively, if preparation was not completely terminated during the CTI used in Experiments 13, a reduced switch cost would be found with the long compared to the short CTI, thereby ruling out the use of a somewhat suboptimal paradigm to reveal potential differences between conditions.

Experiment 4

Methods

Participants

Fifty-fiveFootnote 6 participants (44 women; mean age 21.8 years) were recruited through Sona (online recruitment system at Ghent University).

Stimuli and task

In this experiment, we used only the Between condition (with Baseline and Internal Task) of Experiment 3, with either a Short (600 ms) or Long (1,200 ms) cue-target interval. This manipulation was achieved by means of a block design (ten blocks per cue-target interval). Cue-presentation time was manipulated only for the Post trial of the Baseline Task, by adjusting the cue presentation time of this trial. Participants responded using the “q” and “s” key in the Baseline Task and using the “7” and “4” numpad key in the Internal Task.

Data analysis

Forty-seven participants were retained for the analyses (see footnote 3). We used a 2 x 2 RM ANOVA with Order (Pre, Post) and Interval (Short, Long) to analyze the data.

Results

The ANOVA (Fig. 6D) showed a significant main effect of Interval (F1,21 = 12.81, p < 0.001, ω2 = 0.01) and of Order (F1,46 = 64.59, p < 0.001, ω2 = 0.11), as well as a significant Order x Interval interaction (F1,46 = 9.41, p = 0.004, ω2 = 0.01). In both the Short and Long Intervals, there was a significant increase in RTs when comparing the Pre and Post trial (T46 = -7.27, p < 0.001, Cohen’s dz= -1.06, 95% CI = [-1.41; -0.69] and T46 = -5.92, p < 0.001, Cohen’s dz = -0.86, 95% CI = [-1.19;-0.52], respectively). Crucially, follow-up paired samples t-tests showed that there was a statistically significant difference for the Post trials (T46 = 3.75, p < 0.001, Cohen’s dz= 0.55, 95% CI = [0.24; 0.85]), whereas no such difference was found for the Pre trials (T46 = 0.11, p = 0.91, Cohen’s dz= 0.02, 95% CI = [-0.27; 0.30]). A Bayesian RM ANOVA further showed that the model with Interval + Order + Interval x Order explained the data best, substantially better (4.8 times) than the model without the interaction (BF10 = 0.21). Accuracy was high for all conditions (see Fig. 6C). Table 4.

Fig. 6
figure 6

Results of Experiment 4. (a) Mean reaction times (RTs) and 95% confidence intervals (CIs) (in white) for the Short (Between-Short) and Long (Between-Long) Interval. Mean for each participant (black dots) and their distribution. (b) Mean accuracy for these Intervals. (c) Mean accuracy on the Baseline Task before (Pre) and after (Post) shifting. (d ) Mean RTs and 95% CIs (in white) on the Baseline Task. Mean for each participant (black dots) and their distribution

Table 4 Model comparison for the 2 x 2 Bayesian RM ANOVA. All models were compared with the best model

As revealed by a paired samples t-test, performance on the Internal and External Tasks was not influenced by Interval (Fig. 6A; T46 = -1.82, p = 0.08, Cohen’s dz95% CI = [-0.03; 0.56]). Accuracy (Fig. 6B) was high across all conditions.Footnote 7

Discussion

Results for the short CTI closely replicated those of Experiment 3. Importantly, we found that the (between-domain) switch cost was substantially reduced, yet still significant when using a longer (i.e., twice as long) CTI. Accordingly, even though the short interval used here and in Experiments 13 could be seen as relatively long compared to the ones usually used in the task-switching literature (Monsell, 2003; Vandierendonck et al., 2010), it was, however, not sufficiently long for participants to fully prepare for the upcoming trial of the Baseline Task. This is an important result because we could therefore rule out that the lack of switch cost difference between the two main conditions found repeatedly in Experiments 13 in this study was caused by strong preparation effects obscuring subtle difference between them.

General discussion

In this paper, we first performed a selective review of the literature on psychological processes that allow switching attention between internal and external information. Despite its ubiquity in a myriad of daily-life activities, surprisingly, very few studies have been performed on this specific matter (but see Burgess et al., 2007; Carlson et al., 1993; Dark, 1990; Gilbert et al., 2005); Weber et al., 1986 for notable exceptions), and accordingly, we still lack a clear understanding of how switches of attention across these two domains work. Relatedly, there currently exists no validated experimental paradigm to measure the effects of this fundamental process in standard laboratory conditions. The rich literature on task switching (i.e., when attention switches between two tasks or dimensions within the same modality, usually being externally defined), however, allowed us to outline a new framework able to account for between-domain task switches. Using this framework, we could formulate new and clear predictions, and later test them in a series of four behavioral experiments using a newly developed paradigm fulfilling important methodological criteria and eventually allowing us to compare within- to between-domain switch costs.

In the task-switching literature, switch costs reflect the capacity limitation of a single attentional resource that needs to be reallocated by a serial control mechanism towards the novel task-context as soon as it is called for, usually using a specific (visual) cue and instructions (Liefooghe et al., 2008). We borrowed this notion and transposed it to the case of between-domain switching. In line with evidence for a serial control mechanism (Burgess et al., 2007), we predicted that a between-domain switch cost would occur. In addition, when considering two diverging frameworks concerning the amount of resource-sharing between WM and perceptual attention, we formulated different predictions regarding the relative magnitude of the between- compared to the within-domain switch cost. If resource sharing between them prevails, then we hypothesized that they should be similar. However, if separate and partly independent resources underlie selective attention and WM, we could then deduce that the between-domain switch cost should be smaller compared to the within-domain one.

To test the magnitude of switch costs occurring within vs. between domains, four successive experiments were carried out, examining (1) whether we could find evidence for the existence of a between-domain switch-cost, and (2) whether this cost would fit the resource-sharing or alternatively, the independent processes account. We consistently found that a cost was associated with a between-domain switch, which suggested that some form of serial control was necessary in this condition. This is a first important result emerging from these four experiments. Moreover, the results of Experiment 4 clearly showed that the strength of this between-domain switch cost strongly varied depending on the length of the CTI, thereby ruling out the possibility that this effect found consistently in Experiments 13 was unspecific somehow. In addition, when comparing this cost with the one associated with a within-domain switch, the results were best explained each time by a Bayes-factor statistical model assuming no difference between them. This was the case even when taking into account small variations in the control conditions across experiments. Accordingly, we can conclude that even though the experimental paradigm was sensitive enough to detect subtle differences between conditions, there was no evidence for significant difference between the between- and within-domain switch cost across different experiments (see Experiments 13). Consequently, our new results lend support to the resource sharing account. This is the second main contribution of our study.

Although the results of these four experiments were in accord, some slight differences were found between them, nonetheless. To assess whether the pattern of results was consistent and reliable, we performed a single analysis across them. More specifically, we pooled the data sets from Experiments 13 (but dropped Experiment 4 as there was no Within condition in this experiment) together (see Fig. 7) and carried out a 3 x 2 x 2 Omnibus RM ANOVA with Experiment (1, 2, 3), Type (Between, Within), and Order (Pre, Post) as factors. This analysis revealed a significant main effect of order (F1,64 = 72.68, p < 0.001, ω2 = 0.16), reflecting the consistent slowing down on the Post compared to the Pre trial, and hence the cost associated with switching. This analysis also revealed that the three-way interaction (Type x Order x Experiment) was significant (F2,64 = 5.06, p = 0.009, ω2 = 0.007), probably due to the slight unbalanced design used in Experiment 1 that produced a decreased within-domain switch cost (see Fig. 7). A supplementary Bayesian RM ANOVA run on these data revealed that the best model was the one with Experiment + Order + Experiment x Order, negligibly (1.5 times) better than the one with Order + Experiment as factors (see Table 5). Crucially and despite this difference between experiments, models with Type as factor did not fit the data well, further arguing for a lack of reliable difference between the between- and within-domain switch cost (best model without Type explained data 7.1 times better than with Type).

Fig. 7
figure 7

Summary of main results obtained for Experiments 13. Mean reaction time (RT) on Pre and Post trial on the Baseline Tasks for each experiment separately, when comparing the Between (internal to external attention) and Within (external to external attention) conditions specifically. Each time, a significant switch cost was observed, being similar in size for the within and between-domain conditions

Table 5 Model comparison for the 3 x 2 x 2 Bayesian RM ANOVA. All models were compared with the best model. Only the five best models are shown here

In the task switching literature, either interference (Allport et al., 1994; Waszak et al., 2003) or reconfiguration (Mayr & Kliegl, 2000, 2003; Meiran, 1996; Monsell & Mizon, 2006) is usually proposed to explain the switch cost; these two frameworks being not necessarily mutually exclusive (Vandierendonck et al., 2010). Since we deliberately avoided stimulus and response overlap for the two tasks with our new paradigm, it appears difficult to assume that interference was causing the observed within- and between-domain switch costs in the present case. Interestingly, reconfiguration (as well as interference) entails attentional control as being causally related to resolving conflict occurring between the activation of the previous task-set and the configuration of the current one (Liefooghe et al., 2008; Pashler, 1984; Stoet & Hommel, 1999; Vandierendonck et al., 2010). In this framework, a switch cost basically reflects the time needed by dedicated attentional control processes to redirect a single, capacity-limited resource to the new task context. However, as the psychological refractory period (PRP) demonstrates (Pashler, 1984; for integration with task-switching literature, see Gilbert, 2005; Luria & Meiran, 2003; Oriet & Jolicoeur, 2003; Sigman & Dehaene, 2006; but see Vandierendonck et al., 2010), this redirection only occurs after the termination of the previous task context. Translated to the between-domain switch costs observed across the four experiments in our study (see Fig. 6), this means that whereas attention was lingering on the previously activated – internal (because WM based) – task context, access was temporarily restricted to reconfigure the system for the current – external – task context, given that they both drew on a common pool of resources.

This account implies that dedicated attentional control processes are necessary to redirect resources between the internal and external domain, for which we proposed the supervisory attentional gateway as a likely candidate (Burgess et al., 2007). Such a serial mechanism, in combination with a shared, limited resource, operates as a central bottleneck and can potentially account for the processing cost associated with a between-domain switch. This kind of bottleneck, reflected in delayed RTs, has been extensively investigated through the PRP (Pashler, 1984). In Fig. 8, and in analogy to the PRP, we specify and compare what processes might actually cause the switch costs when they occur within- versus between-domain, and eventually what might potentially differ between resource-sharing and independence. When two trials of the same task (Baseline Task) followed each other, no decoupling or reconfiguration was necessary, as the task set was identical (Fig. 8A). Participants could immediately start processing the new trial, making this a suitable baseline measure (Pre trial). In Fig. 8B, the putative cognitive operations contributing to the standard (i.e., within-domain) switch cost are depicted. In the case of a shared resource, these costs are additive, as this resource can only be implemented to the next stage when the previous one is terminated. Firstly, from t0 to t2, this resource is occupied in decoupling processes related to the previous task set (External Task). On the condition that this is resolved, attentional control can be recruited to reallocate this resource towards the new task context (Baseline Task). The second stage, from t2 to t3, is involved in task set reconfiguration. Both processes result in the standard switch cost, depicted as the difference between t6 and t4 (see Fig. 8), which is roughly equivalent to the difference between Post and Pre (see Experiments 14 above). In Fig. 8C, the processes contributing to the between-domain switch cost are depicted, when resource sharing is assumed. Given this assumption, the same processes as the ones active during within-domain switch cost are considered. More specifically, attentional control redirects the single resource from the internal towards the external task set. Consequently, the switch cost is numerically comparable to the within-domain one, which is precisely what we have found in Experiments 2 and 3 (as well as Experiment 1 to a lesser degree) in this study. In Fig. 8D, the hypothesis assuming independent resources is presented for comparison purposes. In this case, roughly similar processes contribute to the cost, but, crucially, they are not producing additive effects as independent resources (for each modality) can work in parallel. More precisely, the decoupling phase does not interfere with the reconfiguration of the task set. As a result, the only cost then comes from the reconfiguration, occurring from t0 to t1. Consequently, the switch cost is reduced compared to the one in Fig. 8C, as reflected in the difference between t6 and t5. Yet, none of the four experiments conducted in this study yielded conclusive support for this alternative interpretation. As we have argued here in different places, since the within- and between-domain switch costs were comparable in size and expression (see results of Experiments 2 and 3), resource sharing most likely accounted for them (see Fig. 8C).

Fig. 8
figure 8

Presentation and systematic comparison (in terms of underlying cognitive operations) of the different switch types considered in this study. (a) In a repetition trial (= Pre), no reconfiguration – and attentional control – is necessary. Immediately at the appearance of the trial (t0), the processing can begin. (b) Standard (within-domain) switch cost, due to lingering activity from the previous trial (t0 until t2) and reconfiguration for the current trial (t2 until t3). Shifting attention from the previous to the current task set requires enhanced control and utilizing specific resources, which are limited. The resulting switch cost is reflected in the difference between t6 and t4, i.e., Post - Pre. (c) In the case of a between-domain cost happening under resource sharing, a similar logic is considered as the one use for the within-domain one and depicted in panel C. Single resource needs to be reallocated from lingering on the previous task to the configuration of the current one. (d) In the case of a between-domain cost occurring under independent resources for external and internal attention, there is no detrimental effects from (internal) attention lingering on the previous task set. The new task-set can immediately be activated through external attention, resulting in only reconfiguration actually contributing to the cost, which eventually becomes smaller (t0 until t1). This, then, provides a benefit compared to the standard switch cost, reflected in the difference between t6 and t5

Translated to specific task-components, the Baseline Task used in our new paradigm probably involved shape identification, gap detection, gap comparison, response decision, motor plan generation and execution. For the Internal Task, number detection and identification, retrieval of the number-place association held in WM, comparison of these two latter processes, response decision, motor plan generation and execution were probably required. For the External Task, main components involved were the discrimination of both relevant features (i.e., bold and italic), response decision (left vs. right key press), motor plan generation and execution. As we discussed here above, the switch cost arose in both cases due to the time required for dedicated control processes to reallocate a limited amount of resources towards the new task context. In case of a within-domain cost, this was due to the time required for external attention to shift between two different discrimination tasks, whereas in the case of a between-domain cost, redirecting this resource from WM (internal matching task) to external attention (shape discrimination task) probably caused it.

The underlying cognitive processes summarized in Fig. 8 entail attentional control as one of the main, overarching, and key (supervisory) mechanisms through which interference between the previous and current task set is resolved, by enabling a rapid redirection of resources towards the current attentional domain. Accordingly, and following the main tenets of several contemporary models in the cognitive science literature (Liefooghe et al., 2008; Pashler, 1984; Stoet & Hommel, 1999; Vandierendonck et al., 2010), we argue for a direct and causal link between (supervisory) attention control on the one hand, and task switching on the other hand, including when this process operates between modalities, as evidenced here in this study for the case of phasic transitions between representations held in WM and externally-based selective attention.

However, and as a caveat, we acknowledge that the notion of attentional control, drawing on a shared pool of resources, is probably ill-defined, loose, and too broad as such to accurately and satisfactorily grasp the complex cognitive architecture underlying task switching, especially when it operates between modalities. Indeed, resource-sharing between attentional domains is an active research area, where it is still debated as to how this sharing is actually implemented and achieved (Tombu & Jolicœur, 2005). In this respect, a better understanding of this architecture might be obtained if one considers the notion of “the focus of attention” (as opposed to attention control, broadly defined), as proposed earlier in the WM literature. Therein, the focus of attention usually corresponds to a distinctive state in WM that selects one chunk of information for the next cognitive operation required (Cowan, 2010; Garavan, 1998; Oberauer, 2002, 2009, 2013). Interestingly, such a highly selective (and testable) state is assumed to play an important role in the guidance and orienting of external attention as well, as demonstrated in standard cueing paradigms, for example (e.g., Posner, 1980). Consequently, this state seems to correspond to an intrinsic property of attending relevant information in both domains. Interestingly, and as we propose here, it may be the case that the focus of attention is a domain-general, shared resource that can either select internal or external information, depending on the specific task demands. In agreement with this modern notion, it has been previously found that the same representational space is used for internally and externally attended stimuli (Burgess et al., 2007; O'Craven, Downing, & Kanwisher, 1999; Nanay, 2015; Nobre, Coull, Maquet, and Frith, 2004). In addition, it has been argued that information in WM is at the interface between the external and internal environment, as certain WM tasks cause interference for both external and internal attention (Chun et al., 2011). Such a proposal is valuable because it allows to refine attentional control between external and internal attention in terms of general biased competition within the focus of attention (for a similar proposal in terms of competition between external and internal attending, see Burgess et al., 2007). Alternatively, it has been argued that the focus of attention is not a single mechanism, but that there are multiple, domain-specific foci of attention (Fougnie, Zughni, Godwin, & Marois, 2014). In this scenario, it is likely that multiple foci of attention compete among each other for access to consciousness. Whereas the current results do not allow to distinguish between these different models, they are, however, in line with a single, overarching mechanism of control. Additional experimentation, including manipulation of the extent of overlap in representational space (see also Woodman & Luck, 2004; Woodman et al., 2001), perhaps guided by modelling efforts, is necessary to compare more formally these different models, and eventually gain insight into the cognitive architecture underlying attention flexibility. This could be a fruitful avenue for future research, as it would allow making specific predictions about which features of attention could be relevant in taxing this shared resource during shifts between domains, and hence influencing the magnitude of the between-domain switch cost seen at the behavioral level.

Limitations

Some limitations warrant comment. First, our paradigm was not suited to exploring switches of attention from an external to an internal (dominant) task, limiting in turn the conclusions that we could draw based on it and related to attention flexibility and control. This is an important caveat, as whether internal stimuli can interrupt ongoing cognition or not is heavily debated in the existing cognitive psychology literature (Downing, 2000; Soto, Heinke, Humphreys, & Blanco, 2005; Soto, Hodsoll, Rotshtein, & Humphreys, 2008; Woodman & Luck, 2007). Accordingly, it would be extremely valuable to harness the current paradigm in future studies and add a condition where a shift from external to internal would occur. This would require having a Baseline Task being primarily internal, and being interrupted now and then by an External Task. Relatedly, in pursuing this research line, different tasks should ideally be tested and compared to one another to investigate the relative effect of task-set reconfiguration to the observed switch cost. It can be argued that in the current paradigm, the External and Baseline Tasks are more similar than the Internal and Baseline Task with respect to the task set, as both require active visual discrimination. This asymmetry in the task-set reconfiguration when returning to the Baseline Task might obscure potential differences between the within- and between-domain switch costs, although we explicitly sought to keep the Internal and External Tasks as similar as possible

Second, although we devised the External and Internal Tasks to rely primarily on internal (WM) and external (selective attention) features or processes, respectively, in fact, each of them was not pure and was necessarily contaminated by external and internal components to some degree (see also Fig. 8). For example, participants were required to internally maintain information (WM load) and response rules when carrying out the Baseline or External Tasks. Similarly, in the Internal Task, participants compared the internally retrieved information in WM with externally presented information. When interpreting the results, one should therefore be cautious and consider a relative contribution of external versus internal attention to the observed within versus between-domain switch cost.

Last, it can be argued that behavioral data alone may be insufficient to disentangle resource sharing between WM and external attention from their partial independence. In fact, two independent processes could produce similar RTs while processes sharing similar resources could actually result in different RTs when different levels of difficulty are considered for example. To overcome this limitation, future studies could use neuroimaging methods in combination with this new paradigm, with the aim of assessing the amount of resource sharing at the anatomical level between WM and external attention during the occurrence of between-domain switches. Although the current paradigm should be used in different contexts before we can draw more definite conclusions about resource sharing, the results obtained with it across four experiments provide strong evidence in support of it.

Conclusions

After having performed a selective review of the existing literature on task switching when these shifts occur between external (sensory-related) and internal (WM-based) information, we came to the conclusion that at least two different predictions could be formulated to account for them. In a series of four behavioral experiments, we devised a new paradigm to test them and confirmed the existence of a reliable between-domain switch cost that was similar to the standard, within-domain one. Accordingly, these new results corroborated the hypothesis of a shared resource between external and internal attention, as opposed to partly independent or parallel control processes for them. This cost is interpreted as reflecting a bottleneck in attention control, because decoupling and reconfiguration are required. We suggest that defining it in terms of focus of attention and state-based property of WM might turn out to be beneficial in the future to test and explore further the boundaries of this remarkable cognitive ability, enabling human subjects to flexibly shift between external and internal information.