The effect of emotional arousal on visual attentional performance: a systematic review

The present review was primarily motivated by a desire to address confusion regarding the unique role of arousal through the visual search literature, and to emphasize the importance of using precise and uniform terminology. As Reisenzein pointed out “the quality of an emotion depends on the proportion of pleasure-displeasure and arousal (or activation-deactivation) that is experienced” (Reisenzein, 1994, p. 527). Emotional arousal has received little attention in previous research, although it is an integrative and equally important part of dimensional models of emotions along with valence (Bliss-Moreau et al., 2020; Bradley et al., 1992; Feldman Barrett & Russell, 1998; Rubin & Talarico, 2009; Watson et al., 1999). However, this trend seems to change in recent years (Onie & Most, 2021; Singh & Sunny, 2017), prompting the need for a systematic review of past research that could aid future studies in answering the most burning questions. A meta-analysis (Pool et al., 2016) of attentional biases for positive emotional stimuli underscores the importance of arousal by noting that the magnitude of effects is moderated by the arousal level of the stimuli used. The fact that previous research investigating the interaction between emotion and visual attention mainly focused on the effects of the valence of a stimulus (i.e., positive or negative) on various attentional processes, and, thus, often neglected the role of arousal (Yiend, 2010) might not be evident at first. This is mostly due to a major confound in the field: the terminology is not used in a precise and consistent way, and, thus, the two dimensions are conflated. In fact, some previous studies claim to have found an effect of emotionally arousing stimuli, whereas they used positively and negatively valenced stimuli and did not directly manipulate the arousal level of the stimuli used. In the present article, the primary intention is to capture papers where the effect of the arousal dimension of the stimuli was directly manipulated and empirically tested.

Emotional arousal can be defined as a state of physiological activation; heightened levels evoke feelings of excitement, nervousness, or alertness—creating a readiness for action—while decreased levels induce relaxation or boredom (Mather & Sutherland, 2009; Pessoa, 2013; Reisenzein, 1994). Further, emotional arousal can be described as the degree of subjective activation an observer experiences when viewing a stimulus (Pessoa, 2013; Reisenzein, 1994). There is a duality in the definition of emotional arousal concerning the subjective and objective nature of arousal which is also evident in its measurement. On the one hand, subjective (emotional) arousal is often measured by self-report questions (e.g., How aroused you felt while viewing the picture? Rate on a 9-point Likert-type scale where 1 means Calm and 9 means Excited.) asking the participant to report the feelings that an emotionally charged stimulus elicited (Lang et al., 1997). On the other hand, objective (emotional) arousal can be shown by various physiological changes such as increased skin conductance (Cuthbert et al., 2000; Kasos et al., 2019), decreasing nose-tip temperature (Sato et al., 2020), and greater N1, P2, P3 event-related potential amplitudes (Olofsson et al., 2008). Nonetheless, previous studies found a strong correlation between subjective and objective arousal measures (Cuthbert et al., 2000; Kasos et al., 2019; Olofsson et al., 2008; Sato et al., 2020). Still, with arousal (similarly to other emotion-related variables) it is a question if one can be sure to have measured it. Previous studies examining various effects of arousal usually take on the same approach, that is, they use a mix of positively and negatively valenced stimuli that are considered highly arousing (e.g., pictures, music, facial expressions). A correlation between subjective and objective measures is also a validating factor. Since both subjective feelings and objective changes count, one can mostly be sure to have measured arousal if they point in the same direction.

Stimuli with emotional content (e.g., pictures, music, words) initiate generalized arousal in the organism by releasing biogenic amines, such as norepinephrine, and hormones, such as cortisol (LeDoux, 2012). That is, arousal conveyed by the stimulus could initiate a readiness for action due to a heightened state of physiological activation (Bradley et al., 2008; Olofsson et al., 2008; Pessoa, 2013). Further, arousal may enhance processing in the survival circuit—a hardwired, evolutionarily conserved set of defensive mechanisms activated in challenging and possibly life-threatening situations—which facilitates activity in sensory, memory, and attentional processes (LeDoux, 2012, 2022). However, extreme levels of subjective arousal could lead to worse performance compared to lower levels of arousal because the available cognitive capacity is restricted by the arousal level that competes with task-relevant processes (Curci et al., 2013; Kahneman, 1973; Plass & Kalyuga, 2019; Schmeichel, 2007). Further, arousal—if not task-relevant—can also be distracting because stimuli that evoke a higher level of arousal may be prioritized over less arousing stimuli in the competition for processing (Buodo et al., 2002; Mather & Sutherland, 2011; Unsworth & Robison, 2017). In the past 50 years, a few theories were coined to describe the effect of arousal (independently of valence) on visual cognition. In the present article, by collecting the results of visual attention-related studies, I also aim to compare which theory (if any) can best describe the range of findings.

I have prepared this systematic review in accordance with the recommended reporting items for systematic review and meta-analysis protocols (PRISMA) 2020 checklist to include in a systematic review protocol (Page et al., 2021). I used a two-stage systematic approach to identify articles that examined the effect of emotional arousal on visual search performance. The initial search was conducted on 1 March 2022 in the Web of Science database using the following combinations of keywords: ("emotion" OR "emotional") AND ("arousal") AND ("visual attention" or "visual search" OR "visual detection") with (“participants” OR “subjects” OR “patients”). The following predetermined criteria were necessary for further inclusion:

No restrictions were made concerning the modality of the emotional stimuli (e.g., auditory, visual), task or paradigm and methodology (e.g., behavioral, EEG, fMRI) used, and mental or physical health of the participants. After the initial search, I removed duplicates and examined the resulting articles’ references to ensure all relevant papers were included. I did not analyze effect sizes due to differences in the type of tasks, emotional stimuli, and timing methods used.

The search yielded a total of 123 articles published between 2000 and 2022 of which 32 articles fit the search criteria. A summary of our search strategy is presented in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Flow Diagram (see Fig. 1). A summary of each article is presented in Table 1. The papers contain a total of 46 experiments with an average sample size of 49.78 and an average female-to-male ratio¹ of 0.60. The modality of the stimuli used in the studies was not always similar across experiments reported, therefore, here I report the type of stimuli per experiment. A total of 38 experiments used visual stimuli (29 used pictures, seven used emotion-laden words, and one used both), while eight studies used auditory stimuli in addition to visual stimuli (six used sounds, and two used classical music). The majority of the experiments are exclusively behavioral (34), while five studies measured EEG, five studies used an eye-tracking device, and one study included fMRI measures. Of all the papers, the majority (15) used a visual search task (in 19 experiments), three studies used an attentional blink task (in eight experiments), three studies used spatial cueing (in four experiments), three studies used free viewing (in four experiments), one study used a pair-matching task (in one experiment), and one study used ANT (in one experiment). Six additional studies used other paradigms where the emotional stimuli were not central to the task (they were task-irrelevant) and had to be ignored. These studies used the number matrix paradigm (six experiments), solving arithmetic operations (three experiments), target counting (one experiment), or locating other targets (one experiment) as the main task. Furthermore, regarding stimuli selection studies relied on standardized arousal ratings provided by the manuals of the stimuli database. In most cases, there was no further validation, while in some instances the stimuli presented in the experiment were also rated by an independent sample or by the participants in the study. In one study, the was validated by fMRI results with the difference between BOLD responses (see Table 1 for the details). In the results section, I present the results of the papers in three subsections: (1) feature-based effects of arousal, detailing the results of visual search and attentional blink experiments where the emotional stimuli were task-relevant; (2) spatial effects of arousal, detailing the results of spatial cueing and free viewing task with task-relevant emotional content; and (3) facilitating and hindering effects of arousal of task-irrelevant emotional stimuli (on the main task), detailing the results of the experiments where the emotional stimuli were presented shortly before or during the actual task.

Emotional arousal received relatively little attention in the past two decades compared to its equally important counterpart, emotional valence. The aim of the present paper was threefold. First, to fill a gap and systematically review the effects of emotional arousal on visual attentional performance as previous reviews were mostly concerned with the effects of the valence of emotionally charged stimuli (i.e. positive and negative) on visual processing (Gerdes et al., 2014; Olofsson et al., 2008). Second, to clarify how existing arousal theories can explain the effects found on visual attention under various conditions. I sought to examine the effects of task-relevant and task-irrelevant arousing stimuli across various modalities and paradigms relying on behavioral, eye-tracking, and neurological measures. Third, to raise awareness of the fact that emotional arousal (besides and regardless of valence) should be investigated on its own terms. Dimensional theories of emotions propose that valence and arousal are independent dimensions (Bliss-Moreau et al., 2020; Sutherland & Mather, 2018) which is rarely the case in experimental studies (Dan-Glauser & Scherer, 2011; Deák et al., 2010; Lang et al., 1997; Pool et al., 2016). Instead, the two dimensions overlap, and distinguishing their unique effect is often hard (Yiend, 2010). Studies dealing with the effect of one emotional dimension have limited options to account for this. The solution is either to collapse studies across arousal levels (e.g., from low to high) or to control for the level of arousal (e.g., using only moderately arousing stimuli). Here I took a mixed approach, that is, to collapse all studies that examined arousal irrespective of the valence of the stimuli but note valence-dependent associations where possible.

The number of studies found and retained showed that while a lot of papers mention emotional arousal (123), only a handful of these (32, i.e., 26%) systematically addressed how it affects visual search performance. Studies that were not included in this review were rejected mostly for two reasons: (1) they assessed the arousal of the stimuli but did not use it in the analyses or (2) they used the terms interchangeably. The problem is, even if these studies did manipulate arousal, for instance by comparing positively and negatively valenced stimuli that have similar arousal ratings to neutral stimuli, it is impossible to disentangle the effects caused by the arousal of the stimuli used. A reexamination of existing data, similar to a previous study (Lundqvist et al., 2014), would be a good approach to clarify the unique roles of the two emotional dimensions. Further, in future studies, it is important to also report results regarding the arousal level (even null findings) and to make sure that terminology is used in a precise and uniform manner in an effort to make the results comparable to that of other studies.

Overall it seems that the way emotional arousal affects visual search performance is mostly defined by the task relevance of the stimuli that convey the emotion. The findings of this review suggest that task-relevant arousing stimuli (regardless of the modality) draw (evident from e.g., visual search studies) and hold attention (as demonstrated by e.g., free viewing studies), resulting in shorter RTs and increased accuracy. Since such highly salient objects are processed before other stimuli (Mather & Sutherland, 2011; Mather et al., 2016) and a greater capacity is devoted to them (Kahneman, 1973), their evaluation, and consequently, movement initiation happens earlier (Csathó et al., 2008; LeDoux & Daw, 2018; Öhman, 2005). In contrast, when the arousing stimuli are task-irrelevant, the results are mixed and dependent on the presentation and modality used (i.e., visual or auditory) and the stimulus onset properties (i.e., whether the task-irrelevant stimulus precedes the visual attention task, or if it is presented at the same time). When the emotional content precedes the task or music is used during the task, or the content is presented for a long duration (because solving the task takes several seconds), arousal may also increase visual attentional performance. This is in line with previous findings on threatening stimuli (Cohen et al., 2011; O’Toole et al., 2011) and that valence-based attentional biases vary as a function of arousal (Pool et al., 2016). Highly arousing stimuli can increase the available cognitive (working memory) capacity (Nielson et al., 1996; Sakaki et al., 2019) and make attentional movements faster (Bradley et al., 2011; Gomez et al., 2019). When presented as task-irrelevant distractors during the primary task, arousing stimuli impaired task performance. Based on the theories that propose a competition for the limited working memory resources, it seems plausible to claim that the highly arousing stimulus won the competition, and the active inhibition of its processing (and disengagement from it) is necessary to complete the task (Burra et al., 2019; Trujillo et al., 2021). The active suppression of emotional stimuli is harder compared to neutral stimuli, and the cognitive capacity to maintain the suppression throughout the task could be proportional to the arousal the emotional stimuli elicit (Hindi Attar et al., 2010).

While all current arousal theories (Kahneman, 1973; Mather et al., 2016; Mather & Sutherland, 2011; Zsido et al., 2019b) are capable of explaining these results, I found no decisive evidence of whether the arousal-induced increase in capacity is real or just apparent, and how to best explain this phenomenon. A possible explanation behind the mixed result could be the impact of appraisal in processing emotional stimuli (Pool et al., 2016). Such appraisals include the goal relevance, goal congruence, and novelty of the emotion, as well as the perceived agency, and the control or coping potential of the perceiver (Sander et al., 2018). Appraisal theories emphasize the role of individual differences positing that the level of arousal elicited by a particular stimulus can greatly differ between participants in a study (Pool et al., 2016; Sander et al., 2005). The studies presented in this review did not seek to test these aspects of the processing. Further, the objective physiological state of arousal elicited by the stimuli used during the experiment was not measured in most cases. While the emotional stimuli used in such studies are standardized and often participants provided a subjective rating of it after the experiment, these ratings were only used to underscore that emotional stimuli did differ from neutral control stimuli but were not used in the statistical analysis. That is, the variance (or noise in the present case) in data introduced by individual differences in the appraisal was not controlled for. Separating this relevance effect from the arousal effect is difficult, although appraisal theories only posit that stimuli appraised affectively relevant have facilitated access to attention (Brosch et al., 2013; Sander et al., 2003). This is not contradicted by arousal theories arguing that all emotional stimuli could increase arousal (Brosch & Sharma, 2005; Mather et al., 2016; Sutherland & Mather, 2018). Consequently, the fact that often mixed and a wide variety of stimuli were used (positive–negative, erotic-threatening, etc.) to achieve the increase in arousal may be a strength of the studies insofar as it reduces the effect of appraisal and increases the effect of (general) arousal. The inconsistencies in the results could be mended by future research experimentally and systematically testing the various modalities, presentation types, methods used, and the role of appraisal. Similarly, individual differences regarding current mood, anxiety, depression, or the sensitivity of the approach-avoidance system could also play a role in the effect of arousal on visual search performance. Individual differences along these variables are well-documented concerning emotion processing (Fraga et al., 2021; Mueller & Kuchinke, 2016) and attentional bias for threat framework (Koster et al., 2006; McNally, 2018). Yet, the studies in the current review did not analyze such effects. Introducing emotional arousal as a manipulated variable in tasks adds a significant amount of understanding to attentional biases to emotional stimuli, and the possible hindering or facilitating effects of the presence of emotional arousal on task performance. Such studies provided evidence that arousal may have effects independent of valence, and that there is an extended neurological basis for attentional bias toward emotionally charged stimuli. Examining the impact of arousal and valence on visual attention in larger samples would be helpful.

Although this review offered a detailed comparison between the studies included in the literature, there were some limitations. Only studies that investigated visual attentional performance were included, which may not lead to a comprehensive understanding of attentional bias toward emotional arousal and changes in performance elicited by arousing stimuli. I did not analyze effect sizes because of the small amount of similarity between studies. This includes differences in the modality of the stimuli, task design, and the paradigm used. Conducting a more specific search would have resulted in a too-low number of eligible papers. Further, although the anxiety level-dependent changes in attention for negative arousal (i.e., threat) are well-documented (Burra et al., 2019; Kim et al., 2021; Lee et al., 2014; March et al., 2017; Mather & Knight, 2006; Yiend, 2010), I have decided to take a more inclusive approach and focus on arousal in general, regardless of other effects. My conclusions might only apply to non-clinical populations, as the emotional stimuli are processed differently in clinical compared to non-clinical populations (Yiend, 2010). Even with these limitations, the present review highlights the necessity for future research targeting the role and underlying mechanisms of emotional arousal on attentional processes.

Future studies should also focus on the potential effects of individual differences due to the appraisal of the stimuli used and the current mood of the respondent or other personality-related measures (such as anxiety, depression, and the sensitivity of the approach-avoidance system) as there might be great differences in emotion processing along these dimensions (Fraga et al., 2021; Mueller & Kuchinke, 2016). Further, the effects of non-emotional arousal should also be considered, as such stimuli could help separate arousal from other emotional dimensions (especially valence) and these experiments may have further practical relevance. Indeed, by studying cognition and action, the role of arousal is introduced to a completely new branch of research, and the practical effects of how arousal affects visual cognition could be examined.