Abstract
A new method of calculating attentional bias from the dot-probe task measures fluctuations in bias towards and away from emotional stimuli over time using trial level bias score metrics. We assessed the stability and reliability of traditional attentional bias scores and trial level bias score measures of attentional bias across time in two five-block dot-probe task experiments in non-clinical samples. In experiments 1 and 2, both traditional attentional bias scores and trial level bias score measures of attentional bias did not habituate/decrease across time. In general, trial level bias score metrics (i.e., attention bias variability as well as the mean biases toward and away from threat) were more reliable than the traditional attention bias measure. This pattern was observed across both experiments. The traditional bias score, however, did improve in reliability in the later blocks of the fearful face dot-probe task. Although trial level bias score measures did not habituate and were more reliable across blocks, these measures did not correlate with state or trait anxiety. On the other hand, trial level bias score measures were strongly correlated with general reaction time variability—and after controlling for this effect no longer superior in reliability in comparison to the traditional attention bias measure. We conclude that general response variability should be removed from trial level bias score measures to ensure that they truly reflect attention bias variability.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data associated with this manuscript has been shared on the Open Science Framework (OSF) in a project called “The stability and reliability of attentional bias measures in the dot-probe task: Evidence from both traditional mean bias scores and trial-level bias scores” https://doi.org/10.17605/OSF.IO/9M5NB.
Notes
128 participants were recruited, but one participant did not have sufficient data for block 1 of the dot-probe task and was therefore excluded.
Although previous research suggests that there are sex differences in trial level bias score measures of attentional bias (Carlson et al. 2019a), supplementary analyses found no effects of sex in the current dataset (all p ≥ .32). However, this may in part be due to the large percentage of female participants in the current dataset (72%). Thus, participant sex was not included in the analyses reported here.
86 participants were recruited, but one participant did not have sufficient data for block 1 and 2 of the dot-probe task and three did not have STAI score were therefore excluded.
Threatening IAPS images were: 1030, 1080, 1201, 1300, 1930, 2120, 6200, 6260, 6510, 9440; Neutral IAPS images were: 7000, 7004, 2745.1, 7041, 7002, 2383, 5390, 7006, 7510, 7060.
It should be noted that RT variability in these studies was obtained from a very small number of practice trials, which may not be representative of RT variability during the task.
References
Aday, J. S., & Carlson, J. M. (2017). Structural MRI-based measures of neuroplasticity in an extended amygdala network as a target for attention bias modification treatment outcome. Medical Hypotheses, 109, 6–16. https://doi.org/10.1016/j.mehy.2017.09.002.
Aday, J. S., & Carlson, J. M. (2019). Extended testing with the dot-probe task increases test–retest reliability and validity. Cognitive Processing, 20, 65–72. https://doi.org/10.1007/s10339-018-0886-1.
Bar-Haim, Y., Lamy, D., Pergamin, L., Bakermans-Kranenburg, M. J., & van Ijzendoorn, M. H. (2007). Threat-related attentional bias in anxious and nonanxious individuals: A meta-analytic study. Psychological Bulletin, 133(1), 1–24. https://doi.org/10.1037/0033-2909.133.1.1.
Broadbent, D. E., & Broadbent, M. H. P. (1988). Anxiety and attentional bias: State and trait. Cognition and Emotion, 2, 165–183.
Carlson, J. M., Fee, A. L., & Reinke, K. S. (2009). Backward masked snakes and guns modulate spatial attention. Evolutionary Psychology, 7(4), 527–537.
Carlson, J. M., Aday, J. S., & Rubin, D. (2019a). Temporal dynamics in attention bias: Effects of sex differences, task timing parameters, and stimulus valence. Cognition and Emotion, 33(6), 1271–1276. https://doi.org/10.1080/02699931.2018.1536648.
Carlson, J. M., Lehman, B. R., & Thompson, J. L. (2019b). Climate change images produce an attentional bias associated with pro-environmental disposition. Cognitive Processing. https://doi.org/10.1007/s10339-019-00902-5.
Carlson, J. M., & Mujica-Parodi, L. R. (2015). Facilitated attentional orienting and delayed disengagement to conscious and nonconscious fearful faces. Journal of Nonverbal Behavior, 39(1), 69–77. https://doi.org/10.1007/s10919-014-0185-1.
Carlson, J. M., & Reinke, K. S. (2014). Attending to the fear in your eyes: Facilitated orienting and delayed disengagement. Cognition and Emotion, 28(8), 1398–1406. https://doi.org/10.1080/02699931.2014.885410.
Carlson, J. M., Torrence, R. D., & Vander Hyde, M. R. (2016). Beware the eyes behind the mask: The capture and hold of selective attention by backward masked fearful eyes. Motivation and Emotion, 40(3), 498–505. https://doi.org/10.1007/s11031-016-9542-1.
Clarke, P., Marinovic, W., Todd, J., Basanovic, J., Chen, N. T., & Notebaert, L. (2020). What is attention bias variability? Examining the potential roles of attention control and response time variability in its relationship with anxiety.
Cooper, R. M., & Langton, S. R. (2006). Attentional bias to angry faces using the dot-probe task? It depends when you look for it. Behaviour Research and Therapy, 44(9), 1321–1329.
Davis, M. L., Rosenfield, D., Bernstein, A., Zvielli, A., Reinecke, A., Beevers, C. G., et al. (2016). Attention bias dynamics and symptom severity during and following CBT for social anxiety disorder. Journal of Consulting and Clinical Psychology, 84(9), 795–802. https://doi.org/10.1037/ccp0000125.
Draheim, C., Mashburn, C. A., Martin, J. D., & Engle, R. W. (2019). Reaction time in differential and developmental research: A review and commentary on the problems and alternatives. Psychological Bulletin, 145(5), 508–535. https://doi.org/10.1037/bul0000192.
Fox, E. (2002). Processing emotional facial expressions: The role of anxiety and awareness. Cognitive, Affective, & Behavioral Neuroscience, 2(1), 52–63.
Fox, E., & Damjanovic, L. (2006). The eyes are sufficient to produce a threat superiority effect. Emotion, 6(3), 534–539.
Goodhew, S. C., & Edwards, M. (2019). Translating experimental paradigms into individual-differences research: Contributions, challenges, and practical recommendations. Consciousness and Cognition, 69, 14–25. https://doi.org/10.1016/j.concog.2019.01.008.
Gur, R. C., Sara, R., Hagendoorn, M., Marom, O., Hughett, P., Macy, L., et al. (2002). A method for obtaining 3-dimensional facial expressions and its standardization for use in neurocognitive studies. Journal of Neuroscience Methods, 115(2), 137–143.
Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186. https://doi.org/10.3758/s13428-017-0935-1.
Heeren, A., Mogoase, C., Philippot, P., & McNally, R. J. (2015). Attention bias modification for social anxiety: A systematic review and meta-analysis. Clinical Psychology Review, 40, 76–90. https://doi.org/10.1016/j.cpr.2015.06.001.
Herry, C., Bach, D. R., Esposito, F., Di Salle, F., Perrig, W. J., Scheffler, K., et al. (2007). Processing of temporal unpredictability in human and animal amygdala. Journal of Neuroscience, 27(22), 5958–5966. https://doi.org/10.1523/JNEUROSCI.5218-06.2007.
Iacoviello, B. M., Wu, G., Abend, R., Murrough, J. W., Feder, A., Fruchter, E., et al. (2014). Attention bias variability and symptoms of posttraumatic stress disorder. Journal of Traumatic Stress, 27(2), 232–239. https://doi.org/10.1002/jts.21899.
Kappenman, E. S., Farrens, J. L., Luck, S. J., & Proudfit, G. H. (2014). Behavioral and ERP measures of attentional bias to threat in the dot-probe task: Poor reliability and lack of correlation with anxiety. Frontiers in Psychology, 5, 1368. https://doi.org/10.3389/fpsyg.2014.01368.
Koster, E. H., Crombez, G., Verschuere, B., & De Houwer, J. (2004). Selective attention to threat in the dot probe paradigm: Differentiating vigilance and difficulty to disengage. Behaviour Research and Therapy, 42(10), 1183–1192.
Koster, E. H., Verschuere, B., Crombez, G., & Van Damme, S. (2005). Time-course of attention for threatening pictures in high and low trait anxiety. Behaviour Research and Therapy, 43(8), 1087–1098. https://doi.org/10.1016/j.brat.2004.08.004.
Kret, M. E., Jaasma, L., Bionda, T., & Wijnen, J. G. (2016). Bonobos (Pan paniscus) show an attentional bias toward conspecifics’ emotions. Proceedings of the National academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.1522060113.
Kruijt, A.-W., Field, A. P., & Fox, E. (2016). Capturing dynamics of biased attention: New attention bias variability measures the way forward? PLoS ONE, 11(11), e0166600.
Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): affective ratings of pictures and instruction manual. Retrieved from Gainesville, FL.
Lonsdorf, T. B., Juth, P., Rohde, C., Schalling, M., & Ohman, A. (2014). Attention biases and habituation of attention biases are associated with 5-HTTLPR and COMTval158met. Cognitive, Affective, & Behavioral Neuroscience, 14(1), 354–363. https://doi.org/10.3758/s13415-013-0200-8.
Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska directed emotional faces (KDEF). CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet, 91–630.
Macleod, C., Mathews, A., & Tata, P. (1986). Attentional bias in emotional disorders. Journal of Abnormal Psychology, 95(1), 15–20.
Mazidi, M., Vig, K., Ranjbar, S., Ebrahimi, M. R., & Khatibi, A. (2019). Attentional bias and its temporal dynamics among war veterans suffering from chronic pain: Investigating the contribution of post-traumatic stress symptoms. Journal of Anxiety Disorders, 66, 102115. https://doi.org/10.1016/j.janxdis.2019.102115.
Mogg, K., & Bradley, B. P. (1999). Some methodological issues in assessing attentional biases for threatening faces in anxiety: A replication study using a modified version of the probe detection task. Behaviour Research and Therapy, 37(6), 595–604.
Mogg, K., & Bradley, B. P. (2002). Selective orienting of attention to masked threat faces in social anxiety. Behaviour Research and Therapy, 40(12), 1403–1414.
Mogg, K., & Bradley, B. P. (2006). Time course of attentional bias for fear-relevant pictures in spider-fearful individuals. Behaviour Research and Therapy, 44(9), 1241–1250. https://doi.org/10.1016/j.brat.2006.05.003.
Mogg, K., Bradley, B. P., de Bono, J., & Painter, M. (1997). Time course of attentional bias for threat information in non-clinical anxiety. Behaviour Research and Therapy, 35(4), 297–303.
Mogg, K., Bradley, B., Dixon, C., Fisher, S., Twelftree, H., & McWilliams, A. (2000). Trait anxiety, defensiveness and selective processing of threat: An investigation using two measures of attentional bias. Personality and Individual Differences, 28, 1063–1077.
Mogg, K., Waters, A. M., & Bradley, B. (2017). Attention bias modification (ABM): Review of effects of multisession abm training on anxiety and threat-related attention in high-anxious individuals. Clinical Psychological Science. https://doi.org/10.1177/2167702617696359.
Naim, R., Abend, R., Wald, I., Eldar, S., Levi, O., Fruchter, E., et al. (2015). Threat-related attention bias variability and posttraumatic stress. American Journal of Psychiatry. https://doi.org/10.1176/appi.ajp.2015.14121579.
Ohman, A., & Mineka, S. (2001). Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychological Review, 108(3), 483–522.
Pourtois, G., Grandjean, D., Sander, D., & Vuilleumier, P. (2004). Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex, 14(6), 619–633.
Price, R. B., Kuckertz, J. M., Siegle, G. J., Ladouceur, C. D., Silk, J. S., Ryan, N. D., et al. (2015). Empirical recommendations for improving the stability of the dot-probe task in clinical research. Psychological Assessment, 27(2), 365–376. https://doi.org/10.1037/pas0000036.
Puls, S., & Rothermund, K. (2017). Attending to emotional expressions: no evidence for automatic capture in the dot-probe task. Cognition and Emotion. https://doi.org/10.1080/02699931.2017.1314932.
Reutter, M., Hewig, J., Wieser, M. J., & Osinsky, R. (2017). The N2pc component reliably captures attentional bias in social anxiety. Psychophysiology, 54(4), 519–527. https://doi.org/10.1111/psyp.12809.
Salemink, E., van den Hout, M. A., & Kindt, M. (2007). Selective attention and threat: Quick orienting versus slow disengagement and two versions of the dot probe task. Behaviour Research and Therapy, 45(3), 607–615.
Schmukle, S. C. (2005). Unreliability of the dot probe task. European Journal of Personality: Published for the European Association of Personality Psychology, 19(7), 595–605.
Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1970). Manual for the State-Trait Axiety Inventory (Self- Evaluation Questionnaire). Palo Alto, CA: Consulating Psychology Press.
Staugaard, S. R. (2009). Reliability of two versions of the dot-probe task using photographic faces. Psychology Science Quarterly, 51(3), 339–350.
Swick, D., & Ashley, V. (2017). Enhanced attentional bias variability in post-traumatic stress disorder and its relationship to more general impairments in cognitive control. Science Report, 7(1), 14559. https://doi.org/10.1038/s41598-017-15226-7.
Torrence, R. D., Wylie, E., & Carlson, J. M. (2017). The time-course for the capture and hold of visuospatial attention by fearful and happy faces. Journal of Nonverbal Behavior. https://doi.org/10.1007/s10919-016-0247-7.
van Rooijen, R., Ploeger, A., & Kret, M. E. (2017). The dot-probe task to measure emotional attention: A suitable measure in comparative studies? Psychonomic Bulletin Review. https://doi.org/10.3758/s13423-016-1224-1.
Weber, M. A., Morrow, K. A., Rizer, W. S., Kangas, K. J., & Carlson, J. M. (2016). Sustained, not habituated, activity in the human amygdala: A pilot fMRI dot-probe study of attentional bias to fearful faces. Cogent Psychology, 3(1), 1259881.
White, L. K., Britton, J. C., Sequeira, S., Ronkin, E. G., Chen, G., Bar-Haim, Y., et al. (2016). Behavioral and neural stability of attention bias to threat in healthy adolescents. Neuroimage, 136, 84–93. https://doi.org/10.1016/j.neuroimage.2016.04.058.
Zvielli, A., Bernstein, A., & Koster, E. H. (2014). Dynamics of attentional bias to threat in anxious adults: Bias towards and/or away? PLoS ONE, 9(8), e104025. https://doi.org/10.1371/journal.pone.0104025.
Zvielli, A., Bernstein, A., & Koster, E. H. (2015). Temporal dynamics of attentional bias. Clinical Psychological Science, 3(5), 772–788.
Zvielli, A., Vrijsen, J. N., Koster, E. H., & Bernstein, A. (2016). Attentional bias temporal dynamics in remitted depression. Journal of Abnormal Psychology, 125(6), 768–776. https://doi.org/10.1037/abn0000190.
Acknowledgements
Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number R15MH110951. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We would like to thank Jeremy A. Andrzejewski and Taylor Susa, and all the other students in the Cognitive × Affective Behavior & Integrative Neuroscience (CABIN) Lab at Northern Michigan University for assisting in the collection of this data. In addition, we would like to thank three anonymous reviewers and the editor for their helpful feedback on an earlier draft of this manuscript. In particular, reviewer 3′s suggestion to explore the potential influence of general reaction time variability on the reliability of attention bias indices.
Funding
This study was funded by the National Institute of Mental Health of the National Institutes of Health under Award Number R15MH110951.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Joshua Carlson declares that he has no conflict of interest. Lin Fang declares that she has no conflict of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Carlson, J.M., Fang, L. The stability and reliability of attentional bias measures in the dot-probe task: Evidence from both traditional mean bias scores and trial-level bias scores. Motiv Emot 44, 657–669 (2020). https://doi.org/10.1007/s11031-020-09834-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11031-020-09834-6