Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Article

Test-Retest Reliability of Event-Related Potentials Across Three Tasks

Published Online:August 25, 2021https://doi.org/10.1027/0269-8803/a000286

Abstract

Abstract. Event-related potentials (ERPs) constitute a useful and cost-effective method to assess the neural underpinnings of multiple cognitive processes. ERPs have been used to track changes in cognitive processes in longitudinal and clinical studies. However, few studies have assessed their test-retest reliability (i.e., their consistency across time). Therefore, in the current study, we aimed to assess the test-retest reliability of ERPs (P100, N100, P200, N200, P3b, lateralized readiness potentials) across three tasks. In two assessments separated by approximately 4 months, ERPs were recorded in 26 healthy participants, during two oddball tasks (motor and counting) and a stimulus-response compatibility paradigm. Pearson’s correlations and intraclass correlations were used to assess the test-retest reliability of ERPs. Correlations between ERPs elicited by the three tasks were assessed with Pearson’s correlations. Our analyses revealed moderate to very strong test-retest reliability for most ERP components across the three tasks. Test-retest reliability did not differ between the motor and counting oddball tasks. Most ERPs were also correlated across paradigms. Therefore, these results confirm that ERPs have the potential to be reliable markers to serve as robust assessment tools in longitudinal or clinical studies.

References

  • American EEG Society. (1994). Guideline thirteen: Guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology, 11(1), 111–113. First citation in articleCrossref, Google Scholar

  • Beauducel, A., & Debener, S. (2003). Misallocation of variance in event-related potentials: Simulation studies on the effects of test power, topography, and baseline-to-peak versus principal component quantifications. Journal of Neuroscience Methods, 124(1), 103–112. https://doi.org/10.1016/S0165-0270(02)00381-3 First citation in articleCrossref, Google Scholar

  • Beauducel, A., Debener, S., Brocke, B., & Kayser, J. (2000). On the reliability of augmenting/reducing: Peak amplitudes and principal component analysis of auditory evoked potentials. Journal of Psychophysiology, 14(4), 226–240. https://doi.org/10.1027/0269-8803.14.4.226 First citation in articleLink, Google Scholar

  • Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56(6), 893–897. First citation in articleCrossref, Google Scholar

  • Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561–571. First citation in articleCrossref, Google Scholar

  • Bledowski, C., Prvulovic, D., Hoechstetter, K., Scherg, M., Wibral, M., Goebel, R., & Linden, D. E. J. (2004). Localizing P300 generators in visual target and distractor processing: A combined event-related potential and functional magnetic resonance imaging study. The Journal of Neuroscience, 24(42), 9353–9360. https://doi.org/10.1523/JNEUROSCI.1897-04.2004 First citation in articleCrossref, Google Scholar

  • Brunner, J. F., Hansen, T. I., Olsen, A., Skandsen, T., Håberg, A., & Kropotov, J. (2013). Long-term test-retest reliability of the P3 NoGo wave and two independent components decomposed from the P3 NoGo wave in a visual Go/NoGo task. International Journal of Psychophysiology, 89(1), 106–114. https://doi.org/10.1016/j.ijpsycho.2013.06.005 First citation in articleCrossref, Google Scholar

  • Cassidy, S. M., Robertson, I. H., & O’Connell, R. G. (2012). Retest reliability of event-related potentials: Evidence from a variety of paradigms. Psychophysiology, 49(5), 659–664. https://doi.org/10.1111/j.1469-8986.2011.01349.x First citation in articleCrossref, Google Scholar

  • Coles, M. G. (1989). Modern mind-brain reading: Psychophysiology, physiology, and cognition. Psychophysiology, 26(3), 251–269. First citation in articleCrossref, Google Scholar

  • Conners, C. K., Sitarenios, G., & Ayearst, L. E. (2018). Conners’ Continuous Performance Test. In J. S. KreutzerJ. DeLucaB. CaplanEds., Encyclopedia of clinical neuropsychology (3rd ed., pp. 929–933). Springer International Publishing. First citation in articleGoogle Scholar

  • Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009 First citation in articleCrossref, Google Scholar

  • Diedenhofen, B., & Musch, J. (2015). cocor: A comprehensive solution for the statistical comparison of correlations. PLoS One, 10(4), Article e0121945. https://doi.org/10.1371/journal.pone.0121945 First citation in articleCrossref, Google Scholar

  • Dunn, O. J., & Clark, V. (1971). Comparison of tests of the equality of dependent correlation coefficients. Journal of the American Statistical Association, 66(336), 904–908. https://doi.org/10.1080/01621459.1971.10482369 First citation in articleCrossref, Google Scholar

  • Evans, J. D. (1996). Straightforward statistics for the behavioral sciences. Brooks/Cole Publishing. First citation in articleGoogle Scholar

  • Fallgatter, A. J., Aranda, D. R., Bartsch, A. J., & Herrmann, M. J. (2002). Long-term reliability of electrophysiologic response control parameters. Journal of Clinical Neurophysiology, 19(1), 61–66. First citation in articleCrossref, Google Scholar

  • Fallgatter, A. J., Brandeis, D., & Strik, W. K. (1997). A robust assessment of the NoGo-anteriorisation of P300 microstates in a cued Continuous Performance Test. Brain Topography, 9(4), 295–302. https://doi.org/10.1007/bf01464484 First citation in articleCrossref, Google Scholar

  • Fallgatter, A. J., & Strik, W. K. (1999). The NoGo-anteriorization as a neurophysiological standard-index for cognitive response control. International Journal of Psychophysiology, 32(3), 233–238. https://doi.org/10.1016/S0167-8760(99)00018-5 First citation in articleCrossref, Google Scholar

  • Fisher, R. A. (1921). On the probable error of a coefficient of correlation deduced from a small sample. Metron, 1, 3–32. https://doi.org/citeulike-article-id:2346712 First citation in articleGoogle Scholar

  • Folstein, J. R., & Van Petten, C. (2008). Influence of cognitive control and mismatch on the N2 component of the ERP: A review. Psychophysiology, 45(1), 152–170. https://doi.org/10.1111/j.1469-8986.2007.00602.x First citation in articleGoogle Scholar

  • Ford, J. M., Mathalon, D. H., White, P. M., & Pfefferbaum, A. (2000). Left temporal deficit of P300 in patients with schizophrenia: Effects of task. International Journal of Psychophysiology, 38(1), 71–79. https://doi.org/10.1016/S0167-8760(00)00131-8 First citation in articleCrossref, Google Scholar

  • Freeman, J. B., Ambady, N., Midgley, K. J., & Holcomb, P. J. (2011). The real-time link between person perception and action: Brain potential evidence for dynamic continuity. Social Neuroscience, 6(2), 139–155. https://doi.org/10.1080/17470919.2010.490674 First citation in articleCrossref, Google Scholar

  • Fruehwirt, W., Dorffner, G., Robert, S., Gerstgrasser, M., Grossegger, D., Schmidt, R., Dal-Bianco, P., Ransmayr, G., Garn, H., Waser, M., & Benke, T. (2018). Associations of event-related brain potentials and Alzheimer’s disease severity: A longitudinal study. Progress in Neuro-Psychopharmacology & Biological Psychiatry, 92, 31–38. https://doi.org/10.1016/j.pnpbp.2018.12.013 First citation in articleCrossref, Google Scholar

  • Gratton, G., Coles, M. G., & Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalography and Clinical Neurophysiology, 55(4), 468–484. First citation in articleCrossref, Google Scholar

  • Groves, K., Kennett, S., & Gillmeister, H. (2018). Early visual ERPs show stable body-sensitive patterns over a 4-week test period. PLoS One, 13(2), Article e0192583. https://doi.org/10.1371/journal.pone.0192583 First citation in articleCrossref, Google Scholar

  • Hall, M. H., Schulze, K., Rijsdijk, F., Picchioni, M., Ettinger, U., Bramon, E., Freedman, R., Murray, R. M., & Sham, P. (2006). Heritability and reliability of P300, P50 and duration mismatch negativity. Behavior Genetics, 36(6), 845–857. https://doi.org/10.1007/s10519-006-9091-6 First citation in articleCrossref, Google Scholar

  • Hammerer, D., Li, S. C., Volkle, M., Muller, V., & Lindenberger, U. (2013). A lifespan comparison of the reliability, test-retest stability, and signal-to-noise ratio of event-related potentials assessed during performance monitoring. Psychophysiology, 50(1), 111–123. https://doi.org/10.1111/j.1469-8986.2012.01476.x First citation in articleCrossref, Google Scholar

  • Herrmann, C. S., & Knight, R. T. (2001). Mechanisms of human attention: Event-related potentials and oscillations. Neuroscience & Biobehavioral Reviews, 25(6), 465–476. First citation in articleCrossref, Google Scholar

  • Hill, S. Y., Steinhauer, S., & Locke, J. (1995). Event-related potentials in alcoholic men, their high-risk male relatives, and low-risk male controls. Alcoholism: Clinical and Experimental Research, 19(3), 567–576. First citation in articleCrossref, Google Scholar

  • Hittner, J. B., May, K., & Silver, N. C. (2004). Testing dependent correlations with nonoverlapping variables: A Monte Carlo simulation. The Journal of Experimental Education, 73(1), 53–69. https://doi.org/10.3200/JEXE.71.1.53-70 First citation in articleCrossref, Google Scholar

  • Houston, R. J., & Schlienz, N. J. (2018). Event-related potentials as biomarkers of behavior change mechanisms in substance use disorder treatment. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 3(1), 30–40. https://doi.org/10.1016/j.bpsc.2017.09.006 First citation in articleCrossref, Google Scholar

  • Huster, R. J., Westerhausen, R., Pantev, C., & Konrad, C. (2010). The role of the cingulate cortex as neural generator of the N200 and P300 in a tactile response inhibition task. Human Brain Mapping, 31(8), 1260–1271. https://doi.org/10.1002/hbm.20933 First citation in articleGoogle Scholar

  • Ishihara, S. (1917). Tests for color-blindness. Hongo Harukicho. First citation in articleGoogle Scholar

  • Kayser, J., Tenke, C. E., Gil, R., & Bruder, G. E. (2010). ERP Generator patterns in schizophrenia during tonal and phonetic oddball tasks: Effects of response hand and silent count. Clinical EEG and Neuroscience, 41(4), 184–195. First citation in articleCrossref, Google Scholar

  • Kinoshita, S., Inoue, M., Maeda, H., Nakamura, J., & Morita, K. (1996). Long-term patterns of change in ERPs across repeated measurements. Physiology & Behavior, 60(4), 1087–1092. https://doi.org/10.1016/0031-9384(96)00130-8 First citation in articleCrossref, Google Scholar

  • Kok, A. (1988). Overlap between P300 and movement-related-potentials: A response to Verleger. Biological Psychology, 27(1), 51–58. First citation in articleCrossref, Google Scholar

  • Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012 First citation in articleCrossref, Google Scholar

  • Kutas, M., Kiang, M., & Sweeney, K. (2012). Potentials and paradigms: Event-related brain potentials and neuropsychology. In D. I. MostofskyM. FaustEds., The handbook of the neuropsychology of language (pp. 543–564). Wiley-Blackwell. First citation in articleCrossref, Google Scholar

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. First citation in articleCrossref, Google Scholar

  • Luck, S. J. (2005). An introduction to the event-related potential technique. MIT Press. First citation in articleGoogle Scholar

  • Maeda, H., Morita, K., Nakamura, J., Inoue, M., Kinoshita, S., Kodama, E., Maki, S., & Nakazawa, Y. (1995). Reliability of the task-related component (P3b) of P3 event-related potentials. Psychiatry and Clinical Neurosciences, 49(5–6), 281–286. First citation in articleCrossref, Google Scholar

  • Makeig, S., Jung, T. P., Bell, A. J., Ghahremani, D., & Sejnowski, T. J. (1997). Blind separation of auditory event-related brain responses into independent components. Proceedings of the National Academy of Sciences of the USA, 94(20), 10979–10984. First citation in articleCrossref, Google Scholar

  • Martínez, A., Anllo-Vento, L., Sereno, M. I., Frank, L. R., Buxton, R. B., Dubowitz, D. J., Wong, E. C., Hinrichs, H., Heinze, H. J., & Hillyard, S. A. (1999). Involvement of striate and extrastriate visual cortical areas in spatial attention. Nature Neuroscience, 2, 364–369. https://doi.org/10.1038/7274 First citation in articleCrossref, Google Scholar

  • Mathalon, D. H., Ford, J. M., & Pfefferbaum, A. (2000). Trait and state aspects of P300 amplitude reduction in schizophrenia: A retrospective longitudinal study. Biological Psychiatry, 47(5), 434–449. https://doi.org/10.1016/S0006-3223(99)00277-2 First citation in articleCrossref, Google Scholar

  • Morand-Beaulieu, S., O’Connor, K. P., Blanchet, P. J., & Lavoie, M. E. (2018). Electrophysiological predictors of cognitive-behavioral therapy outcome in Tic disorders. Journal of Psychiatric Research, 105, 113–122. https://doi.org/10.1016/j.jpsychires.2018.08.020 First citation in articleCrossref, Google Scholar

  • Morand-Beaulieu, S., O’Connor, K. P., Richard, M., Sauve, G., Leclerc, J. B., Blanchet, P. J., & Lavoie, M. E. (2016). The impact of a cognitive-behavioral therapy on event-related potentials in patients with Tic disorders or body-focused repetitive behaviors. Frontiers in Psychiatry, 7, Article 81. https://doi.org/10.3389/fpsyt.2016.00081 First citation in articleCrossref, Google Scholar

  • Morgan, K. K., Luu, P., & Tucker, D. M. (2016). Changes in P3b latency and amplitude reflect expertise acquisition in a football visuomotor learning task. PLoS One, 11(4), Article e0154021. https://doi.org/10.1371/journal.pone.0154021 First citation in articleCrossref, Google Scholar

  • Munsters, N. M., van Ravenswaaij, H., van den Boomen, C., & Kemner, C. (2019). Test-retest reliability of infant event related potentials evoked by faces. Neuropsychologia, 126, 20–26. https://doi.org/10.1016/j.neuropsychologia.2017.03.030 First citation in articleCrossref, Google Scholar

  • Parvaz, M. A., Maloney, T., Moeller, S. J., Malaker, P., Konova, A. B., Alia-Klein, N., & Goldstein, R. Z. (2014). Multimodal evidence of regional midcingulate gray matter volume underlying conflict monitoring. NeuroImage: Clinical, 5, 10–18. https://doi.org/10.1016/j.nicl.2014.05.011 First citation in articleCrossref, Google Scholar

  • Polich, J. (2007). Updating P300: An integrative theory of P3a and P3b. Clinical Neurophysiology, 118(10), 2128–2148. https://doi.org/10.1016/j.clinph.2007.04.019 First citation in articleCrossref, Google Scholar

  • Raven, J. C. (1938). Progressive matrices: A perceptual test of intelligence. H. K. Lewis. First citation in articleGoogle Scholar

  • Riesel, A., Weinberg, A., Endrass, T., Meyer, A., & Hajcak, G. (2013). The ERN is the ERN is the ERN? Convergent validity of error-related brain activity across different tasks. Biological Psychology, 93(3), 377–385. https://doi.org/10.1016/j.biopsycho.2013.04.007 First citation in articleCrossref, Google Scholar

  • Roggeveen, A. B., Prime, D. J., & Ward, L. M. (2007). Lateralized readiness potentials reveal motor slowing in the aging brain. The Journals of Gerontology: Series B, 62(2), P78–P84. https://doi.org/10.1093/geronb/62.2.P78 First citation in articleCrossref, Google Scholar

  • Salisbury, D. F., Rutherford, B., Shenton, M. E., & McCarley, R. W. (2001). Button-pressing affects P300 amplitude and scalp topography. Clinical Neurophysiology, 112(9), 1676–1684. First citation in articleCrossref, Google Scholar

  • Sandman, C. A., & Patterson, J. V. (2000). The auditory event-related potential is a stable and reliable measure in elderly subjects over a 3 year period. Clinical Neurophysiology, 111(8), 1427–1437. https://doi.org/10.1016/S1388-2457(00)00320-5 First citation in articleCrossref, Google Scholar

  • Segalowitz, S. J., & Barnes, K. L. (1993). The reliability of ERP components in the auditory oddball paradigm. Psychophysiology, 30(5), 451–459. First citation in articleCrossref, Google Scholar

  • Segalowitz, S. J., Santesso, D. L., Murphy, T. I., Homan, D., Chantziantoniou, D. K., & Khan, S. (2010). Retest reliability of medial frontal negativities during performance monitoring. Psychophysiology, 47(2), 260–270. https://doi.org/10.1111/j.1469-8986.2009.00942.x First citation in articleCrossref, Google Scholar

  • Simon, J. R., & Wolf, J. D. (1963). Choice reaction time as a function of angular stimulus-response correspondence and age. Ergonomics, 6(1), 99–105. https://doi.org/10.1080/00140136308930679 First citation in articleCrossref, Google Scholar

  • Smulders, F. T., Kenemans, J. L., & Kok, A. (1996). Effects of task variables on measures of the mean onset latency of LRP depend on the scoring method. Psychophysiology, 33(2), 194–205. First citation in articleCrossref, Google Scholar

  • Snellen, H. (1862). Probebuchstaben zur Bestimmung der Sehschärfe [Sample letters to determine the visual acuity]. P. W. van de Weijer. First citation in articleGoogle Scholar

  • Steinhauer, S. R., & Hill, S. Y. (1993). Auditory event-related potentials in children at high risk for alcoholism. Journal of Studies on Alcohol and Drugs, 54(4), 408–421. First citation in articleCrossref, Google Scholar

  • Vaz, S., Falkmer, T., Passmore, A. E., Parsons, R., & Andreou, P. (2013). The case for using the repeatability coefficient when calculating test–retest reliability. PLoS One, 8(9), Article e73990. https://doi.org/10.1371/journal.pone.0073990 First citation in articleCrossref, Google Scholar

  • Wachinger, C., Volkmer, S., Bublath, K., Bruder, J., Bartling, J., & Schulte-Körne, G. (2018). Does the late positive component reflect successful reading acquisition? A longitudinal ERP study. NeuroImage: Clinical, 17, 232–240. https://doi.org/10.1016/j.nicl.2017.10.014 First citation in articleCrossref, Google Scholar

  • Walhovd, K. B., & Fjell, A. M. (2002). One-year test–retest reliability of auditory ERPs in young and old adults. International Journal of Psychophysiology, 46(1), 29–40. https://doi.org/10.1016/S0167-8760(02)00039-9 First citation in articleCrossref, Google Scholar

  • Weinberg, A., & Hajcak, G. (2011). Longer term test-retest reliability of error-related brain activity. Psychophysiology, 48(10), 1420–1425. https://doi.org/10.1111/j.1469-8986.2011.01206.x First citation in articleCrossref, Google Scholar

  • Williams, L. M., Simms, E., Clark, C. R., Paul, R. H., Rowe, D., & Gordon, E. (2005). The test-retest reliability of a standardized neurocognitive and neurophysiological test battery: “Neuromarker”. International Journal of Neuroscience, 115(12), 1605–1630. https://doi.org/10.1080/00207450590958475 First citation in articleCrossref, Google Scholar

  • Wronka, E., Kaiser, J., & Coenen, A. M. (2008). The auditory P3 from passive and active three-stimulus oddball paradigm. Acta Neurobiologiae Experimentalis (Warsaw), 68(3), 362–372. First citation in articleGoogle Scholar