For whom, and for what, is experience sampling more accurate than retrospective report?

https://doi.org/10.1016/j.paid.2020.110071Get rights and content

Abstract

The experience sampling method (ESM) is often used in research, and promoted for clinical use, with the rationale that it avoids problematic inaccuracies and biases that attend retrospective measures of mental phenomena. Research suggests that averaged scores from ESM data are more accurate than retrospective ratings. However, it is not known how well individuals can remember information about momentary (rather than averaged) mental states, nor how accurately they estimate the dynamic covariation of these states. Individual differences in retrospective accuracy are also poorly understood. In two pre-registered studies, we examined differences between retrospective memory for stress and self-esteem and data gathered via experience sampling and examined whether alexithymia predicted accuracy. Results of both studies revealed substantial discrepancies between retrospective ratings and ESM ratings, especially for momentary states and their covariation. Alexithymia was positively related to recognition of stress means and variability but unrelated to recall of either stress or self-esteem, their variability, or their covariation. These findings suggest that experience sampling may be more useful than self-report when precise information is needed about the timing of mental states and dynamics among them.

Section snippets

Study 1

The first aim of study 1 was to investigate the extent of the memory-experience gap for momentary stress and self-esteem states, including both aggregated information (average stress and self-esteem) and disaggregated information (moment-by-moment stress and self-esteem states, and dynamic covariance of stress and self-esteem). The second aim was to investigate the relationship of trait alexithymia to the discrepancy between these variables rated in an experience-sampling protocol and

Study 2

The results of Study 1 suggested that individuals high in alexithymia more accurately recognized the mean levels and variability in their stress over time. This finding was opposite of what we hypothesized. We reasoned that this surprising effect may have been due to the recognition-based nature of the memory tasks used in this study. If people high in alexithymia had distinctive stress patterns compared to those with more moderate levels of alexithymia, this may have strengthened their ability

General discussion

The current two studies allow us to quantify the degree to which ESM may be more accurate than retrospective self-report for momentary (disaggregated) psychological data. Participants recognized time plots of their momentary stress and self-esteem data only two-thirds of the time, and roughly 20–30% of the area of participants' drawings of these time plots was “error.” Although this degree of inaccuracy does not indicate an absolute advantage for ESM over self-report, it does suggest that in

CRediT authorship contribution statement

William D. Ellison:Conceptualization, Methodology, Formal analysis, Data curation, Writing - original draft.Alec C. Trahan:Conceptualization, Formal analysis, Investigation, Data curation, Writing - review & editing.Joseph C. Pinzon:Methodology, Formal analysis, Investigation, Writing - review & editing.Megan E. Gillespie:Formal analysis, Investigation, Data curation, Writing - review & editing.Logan M. Simmons:Formal analysis, Investigation, Writing - review & editing.Kendel Y. King:

References (56)

  • A.A. Stone et al.

    Understanding recall of weekly pain from a momentary assessment perspective: Absolute agreement, between- and within-person consistency, and judged change in weekly pain

    Pain

    (2004)
  • N. Vermeulen et al.

    Alexithymia factors and memory performances for neutral and emotional words

    Personality and Individual Differences

    (2009)
  • L.T. Apgáua et al.

    Memory for emotional information and alexithymia: A systematic review

    Dementia & Neuropsychologia

    (2019)
  • L.F. Barrett

    The relationships among momentary emotion experiences, personality descriptions, and retrospective ratings of emotion

    Personality and Social Psychology Bulletin

    (1997)
  • D. Ben-Zeev et al.

    Comparing retrospective reports to real-time/real-place mobile assessments in individuals with schizophrenia and a nonclinical comparison group

    Schizophrenia Bulletin

    (2012)
  • D. Ben-Zeev et al.

    Accuracy of hospitalized depressed patients’ and healthy controls’ retrospective symptom reports: An experience sampling study

    Journal of Nervous and Mental Disease

    (2010)
  • D. Ben-Zeev et al.

    Retrospective recall of affect in clinically depressed individuals and controls

    Cognition and Emotion

    (2009)
  • S.E. Cutler et al.

    Repressive coping style and the experience and recall of emotion: A naturalistic study of daily affect

    Journal of Personality

    (1996)
  • U.-S. Donges et al.

    Alexithymia and automatic processing of emotional stimuli: A systematic review

    Reviews in the Neurosciences

    (2017)
  • U.W. Ebner-Priemer et al.

    Ecological momentary assessment of mood disorders and mood dysregulation

    Psychological Assessment

    (2009)
  • U.W. Ebner-Priemer et al.

    Can retrospective interviews assess instability? A comparison of ambulatory assessment and expert interviews regarding DSM–IV criteria for borderline personality disorde

  • A.J. Fisher et al.

    Enhancing the personalization of psychotherapy with dynamic assessment and modeling

    Assessment

    (2016)
  • A.J. Fisher et al.

    Exploring the idiographic dynamics of mood and anxiety via network analysis

    Journal of Abnormal Psychology

    (2017)
  • G.E. Forsythe et al.

    Computer methods for mathematical computations

    (1977)
  • Y. Ganzach et al.

    The retrospective evaluation of positive and negative affect

    Personality and Social Psychology Bulletin

    (2019)
  • J.A. Gibbons et al.

    The fading affect bias begins within 12 hours and persists for 3 months

    Applied Cognitive Psychology

    (2011)
  • A.T. Gloster et al.

    “I feel better when…”: An analysis of the memory-experience gap for peoples’ estimates of the relationship between health behaviors and experiences

    Psychology & Health

    (2017)
  • L. Hu et al.

    Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives

    Structural Equation Modeling

    (1999)
  • Cited by (22)

    • Momentary versus retrospective reports of alcohol or cannabis use, sexual activity, and their co-occurrence

      2021, Addictive Behaviors
      Citation Excerpt :

      Substance-involved sexual activity is common (Herbenick et al., 2010), but reported associations between substance use and sexual activity may be affected by people’s ability to accurately remember each of these behaviors. Because evidence suggests that momentary data—which assess behavior shortly after it happens and in natural environments—are more accurate than retrospective reports (Ellison et al., 2020; Graham, Catania, Brand, Duong, & Canchola, 2003), researchers can investigate recall biases for behaviors like substance use and sexual activity by comparing reports from retrospective surveys to more proximal assessments from the same participants. Extant literature indicates that substance use and sexual activity may be differentially affected by recall biases.

    View all citing articles on Scopus

    William D. Ellison, Department of Psychology, Trinity University; Alec C. Trahan, Department of Psychology, Trinity University; Joseph C. Pinzon, Department of Psychology, Trinity University; Megan E. Gillespie, Department of Psychology, Trinity University; Logan M. Simmons, Department of Psychology, Trinity University; Kendel Y King, Department of Psychology, Trinity University. Joseph Pinzon is now at the Department of Psychology, Pepperdine University. Megan Gillespie is now at the Independent Living Research Utilization Program, TiRR Memorial Hermann Research Center. We thank Kai de Sequera for assistance in developing study materials. Correspondence concerning this article should be addressed to William D. Ellison, Department of Psychology, Trinity University, One Trinity Place, San Antonio, TX 78212. Email: [email protected].

    View full text