Top

Gepubliceerd in:

23-04-2016 | Original Article

Retest reliability of the parameters of the Ratcliff diffusion model

Auteurs: Veronika Lerche, Andreas Voss

Gepubliceerd in: Psychological Research | Uitgave 3/2017

Abstract

In the recent years, there is a growing interest to use the Ratcliff Diffusion Model (1978) for diagnostic purposes as the parameters of the model capture interindividual differences in specific cognitive processes. The parameters are estimated using reaction time data from binary classification tasks. For a potential diagnostic application of parameter values sufficient reliability is a necessary precondition. In two studies, each with two sessions separated by 1 week, the retest reliability of the diffusion model parameters was assessed. In Study 1, 105 participants completed a lexical decision task and a recognition memory task. In Study 2, 128 participants worked on an associative priming task. Results show that the reliability of the main parameters of the Ratcliff Diffusion Model (in particular of the speed of information accumulation and the threshold separation with rs > 0.70 for all three tasks) is satisfying. Besides, we analyzed the influence of the number of trials on the retest reliability using different estimation methods (Kolmogorov–Smirnov, Maximum Likelihood, Chi-square and EZ) and both empirical and simulated datasets.

vorige artikel Post-conflict slowing after incongruent stimuli: from general to conflict-specific

volgende artikel Simple reaction time and size–distance integration in virtual 3D space

Noteworthy, in unpublished studies from our lab, we failed to find correlations of threshold separation with self-reported impulsivity using standard speeded response time tasks (cf. also Stahl et al., 2014). However, when using more difficult tasks that required a long duration of information accumulation (RT > 5 s) weak to moderate correlations emerged.

Words had frequencies below 5 per million (CELEX; Baayen, Piepenbrock, & Gulikers, 1995) and—at the same time—a frequency class of 14 or 15 (online dictionary project of the university of Leipzig in November 2014, see http://wortschatz.unileipzig.de), indicating that the word “der” (“the”) is used 2¹⁴ or 2¹⁵ times as often in German language as our stimuli.

As EZ cannot be applied to datasets with an accuracy rate of 100 %, we applied an edge correction method which has also been used by Wagenmakers et al. (2007): \({\text{accuracy}} = 1 - \frac{1}{2 \times n}\), with n being the number of trials. Similarly, in Study 3 we additionally used a correction for a few datasets due to an observed accuracy of 50 % in one condition (\({\text{accuracy}} = 0.5 + \frac{1}{2 \times n}\)).

Based on the findings by Voss et al. (2013), we did not expect the d-parameter of the diffusion model (Voss et al., 2010) to be influenced by the prime type. Besides, estimation of this parameter requires very high trial numbers (Voss et al., 2010).

Results for non-word targets are also presented in Table 3. Note, that the findings are similar to those reported by Voss et al. (2013).

In each condition, fast-dm requires at least 10 trials (independent of the type of response) for ML and KS estimation and 12 trials (of the same response) for CS estimation (Voss et al., 2015). Thus, for the data of the APT due to the higher number of conditions no retest coefficients could be computed for 32 trials and for CS neither for 48 trials.

Construct-samples is part of fast-dm (Voss et al., 2015). For the simulation of datasets we used a high precision setting of p = 4.

Note that, as already mentioned, it is possible that the contamination in the empiric data is different from the type and amount of contamination that we assumed for the simulation of data. Thus, it could be that the estimation of the drift rate suffers less from contamination than for example the estimation of t ₀ and that not (only) the stability of the parameters is responsible for the different distances between the lines of simulated and empiric data. Our study, thus, only allows getting an approximate idea of the state proportions. A clear disentangling of state and trait proportions would require larger samples of participants and data points.

Results are very similar for Session 2.

Allen, P. A., Lien, M.-C., Ruthruff, E., & Voss, A. (2014). Multitasking and aging: Do older adults benefit from performing a highly practiced task? Experimental Aging Research, 40(3), 280–307.PubMedCrossRef

Arnold, N. R., Bröder, A., & Bayen, U. J. (2015). Empirical validation of the diffusion model for recognition memory and a comparison of parameter-estimation methods. Psychological Research, 79(5), 882–898. doi:10.1007/s00426-014-0608-y.PubMedCrossRef

Aschenbrenner, A. J., Balota, D. A., Gordon, B. A., Ratcliff, R., & Morris, J. C. (2016). A diffusion model analysis of episodic recognition in preclinical individuals with a family history for Alzheimer’s disease: The adult children study. Neuropsychology, 30(2), 225–238. doi:10.1037/neu0000222.PubMedPubMedCentralCrossRef

Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995). The CELEX lexical database (CD-ROM). Linguistic Data Consortium. Philadelphia, PA: University of Pennsylvania.

Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39(3), 445–459.PubMedCrossRef

Bock, O., Baetge, I., & Nicklisch, A. (2014). hroot: Hamburg registration and organization online tool. European Economic Review, 71, 117–120.CrossRef

Borkenau, P., & Ostendorf, F. (2008). NEO-Fünf-Faktoren Inventar: nach Costa u. McCrae; NEO-FFI: Hogrefe, Verlag f. Psychologie.

Bowen, H. J., Spaniol, J., Patel, R., & Voss, A. (2016). A Diffusion Model Analysis of Decision Biases Affecting Delayed Recognition of Emotional Stimuli. Plos ONE, 11(1), 1–20. doi:10.1371/journal.pone.0146769.CrossRef

Dutilh, G., Vandekerckhove, J., Tuerlinckx, F., & Wagenmakers, E.-J. (2009). A diffusion model decomposition of the practice effect. Psychonomic Bulletin & Review, 16(6), 1026–1036. doi:10.3758/16.6.1026.CrossRef

Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: the implicit association test. Journal of Personality and Social Psychology, 74(6), 1464.PubMedCrossRef

Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York: McGraw-Hill.

Hintzman, D. L., & Curran, T. (1997). Comparing retrieval dynamics in recognition memory and lexical decision. Journal of Experimental Psychology: General, 126(3), 228–247. doi:10.1037/0096-3445.126.3.228.CrossRef

Klauer, K. C., & Dittrich, K. (2010). From sunshine to double arrows: An evaluation window account of negative compatibility effects. Journal of Experimental Psychology: General, 139(3), 490.CrossRef

Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-8. University of Florida, Gainesville, FL.

Leite, F. P., & Ratcliff, R. (2011). What cognitive processes drive response biases? A diffusion model analysis. Judgment and Decision Making, 6(7), 651–687.

Lerche, V., & Voss, A. (2016). Model complexity in diffusion modeling: Benefits of making the model more parsimonious. Manuscript submitted for publication.

Lerche, V., Voss, A., & Nagler, M. (2015). How Many Trials are Required for Robust Parameter Estimation in Diffusion Modeling? A Comparison of Different Estimation Algorithms. Manuscript submitted for publication.

Metin, B., Roeyers, H., Wiersema, J. R., van der Meere, J. J., Thompson, M., & Sonuga-Barke, E. (2013). ADHD performance reflects inefficient but not impulsive information processing: A diffusion model analysis. Neuropsychology, 27(2), 193–200. doi:10.1037/a0031533.PubMedCrossRef

Neumann, R., & Strack, F. (2000). Approach and avoidance: the influence of proprioceptive and exteroceptive cues on encoding of affective information. Journal of Personality and Social Psychology, 79(1), 39.PubMedCrossRef

Pe, M. L., Vandekerckhove, J., & Kuppens, P. (2013). A diffusion model account of the relationship between the emotional flanker task and rumination and depression. Emotion, 13(4), 739–747. doi:10.1037/a0031628.PubMedCrossRef

Petrov, A. A., Van Horn, N. M., & Ratcliff, R. (2011). Dissociable perceptual-learning mechanisms revealed by diffusion-model analysis. Psychonomic Bulletin & Review, 18(3), 490–497. doi:10.3758/s13423-011-0079-8 CrossRef

R Core Team. (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/.

Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59–108. doi:10.1037/0033-295x.85.2.59.CrossRef

Ratcliff, R., & Childers, R. (2015). Individual differences and fitting methods for the two-choice diffusion model of decision making. Decision, 2(4), 237–279. doi:10.1037/dec0000030.CrossRef

Ratcliff, R., Gomez, P., & McKoon, G. (2004a). A Diffusion Model Account of the Lexical Decision Task. Psychological Review, 111(1), 159–182. doi:10.1037/0033-295x.111.1.159.PubMedPubMedCentralCrossRef

Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20(4), 873–922. doi:10.1162/neco.2008.12-06-420.PubMedPubMedCentralCrossRef

Ratcliff, R., & McKoon, G. (2015). Aging effects in item and associative recognition memory for pictures and words. Psychology and Aging, 30(3), 669–674.PubMedPubMedCentralCrossRef

Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9(5), 347–356. doi:10.1111/1467-9280.00067.CrossRef

Ratcliff, R., Spieler, D., & McKoon, G. (2000). Explicitly modeling the effects of aging on response time. Psychonomic Bulletin & Review, 7(1), 1–25.CrossRef

Ratcliff, R., Thapar, A., Gomez, P., & McKoon, G. (2004b). A diffusion model analysis of the effects of aging in the lexical-decision task. Psychology and Aging, 19(2), 278.PubMedPubMedCentralCrossRef

Ratcliff, R., Thapar, A., & McKoon, G. (2001). The effects of aging on reaction time in a signal detection task. Psychology and Aging, 16(2), 323.PubMedCrossRef

Ratcliff, R., Thapar, A., & McKoon, G. (2004c). A diffusion model analysis of the effects of aging on recognition memory. Journal of Memory and Language, 50(4), 408–424.CrossRef

Ratcliff, R., Thapar, A., & McKoon, G. (2007). Application of the diffusion model to two-choice tasks for adults 75-90 years old. Psychology and Aging, 22(1), 56–66. doi:10.1037/0882-7974.22.1.56.PubMedPubMedCentralCrossRef

Ratcliff, R., Thapar, A., & McKoon, G. (2010). Individual differences, aging, and IQ in two-choice tasks. Cognitive Psychology, 60(3), 127–157. doi:10.1016/j.cogpsych.2009.09.001.PubMedCrossRef

Ratcliff, R., Thapar, A., & McKoon, G. (2011). Effects of aging and IQ on item and associative memory. Journal of Experimental Psychology: General, 140(3), 464.CrossRef

Ratcliff, R., Thompson, C. A., & McKoon, G. (2015). Modeling individual differences in response time and accuracy in numeracy. Cognition, 137, 115–136. doi:10.1016/j.cognition.2014.12.004.PubMedPubMedCentralCrossRef

Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin & Review, 9(3), 438–481. doi:10.3758/bf03196302.CrossRef

Schmiedek, F., Oberauer, K., Wilhelm, O., Süß, H.-M., & Wittmann, W. W. (2007). Individual differences in components of reaction time distributions and their relations to working memory and intelligence. Journal of Experimental Psychology: General, 136(3), 414–429. doi:10.1037/0096-3445.136.3.414.CrossRef

Schmitz, F., & Voss, A. (2012). Decomposing task-switching costs with the diffusion model. Journal of Experimental Psychology: Human Perception and Performance, 38(1), 222–250. doi:10.1037/a0026003.PubMed

Schmitz, F., & Voss, A. (2014). Components of task switching: A closer look at task switching and cue switching. Acta Psychologica, 151, 184–196. doi:10.1016/j.actpsy.2014.06.009.PubMedCrossRef

Schubert, A.-L., Hagemann, D., Voss, A., Schankin, A., & Bergmann, K. (2015). Decomposing the relationship between mental speed and mental abilities. Intelligence, 51, 28–46. doi:10.1016/j.intell.2015.05.002.CrossRef

Spaniol, J., Madden, D. J., & Voss, A. (2006). A Diffusion Model Analysis of Adult Age Differences in Episodic and Semantic Long-Term Memory Retrieval. Journal of Experimental Psychology: Learning, Memory & Cognition, 32(1), 101–117. doi:10.1037/0278-7393.32.1.101.CrossRef

Spaniol, J., Voss, A., & Grady, C. L. (2008). Aging and emotional memory: Cognitive mechanisms underlying the positivity effect. Psychology and Aging, 23(4), 859–872. doi:10.1037/a0014218.PubMedCrossRef

Stahl, C., Voss, A., Schmitz, F., Nuszbaum, M., Tüscher, O., Lieb, K., & Klauer, K. C. (2014). Behavioral components of impulsivity. Journal of Experimental Psychology: General, 143(2), 850–886. doi:10.1037/a0033981.CrossRef

Stolz, J., Besner, D., & Carr, T. (2005). Implications of measures of reliability for theories of priming: Activity in semantic memory is inherently noisy and uncoordinated. Visual Cognition, 12(2), 284–336.

Stumpf, H., Angleitner, A., Wieck, T., Jackson, D., & Beloch-Till, H. (1985). Deutsche personality research form: Hogrefe.

Vallesi, A., Canalaz, F., Balestrieri, M., & Brambilla, P. (2015). Modulating speed-accuracy strategies in major depression. Journal of Psychiatric Research, 60, 103–108. doi:10.1016/j.jpsychires.2014.09.017.PubMedCrossRef

van Ravenzwaaij, D., Donkin, C., & Vandekerckhove, J. (2016). The EZ Diffusion Model Provides a Powerful Test of Simple Empirical Effects. Manuscript submitted for publication.

van Ravenzwaaij, D., & Oberauer, K. (2009). How to use the diffusion model: Parameter recovery of three methods: Ez, fast-dm, and DMAT. Journal of Mathematical Psychology, 53(6), 463–473. doi:10.1016/j.jmp.2009.09.004.CrossRef

Vandekerckhove, J., & Tuerlinckx, F. (2007). Fitting the Ratcliff diffusion model to experimental data. Psychonomic Bulletin & Review, 14(6), 1011–1026. doi:10.3758/bf03193087.CrossRef

Vandekerckhove, J., & Tuerlinckx, F. (2008). Diffusion model analysis with MATLAB: A DMAT primer. Behavior Research Methods, 40(1), 61–72. doi:10.3758/brm.40.1.61.PubMedCrossRef

Venables, W. N., & Ripley, B. D. (2002). Modern Applied Statistics with S (4th ed.). New York: Springer.CrossRef

Voss, A., Nagler, M., & Lerche, V. (2013a). Diffusion models in experimental psychology. A practical introduction. Experimental Psychology, 60(6), 385–402.PubMedCrossRef

Voss, A., Rothermund, K., & Brandtstädter, J. (2008). Interpreting ambiguous stimuli: Separating perceptual and judgmental biases. Journal of Experimental Social Psychology, 44(4), 1048–1056. doi:10.1016/j.jesp.2007.10.009.CrossRef

Voss, A., Rothermund, K., Gast, A., & Wentura, D. (2013b). Cognitive processes in associative and categorical priming: A diffusion model analysis. Journal of Experimental Psychology: General, 142(2), 536.CrossRef

Voss, A., Rothermund, K., & Voss, J. (2004). Interpreting the parameters of the diffusion model: An empirical validation. Memory & Cognition, 32(7), 1206–1220.CrossRef

Voss, A., & Voss, J. (2007). Fast-dm: A free program for efficient diffusion model analysis. Behavior Research Methods, 39(4), 767–775. doi:10.3758/bf03192967.PubMedCrossRef

Voss, A., & Voss, J. (2008). A fast numerical algorithm for the estimation of diffusion model parameters. Journal of Mathematical Psychology, 52(1), 1–9.CrossRef

Voss, A., Voss, J., & Klauer, K. C. (2010). Separating response-execution bias from decision bias: Arguments for an additional parameter in Ratcliff’s diffusion model. British Journal of Mathematical and Statistical Psychology, 63(3), 539–555. doi:10.1348/000711009x477581.PubMedCrossRef

Voss, A., Voss, J., & Lerche, V. (2015). Assessing Cognitive Processes with Diffusion Model Analyses: A Tutorial based on fast-dm-30. Frontiers in Psychology, 6. doi:10.3389/fpsyg.2015.00336.

Wagenmakers, E.-J., Grasman, R. P. P. P., & Molenaar, P. C. M. (2005). On the relation between the mean and the variance of a diffusion model response time distribution. Journal of Mathematical Psychology, 49(3), 195–204. doi:10.1016/j.jmp.2005.02.003.CrossRef

Wagenmakers, E.-J., Ratcliff, R., Gomez, P., & McKoon, G. (2008a). A diffusion model account of criterion shifts in the lexical decision task. Journal of Memory and Language, 58(1), 140–159. doi:10.1016/j.jml.2007.04.006.PubMedPubMedCentralCrossRef

Wagenmakers, E.-J., van der Maas, H. L. J., Dolan, C. V., & Grasman, R. P. P. P. (2008b). EZ does it! Extensions of the EZ-diffusion model. Psychonomic Bulletin & Review, 15(6), 1229–1235. doi:10.3758/pbr.15.6.1229.CrossRef

Wagenmakers, E.-J., van der Maas, H. L. J., & Grasman, R. P. P. P. (2007). An EZ-diffusion model for response time and accuracy. Psychonomic Bulletin & Review, 14(1), 3–22. doi:10.3758/bf03194023.CrossRef

Weigard, A., & Huang-Pollock, C. (2014). A diffusion modeling approach to understanding contextual cueing effects in children with ADHD. Journal of Child Psychology and Psychiatry, 55(12), 1336–1344. doi:10.1111/jcpp.12250.PubMedPubMedCentralCrossRef

Wessa, M., Kanske, P., Neumeister, P., Bode, K., Heissler, J., & Schönfelder, S. (2010). EmoPics: Subjektive und psychophysiologische Evaluation neuen Bildmaterials für die klinisch-biopsychologische Forschung. Zeitschrift für Klinische Psychologie und Psychotherapie, Supplementum, 1(11), 77.

White, C. N., Ratcliff, R., Vasey, M., & McKoon, G. (2009). Dysphoria and memory for emotional material: A diffusion-model analysis. Cognition and Emotion, 23(1), 181–205. doi:10.1080/02699930801976770.PubMedPubMedCentralCrossRef

White, C. N., Ratcliff, R., Vasey, M. W., & McKoon, G. (2010a). Anxiety enhances threat processing without competition among multiple inputs: A diffusion model analysis. Emotion, 10(5), 662–677. doi:10.1037/a0019474.PubMedCrossRef

White, C. N., Ratcliff, R., Vasey, M. W., & McKoon, G. (2010b). Using diffusion models to understand clinical disorders. Journal of Mathematical Psychology, 54(1), 39–52. doi:10.1016/j.jmp.2010.01.004.PubMedPubMedCentralCrossRef

Whiteside, S. P., & Lynam, D. R. (2001). The five factor model and impulsivity: Using a structural model of personality to understand impulsivity. Personality and Individual Differences, 30(4), 669–689.CrossRef

Wiecki, T. V., Sofer, I., & Frank, M. J. (2013). HDDM: hierarchical bayesian estimation of the drift-diffusion model in python. Frontiers in neuroinformatics, 7.

Yap, M. J., Balota, D. A., Sibley, D. E., & Ratcliff, R. (2012). Individual differences in visual word recognition: Insights from the English Lexicon Project. Journal of Experimental Psychology: Human Perception and Performance, 38(1), 53.PubMed

Yap, M. J., Sibley, D. E., Balota, D. A., Ratcliff, R., & Rueckl, J. (2015). Responding to nonwords in the lexical decision task: Insights from the English Lexicon Project. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(3), 597–613. doi:10.1037/xlm0000064.PubMed

Titel: Retest reliability of the parameters of the Ratcliff diffusion model
Auteurs: Veronika Lerche
Andreas Voss
Publicatiedatum: 23-04-2016
Uitgeverij: Springer Berlin Heidelberg
Gepubliceerd in: Psychological Research / Uitgave 3/2017
Print ISSN: 0340-0727
Elektronisch ISSN: 1430-2772
DOI: https://doi.org/10.1007/s00426-016-0770-5

Bohn Stafleu van Loghum

Deel dit onderdeel of sectie (kopieer de link)

Abstract

Log in om toegang te krijgen

Andere artikelen Uitgave 3/2017

Simple reaction time and size–distance integration in virtual 3D space

Interaction between numbers and size during visual search

Self-perception beyond the body: the role of past agency

That’s a good idea, but let’s keep thinking! Can we prevent our initial ideas from being forgotten as a consequence of thinking of new ideas?

Distinctiveness as a function of spatial expansion in verbal working memory: comment on Kreitz, Furley, Memmert, and Simons (2015)

Investigating the characteristics of “not responding”: backward crosstalk in the PRP paradigm with forced vs. free no-go decisions