Abstract
In signal detection theory (SDT), responses are governed by perceptual noise and a flexible decision criterion. Recent criticisms of SDT (see, e.g., Balakrishnan, 1999) have identified violations of its assumptions, and researchers have suggested that SDT fundamentally misrepresents perceptual and decision processes. We hypothesize that, instead, these violations of SDT stem from decision noise: the inability to use deterministic response criteria. In order to investigate this hypothesis, we present a simple extension of SDT—the decision noise model—with which we demonstrate that shifts in a decision criterion can be masked by decision noise. In addition, we propose a new statistic that can help identify whether the violations of SDT stem from perceptual or from decision processes. The results of a stimulus classification experiment—together with model fits to past experiments—show that decision noise substantially affects performance. These findings suggest that decision noise is important across a wide range of tasks and needs to be better understood in order to accurately measure perceptual processes.
Article PDF
Similar content being viewed by others
References
Balakrishnan, J. D. (1998a). Measures and interpretations of vigilance performance: Evidence against the detection criterion. Human Factors, 40, 601–623.
Balakrishnan, J. D. (1998b). Some more sensitive measures of sensitivity and response bias. Psychological Methods. 3, 68–90.
Balakrishnan, J. D. (1999). Decision processes in discrimination: Fundamental misrepresentations of signal detection theory. Journal of Experimental Psychology: Human Perception & Performance. 25, 1189–1206.
Balakrishnan, J. D., & MacDonald, J. A. (2002). Decision criteria do not shift: Reply to Treisman. Psychonomic Bulletin & Review. 9, 858–865.
Benjamin. A. S., Diaz. M., & Wee. S. (2007). Signal detection with criterion variability: Applications to recognition. Manuscript submitted for publication.
Busemeyer, J. R., & Myung, I. J. (1992). An adaptive approach to human decision making: Learning theory, decision theory, and human performance. Journal of Experimental Psychology: General. 121, 177–194.
Diederich, A., & Busemeyer, J. R. (2006). Modeling the effects of payoff on response bias in a perceptual discrimination task: Boundchange, drift-rate-change, or two-stage-processing hypothesis. Perception & Psychophysics. 68 194–207.
Dorfman, D. D., & Biderman, M. (1971). A learning model for a continuum of sensory states. Journal of Mathematical Psychology. 8, 264–284.
Dorfman, D. D., Saslow, C. F., & Simpson, J. C. (1975). Learning models for a continuum of sensory states reexamined. Journal of Mathematical Psychology. 12, 178–211.
Dusoir, A. E. (1974). Thomas and Legge’s matching hypothesis for detection and recognition tasks: Two tests. Perception & Psychophysics, 16, 466–470.
Erev, I. (1998). Signal detection by human observers: A cutoff reinforcement learning model of categorization decisions under uncertainty. Psychological Review. 105, 280–298.
Gold, J. M., Murray, R. F., Bennett, P. J., & Sekuler, A. B. (2000). Deriving behavioral receptive fields for visually completed contours. Current Biology. 10, 663–666.
Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York: Wiley.
Healy, A. F., & Kubovy, M. (1981). Probability matching and the formation of conservative decision rules in a numerical analog of signal detection. Journal of Experimental Psychology: Human Learning & Memory. 7, 344–354.
Jones, M., Love, B. C., & Maddox, W. T. (2006). Recency effects as a window to generalization: Separating decisional and perceptual sequential effects in category learning. Journal of Experimental Psychology: Learning, Memory, & Cognition. 32, 316–332.
Kac, M. (1969). Some mathematical models in science. Science. 166, 695–699.
Kubovy, M., & Healy, A. F. (1977). The decision rule in probabilistic categorization: What it is and how it is learned. Journal of Experimental Psychology: General. 106, 427–446.
Larkin, W. (1971). Response mechanisms in detection experiments. Journal of Experimental Psychology. 91, 140–153.
Lee, W. (1963). Choosing among confusably distributed stimuli with specified likelihood ratios. Perceptual & Motor Skills. 16, 445–467.
Lee, W., & Janke, M. (1964). Categorizing externally distributed stimulus samples for three continua. Journal of Experimental Psychology, 68, 376–382.
Lee, W., & Zentall, T. R. (1966). Factorial effects in the categorization of externally distributed stimulus samples. Perception & Psychophysics, 1, 120–124.
Loh, S., Lamond, N., Dorian, J., Roach, G., & Dawson, D. (2004). The validity of psychomotor vigilance tasks of less than 10-minute duration. Behavior Research Methods, Instruments, & Computers, 36, 339–346.
Lu, Z. L., & Dosher, B. A. (1998). External noise distinguishes attention mechanisms. Vision Research. 38, 1183–1198.
Malmberg, K. J. (2002). On the form of ROCs constructed from confidence ratings. Journal of Experimental Psychology: Learning, Memory, & Cognition. 28, 380–387.
Mueller, S. T. (1998). Reinforcement learning in signal detection. Unpublished master’s thesis, University of Michigan, Ann Arbor.
Mueller, S. T. (2005). PEBL: The psychology experiment building language (Version 0.05) [Computer experiment programming language]. Retrieved August 2005 from pebl.sourceforge.net.
Newsome, W. T., Britten, K. H., & Movshon, J. A. (1989). Neural correlates of a perceptual decision. Nature. 341, 52–54.
Parducci, A., & Sandusky, A. (1965). Distribution and sequence effects in judgment. Journal of Experimental Psychology. 69, 450–459.
Philiastides, M. G., & Sajda, P. (2006). Temporal characterization of the neural correlates of perceptual decision making in the human brain. Cerebral Cortex. 16, 509–518.
R Development Core Team (2006). R: A language and environment for statistical computing (Reference index version 2.6.2). Vienna: R Foundation for Statistical Computing.
Sandusky, A. (1971). Signal recognition models compared for random and Markov presentation sequences. Perception & Psychophysics. 10, 339–347.
Schoeffler, M. S. (1965). Theory for psychophysical learning. Journal of the Acoustical Society of America. 37, 1124–1133.
Thomas, E. A. (1973). On a class of additive learning models: Errorcorrecting and probability matching. Journal of Mathematical Psychology, 10, 241–264.
Thomas, E. A.] (1975). Criterion adjustment and probability matching. Perception & Psychophysics. 18, 158–162.
Treisman, M. (2002). Is signal detection theory fundamentally flawed? A response to Balakrishnan (1998a, 1998b, 1999). Psychonomic Bulletin & Review. 9, 845–857.
Treisman, M., & Williams, T. C. (1984). A theory of criterion setting with an application to sequential dependencies. Psychological Review, 91, 68–111.
Van Zandt, T. (2000). ROC curves and confidence judgments in recognition memory. Journal of Experimental Psychology: Learning, Memory, & Cognition. 26, 582–600.
Ward, L. M. (1973). Use of Markov-encoded sequential information in numerical signal detection. Perception & Psychophysics. 14, 337–342.
Zhang, J., & Mueller, S. T. (2005). A note on ROC analysis and nonparametric estimate of sensitivity. Psychometrika. 70, 145–154.
Zhang, J., Riehle, A., & Requin, J. (1997). Analyzing neuronal processing locus in stimulus-response association tasks. Journal of Mathematical Psychology. 41, 219–236.
Author information
Authors and Affiliations
Corresponding author
Additional information
S.T.M. is now employed at Klein Associates Division of ARA Inc. (Fairborn, OH). Much of the research presented in the present article was carried out while both authors were affiliated with the Department of Psychological and Brain Sciences at Indiana University, Bloomington. This research was supported by NIMH Grant MH12717, and by a postdoctoral fellowship to C.T.W. from the German Academic Exchange Service (DAAD).
Rights and permissions
About this article
Cite this article
Mueller, S.T., Weidemann, C.T. Decision noise: An explanation for observed violations of signal detection theory. Psychonomic Bulletin & Review 15, 465–494 (2008). https://doi.org/10.3758/PBR.15.3.465
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/PBR.15.3.465