Abstract
The vast majority of psychology labs rely on prepackaged software applications (e.g., E-Prime) for the programming of experiments. These programs are often used for stimulus selection, and many use a selection method referred to as random without replacement. We demonstrate how random without replacement deviates from random selection, and we detail selection biases that result. We also demonstrate, in a simple experiment, how these selection biases, if left unchecked, can influence behavior. Recommendations for reducing the impact of these biases on performance when random without replacement is used are discussed.
Article PDF
Similar content being viewed by others
References
Bertelson, P. (1965). Serial choice reaction-time as a function of response versus signal-and-response repetition. Nature, 206, 217–218.
Boyer, M., Destrebecqz, A., & Cleeremans, A. (2005). Processing abstract sequence structure: Learning without knowing, or knowing without learning? Psychological Research, 69, 383–398.
Brysbaert, M. (1991). Algorithms for randomness in the behavioral sciences: A tutorial. Behavior Research Methods, Instruments, & Computers, 23, 45–60.
Castellan, N. J. (1992). Shuffling arrays: Appearances may be deceiving. Behavior Research Methods, Instruments, & Computers, 24, 72–77.
Cohen, J. D., MacWhinney, B., Flatt, M. R., & Provost, J. (1993). PsyScope: A new graphic interactive environment for designing psychology experiments. Behavior Research Methods, Instruments, & Computers, 25, 257–271.
Emerson, P. L., & Tobias, R. D. (1995). Computer program for quasirandom stimulus sequences with equal transition frequencies. Behavior Research Methods, Instruments, & Computers, 27, 88–98.
Gratton, G., Coles, M. G., & Donchin, E. (1992). Optimizing the use of information: Strategic control of activation of responses. Journal of Experimental Psychology: General, 121, 480–506.
Haber, R. N. (1966). Nature of the effect of set on perception. Psychological Review, 73, 335–351.
Logan, G. D., & Zbrodoff, N. J. (1982). Constraints on strategy construction in a speeded discrimination task. Journal of Experimental Psychology: Human Perception & Performance, 8, 502–520.
Nissen, M. J., & Bullemer, P. (1987). Attentional requirements of learning: Evidence from performance measures. Cognitive Psychology, 19, 1–32.
Psychology Software Tools (2002). E-Prime (Version 1.2) [Computer software]. Pittsburgh, PA: Author.
Remillard, G., & Clark, J. M. (1999). Generating fixed-length sequences satisfying any given nth-order transition probability matrix. Behavior Research Methods, Instruments, & Computers, 31, 235–243.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by a Natural Sciences and Engineering Research Council of Canada (NSERC) postdoctoral fellowship to C.B.
Rights and permissions
About this article
Cite this article
Blais, C. Random without replacement is not random: Caveat emptor. Behavior Research Methods 40, 961–968 (2008). https://doi.org/10.3758/BRM.40.4.961
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BRM.40.4.961