Abstract
The use of computerized psychological assessment is a growing practice among contemporary mental health professionals. Many popular and frequently used paper-and-pencil instruments have been adapted into computerized versions. Although equivalence for many instruments has been evaluated and supported, this issue is far from resolved. This literature review deals with recent research findings that suggest that computer aversion negatively impacts computerized assessment, particularly as it relates to measures of negative affect. There is a dearth of equivalence studies that take into account computer aversion’s potential impact on the measurement of negative affect. Recommendations are offered for future research in this area.
Article PDF
Similar content being viewed by others
References
American Psychological Association (1986).Guidelines for computer-based tests and interpretations. Washington, DC: Author.
Beck, A. T. (1967a).Depression: Causes and treatments. Philadelphia: University of Pennsylvania Press.
Beck, A. T. (1967b).Depression: Clinical, experimental and theoretical aspects. New York: Harper & Row.
Beck, A. T. (1978).Depression inventory. Philadelphia: Center for Cognitive Therapy.
Beck, A. T., &Steer, R. A. (1987).Manual for the revised Beck Depression Inventory. San Antonio, TX: Psychological Corporation.
Beck, A. T., &Steer, R. A. (1988).Manual for the Beck Hopelessness Scale. San Antonio, TX: Psychological Corporation.
Beck, A. T., Steer, R. A., &Brown, G. K. (1996).Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation.
Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., &Erbaugh, J. (1961). An inventory for measuring depression.Archives for General Psychiatry,4, 561–571.
Berger, S. G., Chibnall, J. T., &Gfeller, J. D. (1994). The Category Test: A comparison of computerized and standard versions.Assessment,1, 255–258.
Berger, S. G., Chibnall, J. T., &Gfeller, J. D. (1997). Construct validity of the computerized version of the Category Test.Journal of Clinical Psychology,53, 723–726.
Brown, D. T. (1984). Automated assessment systems in school and clinical psychology: Present status and future directions.School Psychology Review,13, 455–460.
Burke, M. J., &Normand, J. (1987). Computerized psychological testing: Overview and critique.Professional Psychology: Research & Practice,18, 42–51.
Burkhead, E. J., &Sampson, J. P. (1985). Computer-assisted assessment in support of the rehabilitation process.Rehabilitation Counseling Bulletin,28, 262–274.
Carroll, B. J., Feinberg, M., Smouse, P. E., Rawson, S. G., &Greden, J. F. (1981). The Carroll Rating Scale for Depression: I. Development, reliability and validation.British Journal of Psychiatry,138, 194–200.
Choca, J., &Morris, J. (1992). Administering the Category Test by computer: Equivalence of results.Clinical Neuropsychologist,6, 9–15.
Cox, J. L., Holden, J. M., &Sagovsky, R. (1987). Detection of postnatal depression: Development of the 10-item Edinburgh Postnatal Depression Scale.British Journal of Psychiatry,150, 782–786.
Dimock, P. H., &Cormier, P. (1991). The effects of format differences and computer experience on performance and anxiety on a computer-administered test.Measurement & Evaluation in Counseling & Development,24, 119–126.
Dowd, E. T.,Milne, C. R., &Wise, S. L. (1984, August).The Therapeutic Reactance Scale: Development and reliability. Paper presented at the annual convention of the American Psychological Association, Toronto.
Elwood, D. L., &Clark, C. L. (1978). Computer administration of the Peabody Picture Vocabulary Test to young children.Behavior Research Methods & Instrumentation,10, 43–46.
Eysenck, H. J., &Eysenck, S. B. G. (1991).Manual of the Eysenck Personality Scales (EPS Adult). London: Hodder & Stoughton.
Farrell, A. D. (1989). Impact of computers on professional practice: A survey of current practices and attitudes.Professional Psychology: Research & Practice,20, 172–178.
Ford, B. D., Vitelli, R., &Stuckless, N. (1996). The effects of computer versus paper-and-pencil administration on measures of anger and revenge with an inmate population.Computers in Human Behavior,12, 159–166.
Gardner, D. G., Discenza, R., &Dukes, R. L. (1993). The measurement of computer attitudes: An empirical comparison of available scales.Journal of Educational Computing Research,9, 487–507.
George, C. E., Lankford, J. S., &Wilson, S. E. (1992). The effects of computerized versus paper-and-pencil administration on measures of negative affect.Computers in Human Behavior,8, 203–209.
Glaze, R., &Cox, J. L. (1991). Validation of a computerised version of the 10-item (self-rating) Edinburgh Postnatal Depression Scale.Journal of Affective Disorders,22, 73–77.
Golden, C. J. (1987). Computers in neuropsychology. In J. N. Butcher (Ed.),Computerized psychological assessment: A practitioner’s guide (pp. 344–354). New York: Basic Books.
Goodman, W. K., Price, L. H., Rasmussen, S. A., Mazure, C., Fleischmann, R. L., Hill, C. L., Heninger, G. R., &Charney, D. S. (1989a). The Yale-Brown Obsessive Compulsive Scale: I. Development, use and reliability.Archives of General Psychiatry,46, 1006–1011.
Goodman, W. K., Price, L. H., Rasmussen, S. A., Mazure, C., Fleischmann, R. L., Hill, C. L., Heninger, G. R., &Charney, D. S. (1989b). The Yale-Brown Obsessive Compulsive Scale: II. Validity.Archives of General Psychiatry,46, 1006–1011.
Gummow, L. J. (1991). Use of the computer in forensic psychology.American Journal of Forensic Psychology,9, 5–31.
Hakstian, R. A., &McLean, P. D. (1989). Brief screen for depression.Psychological Assessment,1, 139–141.
Hamilton, M. (1959). The assessment of anxiety states by rating.British Journal of Medical Psychology,32, 50–55.
Hamilton, M. (1967). Diagnosis and rating of anxiety.British Journal of Psychiatry,3, 76–79.
Harrell, T. H., Honaker, L. M., Hetu, M., &Oberwager, J. (1987). Computerized versus traditional administration of the Multidimensional Aptitude Battery-Verbal scale: An examination of reliability and validity.Computers in Human Behavior,3, 129–137.
Hathaway, S. R., &McKinley, J. C. (1940). A Multiphasic Personality Schedule (Minnesota): I. Construction of the schedule.Journal of Psychology,10, 249–254.
Hathaway, S. R., &McKinley, J. C. (1943).Manual for the Minnesota Multiphasic Personality Inventory. New York: Psychological Corporation.
Heinssen, R. K, Glass, C. R., &Knight, L. A. (1987). Assessing computer anxiety: Development and validation of the Computer Anxiety Rating Scale.Computers in Human Behavior,3, 49–59.
Hinkle, J. S., Sampson, J. P., &Radonsky, V. (1991). Computer-assisted versus paper-and-pencil assessment of personal problems in a clinical population.Computers in Human Behavior,7, 237–242.
Hofer, P. J. (1985). Developing standards for computerized psychological testing.Computers in Human Behavior,1, 301–315.
Johnson, J. H. (1984). An overview of computerized testing. In M. D. Schwartz (Ed.),Using computers in clinical practice: Psychotherapy and mental health applications (pp. 131–133). New York: Haworth Press.
Katz, L., &Dalby, J. T. (1981). Computer and manual administration of the Eysenck Personality Inventory.Journal of Clinical Psychology,37, 586–588.
King, W. C., &Miles, E. W. (1995). A quasi-experimental assessment of the effect of computerizing noncognitive paper-and-pencil measurements: A test of measurement equivalence.Journal of Applied Psychology,80, 643–651.
Kobak, K. A.,Reynolds, W. M., &Greist, J. H. (1990).Hamilton Anxiety Rating Scale: Computer-administered version. Unpublished computer interview.
Kobak, K. A., Reynolds, W. M., &Greist, J. H. (1993). Development and validation of a computer-administered version of the Hamilton Anxiety Scale.Psychological Assessment,5, 487–492.
Kobak, K. A., Reynolds, W. M., &Greist, J. H. (1994). Computerized and clinician assessment of depression and anxiety: Respondent evaluation and satisfaction.Journal of Personality Assessment,63, 173–180.
Kramer, J. J., &Gutkin, T. B. (1990). School psychology, assessment, and the computer: An analysis of current relationships and future potential. In T. R. Kratochwill (Ed.),Advances in school psychology (Vol. VII, pp. 131–150). Hillsdale, NJ: Erlbaum.
Kratzmeier, H., &Horn, R. (1987).SPM. RAVEN-Matrizen-Test. Standard Progressive Matrices-Manual. Weinheim: Beltz.
Kubinger, K. D., Formann, A. K., &Farkas, M. G. (1991). Psychometric shortcomings of Raven’s Standard Progressive Matrices, in particular for computerized testing.European Review of Applied Psychology,41, 295–300.
Laguna, K., &Babcock, R. L. (1997). Computer anxiety in young and older adults: Implications for human-computer interactions in older populations.Computers in Human Behavior,13, 317–326.
LaLomia, M. J., &Sidowski, J. B. (1993). Measurements of computer anxiety: A review.International Journal of Human-Computer Interaction,5, 239–266.
Lanyon, R. I. (1987). The validity of computer-based personality assessment products: Recommendations for the future.Computers in Human Behavior,3, 225–238.
Lockshin, S. B., &Harrison, K. (1991). Computer-assisted assessment of psychological problems. In A. Ager & S. Bendall (Eds.),Microcomputers and clinical psychology: Issues, applications and future developments (pp. 47–63). Chicester, U.K.: Wiley.
Lukin, M. E., Dowd, E. T., Plake, B. S., &Kraft, R. G. (1985). Comparing computerized versus traditional psychological assessment.Computers in Human Behavior,1, 49–58.
McCormack, H. M., de L. Horne, D. J., &Sheather, S. (1988). Clinical applications of visual analog scales: A critical review.Psychological Medicine,18, 1007–1019.
Meier, S. T. (1985). Computer aversion.Computers in Human Behavior,1, 171–179.
Meier, S. T. (1988). Predicting individual differences in performance on computer-administered tests and tasks: Development of the Computer Aversion Scale.Computers in Human Behavior,4, 175–187.
Meier, S. T., &Lambert, M. E. (1991). Psychometric properties and correlates of three computer aversion scales.Behavior Research Methods, Instruments, & Computers,23, 9–15.
Merten, T., &Ruch, W. (1996). A comparison of computerized and conventional administration of the German versions of the Eysenck Personality Questionnaire and the Carroll Rating Scale for Depression.Personality & Individual Differences,20, 281–291.
Moreland, K. L. (1985). Computer-assisted psychological assessment in 1986: A practical guide.Computers in Human Behavior,1, 221–233.
Neuman, G., &Baydoun, R. (1998). Computerization of paper-andpencil tests: When are they equivalent?Applied Psychological Measurement,22, 71–83.
Nurius, P. S. (1990). A review of automated assessment.Computers in Human Services,6, 265–281.
Nurius, P. S., &Hudson, W. W. (1989). Computers and social diagnosis: The client’s perspective.Computers in Human Services,5, 21–35.
Ogles, B. M., France, C. R., Lunnen, K. M., Bell, M. T., &Goldfarb, M. (1998). Computerized depression screening and awareness.Community Mental Health Journal,34, 27–38.
Pardeck, J. T., &Schulte, R. S. (1990). Computers in social intervention: Implications for professional social work practice and education.Family Therapy,17, 109–121.
Peterson, L., Johannsson, V., &Carlsson, S. G. (1996). Computerized testing in a hospital setting: Psychometric and psychological effects.Computers in Human Behavior,12, 339–350.
Pinsoneault, T. B. (1996). Equivalency of computer-assisted and paperand-pencil administered versions of the Minnesota Multiphasic Personality Inventory-2.Computers in Human Behavior,12, 291–300.
Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population.Applied Psychological Measurement,1, 385–401.
Rasulis, R., Schuldberg, D., &Murtagh, M. (1996). Computer-administered testing with the Rotter Incomplete Sentences Blank.Computers in Human Behavior,12, 497–513.
Reardon, R., &Loughead, T. (1988). A comparison of paper-and-pencil and computer versions of the Self-Directed Search.Journal of Counseling & Development,67, 249–252.
Rosenfeld, R., Dar, R., Anderson, D., Kobak, K., &Greist, J. H. (1992). A computer-administered version of the Yale-Brown Obsessive-Compulsive Scale.Psychological Assessment,4, 329–332.
Rydberg, S., &Höghielm, R. (1974).SPIQ: Snabbt performancetest på intelligens [SPIQ: A fast performance-test for intelligence] [Manual]. Stockholm: Skandinaviska Testförlaget AB.
Schulenberg, S. E. (1998).A comparison between the computerized and paper-and-pencil versions of the Beck Depression Inventory-II. Unpublished master’s thesis, University of South Dakota, Vermillion.
Seibert, P. S., &Ellis, H. C. (1991). A convenient self-referencing mood induction procedure.Bulletin of the Psychonomic Society,29, 121–124.
Simola, S. K., &Holden, R. R. (1992). Equivalence of computerized and standard administration of the Piers-Harris children’s self-concept scale.Journal of Personality Assessment,58, 287–294.
Sjöberg, L., Svensson, E., &Persson, L.-O. (1979). The measurement of mood.Scandinavian Journal of Psychology,20, 1–18.
Smith, R. E. (1963). Examination by computer.Behavioral Science,8, 76–79.
Space, L. G. (1981). The computer as psychometrician.Behavior Research Methods & Instrumentation,13, 595–606.
Spielberger, C. D., Gorsuch, R. L., &Lushene, R. E. (1970).Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press.
Steer, R. A., Rissmiller, D. J., Ranieri, W. F., &Beck, A. T. (1994). Use of the computer-administered Beck Depression Inventory and Hopelessness Scale with psychiatric inpatients.Computers in Human Behavior,10, 223–229.
Sturges, J. W. (1998). Practical use of technology in professional practice.Professional Psychology: Research & Practice,29, 183–188.
Tseng, H.-M., Macleod, H. A., &Wright, P. (1997). Computer anxiety and measurement of mood change.Computers in Human Behavior,13, 305–316.
Turban, D. B., Sanders, P. A., Francis, D. J., &Osburn, H. G. (1989). Construct equivalence as an approach to replacing validated cognitive ability selection tests.Journal of Applied Psychology,74, 62–71.
Vansickle, T. R., &Kapes, J. T. (1993). Comparing paper-pencil and computer-based versions of the Strong-Campbell Interest Inventory.Computers in Human Behavior,9, 441–449.
Velton, E. J. (1968). A laboratory task for induction of mood states.Behavior Research & Therapy,6, 473–482.
Watson, C. G., Thomas, D., &Anderson, P. E. D. (1992). Do computer-administered Minnesota Multiphasic Personality Inventories underestimate booklet-based scores?Journal of Clinical Psychology,48, 744–748.
Author information
Authors and Affiliations
Corresponding author
Additional information
This review is a portion of the first author’s master’s thesis research. The authors acknowledge the contributions of three anonymous reviewers with regard to editing drafts of this manuscript.
Rights and permissions
About this article
Cite this article
Schulenberg, S.E., Yutrzenka, B.A. The equivalence of computerized and paper-and-pencil psychological instruments: Implications for measures of negative affect. Behavior Research Methods, Instruments, & Computers 31, 315–321 (1999). https://doi.org/10.3758/BF03207726
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03207726