Abstract
A variance-components analysis is presented for paired comparisons in terms of three components:s, the scale value of the stimuli;d, a deviation from the linear model specified by the law of comparative judgment; andb, a binomial error component. Estimates are given for each of the three variances, σ 2 s , σ 2 d , and σ 2 b . Several coefficients, analogous to reliability coefficients, based on these three variances are indicated. The techniques are illustrated in a replicated comparison of handwriting specimens.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ayres, L. P.A scale for measuring the quality of handwriting of school children. New York: Russell Sage Foundation, 1912.
Bennett, C. A. and Franklin, N. L.Statistical analysis in chemistry and the chemical industry. New York: Wiley, 1954.
Cochran, W. G. The χ2 test of goodness of fit.Ann. math. Statist., 1952,23, 315–345.
Davies, O. L. (Ed.)Design and analysis of industrial experiments. New York: Hafner, 1954.
Duncan, A. J.Quality control and industrial statistics. Chicago: Richard D. Irwin, 1952.
Fisher, R. A.Statistical methods for research workers. (10th ed.) London: Oliver and Boyd, 1946.
Fisher, R. A. and Yates, F.Statistical tables for biological, agricultural and medical research. New York: Hafner, 1953.
Freeman, M. F. and Tukey, J. W. Transformations related to the angular and the square root.Ann. math. Statist., 1950,21, 607–611.
Hald, A.Statistical tables and formulas. New York: Wiley, 1952.
Kendall, M. G.Rank correlation methods. London: Griffin, 1948.
Kendall, M. G. and Babington-Smith, B. On the method of paired comparisons.Biometrika, 1940,31, 324–345.
Mood, A. M.Introduction to the theory of statistics. New York: McGraw-Hill, 1950.
Mosteller, F. Remarks on the method of paired comparisons. III. A test of significance for paired comparisons when equal standard deviations and equal correlations are assumed.Psychometrika, 1951,16, 207–218.
Thurstone, L. L. Psychophysical analysis.Amer. J. Psychol., 1927,38, 368–389.
Thurstone, L. L. A law of comparative judgment.Psychol. Rev., 1927,34, 273–286.
Tippett, L. H. C.The methods of statistics. (4th ed.) New York: Wiley, 1952.
Author information
Authors and Affiliations
Additional information
This research was jointly supported in part by Princeton University, the Office of Naval Research under contract Nonr-1858(15), and the National Science Foundation under grant NSF G-642, and in part by Educational Testing Service. Reproduction in whole or in part is permitted for any purpose of the United States Government.
Thanks are due to Ledyard Tucker and Frederic Lord for valuable suggestions on the development presented here.
Rights and permissions
About this article
Cite this article
Gulliksen, H., Tukey, J.W. Reliability for the law of comparative judgment. Psychometrika 23, 95–110 (1958). https://doi.org/10.1007/BF02289008
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02289008