Skip to main content

Advertisement

Log in

Intraclass correlation coefficients: clearing the air, extending some cautions, and making some requests

  • Published:
Health Services and Outcomes Research Methodology Aims and scope Submit manuscript

Abstract

Intraclass correlation coefficients (ICCs) are frequently employed in health science research, often to assess intrarater and interrater reliability. In many cases, insufficient details are provided about these ICCs and there seem to be misunderstandings about their selection and how they should be interpreted. This paper is intended primarily to provide a clear, accessible description of ICCs, including how they should be selected, interpreted, and reported. Emphasis is given to areas where researchers seem to encounter the greatest conceptual difficulties and to exhibit the greatest misconceptions. Two extended examples are used to support the points being made. Major additional aims of this paper are to raise the awareness of authors, reviewers, and editors concerning the importance of using appropriate ICCs, and to encourage them to ensure that complete and accurate information about ICCs is reported in journal articles. Failure to do so perpetuates a risk that incorrect decisions might be made about matters that are of crucial importance for people’s health.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Brozek, J., Alexander, H.: Components of variance and the consistency of repeated measurements. Res. Q. 18, 152–166 (1947)

    CAS  PubMed  Google Scholar 

  • Carrasco, J.L., Jover, L.: Estimating the generalized concordance correlation coefficient through variance components. Biometrics 59, 849e58 (2003)

    Article  Google Scholar 

  • Fleiss, J.: The Design and Analysis of Clinical Experiments. Wiley, New York (1986)

    Google Scholar 

  • Franco, A., Malhotra, N., Simonovits, G.: Publication bias in the social sciences: unlocking the file drawer. Science 345, 1502–1505 (2014). doi:10.1126/science.1255484

    Article  CAS  PubMed  Google Scholar 

  • Hallgren, K.A.: Computing inter-rater reliability for observational data: an overview and tutorial. Tutor. Quant. Methods Psychol. 8(1), 23–34 (2012)

    Article  PubMed  PubMed Central  Google Scholar 

  • Hayen, A., Dennis, R., Finch, C.: Determining the intra- and inter-observer reliability of screening tools used in sports injury research. J. Sci. Med. Sport 10, 201–210 (2007). doi:10.1016/j.jsams.2006.09.002

    Article  PubMed  Google Scholar 

  • Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B.J., Hróbjartsson, A., Roberts, C., Shoukri, M., Streiner, D.L.: Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J. Clin. Epidemiol. 64, 96–106 (2011). doi:10.1016/j.jclinepi.2010.03.002

    Article  PubMed  Google Scholar 

  • Krebs, D.E.: Declare your ICC type. Phys. Ther. 66, 1431 (1986)

    Article  CAS  PubMed  Google Scholar 

  • Landers, R.N.: Computing intraclass correlations (ICC) as estimates of interrater reliability in SPSS. The Winnower 2, e143518.81744 (2015). doi:10.15200/winn.143518.81744

    Google Scholar 

  • Lee, K.M., Lee, J., Chung, C.Y., Ahn, S., Sung, K.H., Kim, T.W., Lee, H.J., Park, M.S.: Pitfalls and important issues in testing reliability using intraclass correlation coefficients in orthopaedic research. Clin. Orthop. Surg. 4, 149–155 (2012). doi:10.4055/cios.2012.4.2.149

    Article  PubMed  PubMed Central  Google Scholar 

  • McAra, S.J.: Glygeryl trinitrate and toe-brachial indexes in pedal ischaemia. Unpublished Ph.D. thesis. Charles Sturt University, Albury, Australia (2015). http://primo.unilinc.edu.au/CSU:CSU_ALL:dtl_csu80101

  • McGraw, K.O., Wong, S.P.: Forming inferences about some intraclass correlation coefficients. Psychol. Methods 1, 30–46 (1996)

    Article  Google Scholar 

  • Müller, R., Büttner, P.: A critical discussion of intraclass correlation coefficients. Stat. Med. 13, 2465–2476 (1994)

    Article  PubMed  Google Scholar 

  • Nunnally, J.C., Bernstein, I.H.: Psychometric Theory, 3rd edn. McGraw Hill, New York (1994)

    Google Scholar 

  • Portney, L.G., Watkins, M.P.: Foundations of clinical research: applications to practice, 3rd edn. Pearson Education, Upper Saddle River (2009)

    Google Scholar 

  • Rankin, G., Stokes, M.: Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clin. Rehabil. 12, 187–199 (1998)

    Article  CAS  PubMed  Google Scholar 

  • Romanos, M.T., Raspovic, A., Perrin, B.M.: The reliability of toe systolic pressure and the toe brachial index in patients with diabetes. J. Foot Ankle Res. 3, 31 (2010). doi:10.1186/1757-1146-3-31

    Article  PubMed  PubMed Central  Google Scholar 

  • Raykov, T., Dimitrov, D.M., von Eye, A., Marcoulides, G.A.: Interrater agreement evaluation: a latent variable modeling approach. Educ. Psychol. Meas. 73, 512–531 (2013). doi:10.1177/0013164412449016

    Article  Google Scholar 

  • Scanlon, C., Park, K., Mapletoft, D., Begg, L., Burns, J.: Interrater and intrarater reliability of photoplethysmography for measuring toe blood pressure and toe-brachial index in people with diabetes mellitus. J. Foot Ankle Res. 5, 13 (2012). doi:10.1186/1757-1146-5-13

    Article  PubMed  PubMed Central  Google Scholar 

  • Shrout, P.E., Fleiss, J.L.: Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 86, 420–428 (1979)

    Article  CAS  PubMed  Google Scholar 

  • Sonter, J., Chuter, V., Casey, S.: Intratester and intertester reliability of toe pressure measurements in people with and without diabetes performed by podiatric physicians. J. Am. Podiatr. Med. Assoc. 105, 201–208 (2015a). doi:10.7547/0003-0538-105.3.201

    Article  PubMed  Google Scholar 

  • Sonter, J., Sadler, S., Chuter, V.: Inter-rater reliability of automated devices for measurement of toe systolic blood pressure and the toe brachial index. Blood Press. Monit. 20, 47–51 (2015b). doi:10.1097/MBP.0000000000000083

    Article  PubMed  Google Scholar 

  • Tehan, P.E., Bray, A., Chuter, V.H.: Non-invasive vascular assessment in the foot with diabetes: sensitivity and specificity of the ankle brachial index, toe brachial index and continuous wave Doppler for detecting peripheral arterial disease. J. Diabetes Complicat. 30, 155–160 (2015). doi:10.1016/j.jdiacomp.2015.07.019

    Article  PubMed  Google Scholar 

  • Wrobel, J.S., Armstrong, D.G.: Reliability and validity of current physical examination techniques of the foot and ankle. J. Am. Podiatr. Med. Assoc. 98, 197–206 (2008)

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

I thank Michela Betta, Sylvia McAra, Melissa Nott, Rod Pope, Therese Schmid, and two anonymous reviewers for valuable comments and feedback concerning earlier drafts of this paper. Associate Professor Pope also engaged in thought-provoking interchanges, drew my attention to literature about ICCs that I had not been aware of, and shared computer output that provided confirming insights. Dr McAra granted permission to use data that had been acquired as part of her doctoral research.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Robert Trevethan.

Ethics declarations

Conflict of interest

None.

Human and animal rights

This paper does not contain any studies with human participants performed by the author.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Trevethan, R. Intraclass correlation coefficients: clearing the air, extending some cautions, and making some requests. Health Serv Outcomes Res Method 17, 127–143 (2017). https://doi.org/10.1007/s10742-016-0156-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10742-016-0156-6

Keywords

Navigation