Abstract
The purpose of multiple choice tests of medical knowledge is to estimate as accurately as possible a candidate’s level of knowledge. However, concern is sometimes expressed that multiple choice tests may also discriminate in undesirable and irrelevant ways, such as between minority ethnic groups or by sex of candidates. There is little literature to establish whether multiple choice tests may also discriminate against students with specific learning disabilities (SLDs), in particular those with a diagnosis of dyslexia, and whether the commonly-used accommodations allow such students to perform up to their capability. We looked for evidence to help us determine whether multiple choice tests could be relied upon to test all medical students fairly, regardless of disability. We analyzed the mean scores of over 900 undergraduate medical students on eight multiple-choice progress tests containing 1,000 items using a repeated-measures analysis of variance. We included disability, gender and ethnicity as possible explanatory factors, as well as year group. There was no significant difference between mean scores of students with an SLD who had test accommodations and students with no SLD and no test accommodation. Virtually all students were able to complete the tests within the allowed time. There were no significant differences between the mean scores of known minority ethnic groups or between the genders. We conclude that properly-designed multiple-choice tests of medical knowledge do not systematically discriminate against medical students with specific learning disabilities.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington: American Educational Research Association.
Auewarakul, C., Downing, S. M., Jaturatamrong, U., & Praditsuwan, R. (2005). Sources of validity evidence for an internal medicine student evaluation system: an evaluative study of assessment methods. Medical Education, 39, 276–283.
BMA Equal Opportunities Committee. (2007). Disability equality in the medical profession. London: British Medical Association.
BMA Medical Students’ Committee. (2006). Studying medicine with dyslexia [electronic version]. London: British Medical Association. Retrieved 25 February, 2009 from www.bma.org.uk/ap.nsf/content/dyslexia.
Case, S. M., & Swanson, D. B. (2003). Constructing written test questions for the basic and clinical sciences (3rd ed.) [electronic version]. Philadelphia: National Board of Medical Examiners. Retrieved 21 February, 2009 from http://www.nbme.org/PDF/ItemWriting_2003/2003IWGwhole.pdf.
Cole, N. (1997). The ETS gender study: How females and males perform in educational settings [electronic version]. ETS technical report. Retrieved 16 July, 2009 from http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/16/fb/56.pdf.
DeLisa, J. A. (2006). Physicians with disabilities: Why aren’t there more of them? [online article]. Washington, DC: AAMC Reporter (February). Retrieved from http://www.aamc.org/newsroom/reporter/feb06/viewpoint.htm.
Ferguson, E., James, D., & Madeley, L. (2002). Factors associated with success in medical school: Systematic review of the literature. British Medical Journal, 324, 952–957.
Fowell, S. L., Maudsley, G., Maguire, P., Leinster, S. J., & Bligh, J. (2000). Student assessment in undergraduate medical education in the United Kingdom, 1998. Medical Education, 34(s1), 1–49.
Gipps, C., & Murphy, P. (1994). A fair test? Assessment, achievement and equity. Buckingham: Open University Press. ISBN 0335156738.
Haq, I., Higham, J., Morris, R., & Dacre, J. (2005). Effect of ethnicity and gender on performance in undergraduate medical examinations. Medical Education, 39, 1126–1128.
Leinster, S., & Gibson, S. (2008). Dyslexia: A real or perceived problem in medical assessments. Poster presented at the 13th Ottawa international conference on clinical competence, Melbourne. Retrieved 21 February, 2009 from http://www.ozzawa13.com/conferenceabstracts.php, p. 651.
Li, H., & Hamil, C. M. (2003). Writing issues in college students with learning disabilities: A synthesis of the literature from 1990–2000. Learning Disability Quarterly, 26, 29–46.
Lindstrom, J. H. (2007). Determining appropriate accommodations for postsecondary students with reading and written expression disorders. Learning Disabilities Research and Practice, 22(4), 229–236.
McCoubrie, P. (2004). Improving the fairness of multiple-choice questions: A literature review. Medical Teacher, 26(8), 709–712.
McKendree, J. (2001). I have heard that negative marking of multiple-choice questions (MCQs) can have undesirable results. What is the research on this? [online article]. Newcastle upon Tyne: Higher Education Academy Medicine, Dentistry and Veterinary Medicine Subject Centre. Retrieved 12 February, 2009 from http://www.medev.ac.uk/resources/faq/display_single?autonum=24.
McKendree, J. (2002). Negative marking & gender bias. CAA Centre Website. Retrieved 25 February, 2009 from http://www.caacentre.ac.uk/resources/faqs/negative_marking1.shtml.
Medical Schools Council. (2005). Recommendations on selection of medical students with specific learning disabilities including dyslexia [electronic version]. London: Medical Schools Council. Retrieved 25 February, 2009 from http://www.chms.ac.uk/publications.htm#publications.
Mercer, S., & Pinder, R. (2000). Doctors and medical students with disabilities. Medical Education, 34, 962.
Palmer, E. J., & Devitt, P. G. (2007). Assessment of higher order cognitive skills in undergraduate education: Modified essay or multiple choice questions? BMC Medical Education, 7, 49. Retrieved 16 February, 2009 from http://www.biomedcentral.com/1472-6920/7/49.
Rosebraugh, C. J. (2000). Learning disabilities and medical schools. Medical Education, 34, 994–1000.
Schuwirth, L., & van der Vleuten, C. (2004). Different written assessment methods: What can be said about their strengths and weaknesses? Medical Education, 38, 974–979.
Sharp, K., & Earle, S. (2000). Assessment, disability and the problem of compensation. Assessment and Evaluation in Higher Education, 25(2), 191–199.
Sireci, S. G., Scarpati, S. E., & Li, S. (2005). Test accommodations for students with disabilities: An analysis of the interaction hypothesis. Review of Educational Research, 75(4), 457–490.
Specific Learning Disabilities Working Group. (2005). Assessment of dyslexia, dyspraxia, dyscalculia and attention deficit disorder (ADD) in higher education. London: DfES.
Tombleson, P., Fox, R. A., & Dacre, J. A. (2000). Defining the content for the objective structured clinical examination component of the Professional and Linguistic Assessments Board examination: Development of a blueprint. Medical Education, 34, 566–572.
van der Vleuten, C. P. M. (1996). The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education, 1(1), 41–67.
Wass, V., van der Vleuten, C. P. M., Shatzer, J., & Jones, R. (2001). Assessment of clinical competence. The Lancet, 357, 945–949.
Waterfield, J., & West, B. (2006). Inclusive assessment in higher education: A resource for change [electronic version]. Retrieved 25 February, 2009 from University of Plymouth website http://www.plymouth.ac.uk/pages/view.asp?page=10494.
Woolf, K., Haq, I., McManus, C., Higham, J., & Dacre, J. (2008). Exploring the underperformance of male and minority ethnic medical students in first year clinical examinations. Advances in Health Sciences Education, 13, 607–616.
Zuriff, G. E. (2000). Extra examination time for students with learning disabilities: An examination of the maximum potential thesis. Applied Measurement in Education, 13(1), 99–117.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ricketts, C., Brice, J. & Coombes, L. Are multiple choice tests fair to medical students with specific learning disabilities?. Adv in Health Sci Educ 15, 265–275 (2010). https://doi.org/10.1007/s10459-009-9197-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-009-9197-8