Skip to main content

Advertisement

SpringerLink
Log in

Time and learning efficiency in Internet-based learning: a systematic review and meta-analysis

  • Review
  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Authors have claimed that Internet-based instruction promotes greater learning efficiency than non-computer methods. Objectives Determine, through a systematic synthesis of evidence in health professions education, how Internet-based instruction compares with non-computer instruction in time spent learning, and what features of Internet-based instruction are associated with improved learning efficiency. Data sources We searched databases including MEDLINE, CINAHL, EMBASE, and ERIC from 1990 through November 2008. Study selection and data abstraction We included all studies quantifying learning time for Internet-based instruction for health professionals, compared with other instruction. Reviewers worked independently, in duplicate, to abstract information on interventions, outcomes, and study design. Results We identified 20 eligible studies. Random effects meta-analysis of 8 studies comparing Internet-based with non-Internet instruction (positive numbers indicating Internet longer) revealed pooled effect size (ES) for time −0.10 (p = 0.63). Among comparisons of two Internet-based interventions, providing feedback adds time (ES 0.67, p = 0.003, two studies), and greater interactivity generally takes longer (ES 0.25, p = 0.089, five studies). One study demonstrated that adapting to learner prior knowledge saves time without significantly affecting knowledge scores. Other studies revealed that audio narration, video clips, interactive models, and animations increase learning time but also facilitate higher knowledge and/or satisfaction. Across all studies, time correlated positively with knowledge outcomes (r = 0.53, p = 0.021). Conclusions On average, Internet-based instruction and non-computer instruction require similar time. Instructional strategies to enhance feedback and interactivity typically prolong learning time, but in many cases also enhance learning outcomes. Isolated examples suggest potential for improving efficiency in Internet-based instruction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Bell, D. S., Fonarow, G. C., Hays, R. D., & Mangione, C. M. (2000). Self-study from web-based and printed guideline materials: A randomized, controlled trial among resident physicians. Annals of Internal Medicine, 132, 938–946.

    Google Scholar 

  • Berman, N. B., Fall, L. H., Maloney, C. G., & Levine, D. A. (2008). Computer-assisted instruction in clinical education: A roadmap to increasing CAI implementation. Advances in Health Sciences Education, 13, 373–383.

    Article  Google Scholar 

  • Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A., et al. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79, 1243–1289.

    Article  Google Scholar 

  • Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74, 379–439.

    Article  Google Scholar 

  • Blackmore, C., Tantam, D., & Van Deurzen, E. (2006). The role of the eTutor—evaluating tutor input in a virtual learning community for psychotherapists and psychologists across Europe. International Journal of Psychotherapy, 10(2), 35–46.

    Google Scholar 

  • Chumley-Jones, H. S., Dobbie, A., & Alford, C. L. (2002). Web-based learning: Sound educational method or hype? A review of the evaluation literature. Academic Medicine, 77(10 suppl), S86–S93.

    Article  Google Scholar 

  • Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445–459.

    Google Scholar 

  • Clark, D. (2002). Psychological myths in e-learning. Medical Teacher, 24(6), 598–604.

    Article  Google Scholar 

  • Clark, R. C., & Mayer, R. E. (2007). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (2nd ed.). San Francisco: Pfeiffer.

    Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Cohen, P. A., & Dacanay, L. D. (1992). Computer-based instruction and health professions education: A meta-analysis of outcomes. Evaluation and the Health Professions, 15, 259–281.

    Article  Google Scholar 

  • Cohen, P. A., & Dacanay, L. D. (1994). A meta-analysis of computer-based instruction in nursing education. Computers in Nursing, 12(2), 89–97.

    Google Scholar 

  • Cook, D. A. (2005). The research we still are not doing: An agenda for the study of computer-based learning. Academic Medicine, 80, 541–548.

    Article  Google Scholar 

  • Cook, D. A. (2009). The failure of e-learning research to inform educational practice, and what we can do about it. Medical Teacher, 31, 158–162.

    Article  Google Scholar 

  • Cook, D. A., Beckman, T. J., Thomas, K. G., & Thompson, W. G. (2008a). Adapting Web-based instruction to residents’ knowledge improves learning efficiency: A randomized controlled trial. Journal of General Internal Medicine, 23, 985–990.

    Article  Google Scholar 

  • Cook, D. A., Beckman, T. J., Thomas, K. G., & Thompson, W. G. (2008b). Introducing resident physicians to complexity in ambulatory medicine. Medical Education, 42, 838–848.

    Article  Google Scholar 

  • Cook, D. A., & Dupras, D. M. (2004). A practical guide to developing effective web-based learning. Journal of General Internal Medicine, 19, 698–707.

    Article  Google Scholar 

  • Cook, D. A., Dupras, D. M., Thompson, W. G., & Pankratz, V. S. (2005). Web-based learning in residents’ continuity clinics: A randomized, controlled trial. Academic Medicine, 80, 90–97.

    Article  Google Scholar 

  • Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008c). Internet-based learning in the health professions: A meta-analysis. JAMA, 300, 1181–1196.

    Article  Google Scholar 

  • Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2010). Instructional design variations in internet-based learning for health professions education: A systematic review and meta-analysis. Academic Medicine, 85, 909–922.

    Google Scholar 

  • Cook, D. A., Thompson, W. G., & Thomas, K. G. (2009a). Case-based or non-case-based questions for teaching postgraduate physicians: A randomized crossover trial. Academic Medicine, 84, 1419–1425.

    Article  Google Scholar 

  • Cook, D. A., Thompson, W. G., Thomas, K. G., & Thomas, M. R. (2009b). Lack of interaction between sensing–intuitive learning styles and problem-first versus information-first instruction: A randomized crossover trial. Advances in Health Sciences Education, 14, 79–90.

    Article  Google Scholar 

  • Cook, D. A., Thompson, W. G., Thomas, K. G., Thomas, M. R., & Pankratz, V. S. (2006). Impact of self-assessment questions and learning styles in web-based learning: A randomized, controlled, crossover trial. Academic Medicine, 81, 231–238.

    Article  Google Scholar 

  • Cooper, E. H., & Pantle, A. J. (1967). The total-time hypothesis in verbal learning. Psychological Bulletin, 68, 221–234.

    Article  Google Scholar 

  • Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal ol Experimental Psychology: General, 104, 268–294.

    Article  Google Scholar 

  • Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington Publishers.

    Google Scholar 

  • Curtin, F., Altman, D. G., & Elbourne, D. (2002). Meta-analysis combining parallel and cross-over clinical trials. I: Continuous outcomes. Statistics in Medicine, 21, 2131–2144.

    Article  Google Scholar 

  • Dennis, J. K. (2003). Problem-based learning in online vs. face-to-face environments. Education for Health, 16(2), 198–209.

    Article  Google Scholar 

  • Effective Use of Educational Technology in Medical Education: Summary Report of the 2006 AAMC Colloquium on Educational Technology. (2007). Washington, DC: Association of American Medical Colleges.

  • Fletcher, J. D. (2009). Education and training technology in the military. Science, 323, 72–75.

    Article  Google Scholar 

  • Fordis, M., King, J. E., Ballantyne, C. M., Jones, P. H., Schneider, K. H., Spann, S. J., et al. (2005). Comparison of the instructional efficacy of internet-based CME with live interactive CME workshops: A randomized controlled trial. JAMA, 294, 1043–1051.

    Article  Google Scholar 

  • Fredrick, W. C., & Walberg, H. J. (1980). Learning as a function of time. Journal of Educational Research, 73, 183–194.

    Google Scholar 

  • Friedl, R., Hoppler, H., Ecard, K., Scholz, W., Hannekum, A., Ochsner, W., et al. (2006a). Multimedia-driven teaching significantly improves students’ performance when compared with a print medium. Annals of Thoracic Surgery, 81, 1760–1766.

    Article  Google Scholar 

  • Friedl, R., Hoppler, H., Ecard, K., Scholz, W., Hannekum, A., Oechsner, W., et al. (2006b). Comparative evaluation of multimedia driven, interactive, and case-based teaching in heart surgery. Annals of Thoracic Surgery, 82, 1790–1795.

    Article  Google Scholar 

  • Gijselaers, W. H., & Schmidt, H. G. (1995). Effects of quantity of instruction on time spent on learning and achievement. Educational Research and Evaluation, 1(2), 183–201.

    Article  Google Scholar 

  • Goitein, L., Shanafelt, T. D., Wipf, J. E., Slatore, C. G., & Back, A. L. (2005). The effects of work-hour limitations on resident well-being, patient care, and education in an internal medicine residency program. Archives of Internal Medicine, 165, 2601–2606.

    Article  Google Scholar 

  • Greenhalgh, T. (2001). Computer assisted learning in undergraduate medical education. BMJ, 322, 40–44.

    Article  Google Scholar 

  • Grundman, J., Wigton, R., & Nickol, D. (2000). A controlled trial of an interactive, web-based virtual reality program for teaching physical diagnosis skills to medical students. Academic Medicine, 75(10 Suppl), S47–S49.

    Article  Google Scholar 

  • Higgins, J. P. T., & Green, S. (2008). Cochrane handbook for systematic reviews of interventions 5.0.1 [updated September 2008]. Retrieved May 11, 2009, from http://www.cochrane.org/resources/handbook/index.htm.

  • Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557–560.

    Article  Google Scholar 

  • Kopp, V., Stark, R., & Fischer, M. R. (2008). Fostering diagnostic knowledge through computer-supported, case-based worked examples: Effects of erroneous examples and feedback. Medical Education, 42, 823–829.

    Article  Google Scholar 

  • Kulik, C.-L. C., Kulik, J. A., & Shwalb, B. J. (1986). The effectiveness of computer-based adult education: A meta-analysis. Journal of Educational Computing Research, 2, 235–252.

    Article  Google Scholar 

  • Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76(4), 567–605.

    Article  Google Scholar 

  • Leong, S. L., Baldwin, C. D., & Adelman, A. M. (2003). Integrating web-based computer cases into a required clerkship: Development and evaluation. Academic Medicine, 78, 295–301.

    Article  Google Scholar 

  • Lewis, M. J., Davies, R., Jenkins, D., & Tait, M. I. (2001). A review of evaluative studies of computer-based learning in nursing education. Nurse Education Today, 21, 26–37.

    Article  Google Scholar 

  • Mattheos, N., Nattestad, A., Christersson, C., Jansson, H., & Attstrom, R. (2004). The effects of an interactive software application on the self-assessment ability of dental students. European Journal of Dental Education, 8(3), 97–104.

    Google Scholar 

  • McKimm, J., Jollie, C., & Cantillon, P. (2003). ABC of learning and teaching in medicine: Web based learning. BMJ, 326(7394), 870–873.

    Article  Google Scholar 

  • Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. Quality of reporting of meta-analyses. Lancet, 354, 1896–1900.

    Article  Google Scholar 

  • Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement. Annals of Internal Medicine, 151, 264–269.

    Google Scholar 

  • Nicholson, D. T., Chalk, C., Funnell, W. R. J., & Daniel, S. J. (2006). Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Medical Education, 40(1), 1081–1087.

    Article  Google Scholar 

  • Papa, F. J., Aldrich, D., & Schumacker, R. E. (1999). The effects of immediate online feedback upon diagnostic performance. Academic Medicine, 74(10 suppl), S16–S18.

    Article  Google Scholar 

  • Papa, F. J., Oglesby, M. W., Aldrich, D. G., Schaller, F., & Cipher, D. J. (2007). Improving diagnostic capabilities of medical students via application of cognitive sciences-derived learning principles. Medical Education, 41, 419–425.

    Article  Google Scholar 

  • Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231–244). New York: Russell Sage Foundation.

    Google Scholar 

  • Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of e-learning in medical education. Academic Medicine, 81, 207–212.

    Article  Google Scholar 

  • Ryan, J. (2005). Unintended consequences: The accreditation council for graduate medical education work-hour rules in practice. Annals of Internal Medicine, 143, 82–83.

    Google Scholar 

  • Schittek Janda, M., Tani Botticelli, A., Mattheos, N., Nebel, D., Wagner, A., Nattestad, A., et al. (2005). Computer-mediated instructional video: A randomised controlled trial comparing a sequential and a segmented instructional video in surgical hand wash. European Journal of Dental Education, 9(2), 53–58.

    Article  Google Scholar 

  • Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.

    Article  Google Scholar 

  • Spickard, A. III, Alrajeh, N., Cordray, D., & Gigante, J. (2002). Learning about screening using an online or live lecture: Does it matter? Journal of General Internal Medicine, 17, 540–545.

    Article  Google Scholar 

  • Spickard, A. III., Smithers, J., Cordray, D., Gigante, J., & Wofford, J. L. (2004). A randomised trial of an online lecture with and without audio. Medical Education, 38, 787–790.

    Article  Google Scholar 

  • Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., et al. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA, 283, 2008–2012.

    Article  Google Scholar 

  • Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., et al. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76(1), 93–135.

    Article  Google Scholar 

  • Tunuguntla, R., Rodriguez, O., Ruiz, J. G., Qadri, S. S., Mintzer, M. J., Van Zuilen, M. H., et al. (2008). Computer-based animations and static graphics as medical student aids in learning home safety assessment: A randomized controlled trial. Medical Teacher, 30, 815–817.

    Article  Google Scholar 

  • Wells, G. A., Shea, B., O’Connell, D., Peterson, J., Welch, V., Losos, M., et al. (2007). The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Retrieved March 20, 2010, from http://www.ohri.ca/programs/clinical_epidemiology/oxford.htm.

  • Wutoh, R., Boren, S. A., & Balas, E. A. (2004). eLearning: A review of internet-based continuing medical education. Journal of Continuing Education in the Health Professions, 24, 20–30.

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank Denise M. Dupras, MD, PhD, Patricia J. Erwin, MLS, and Victor M. Montori, MD, MSc for their role in study identification and data abstraction. This work was supported by intramural funds and a Mayo Foundation Education Innovation award. Dr. Levinson is supported in part through the John R. Evans Chair in Health Sciences Education Research. Dr. Cook had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Author information

Authors and Affiliations

  1. Division of General Internal Medicine and Office of Education Research, Mayo Clinic College of Medicine, Baldwin 4-A, 200 First Street SW, Rochester, MN, 55905, USA

    David A. Cook

  2. Division of e-Learning Innovation, McMaster University, Hamilton, ON, Canada

    Anthony J. Levinson & Sarah Garside

Authors
  1. David A. Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anthony J. Levinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sarah Garside
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David A. Cook.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cook, D.A., Levinson, A.J. & Garside, S. Time and learning efficiency in Internet-based learning: a systematic review and meta-analysis. Adv in Health Sci Educ 15, 755–770 (2010). https://doi.org/10.1007/s10459-010-9231-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-010-9231-x

Keywords

Search

Navigation