Wikipedia is an online encyclopedia consisting of over 6 million English-language entries, which is viewed more than 4.5 million times per hour, making it the seventh most visited site on the internet [
]. While Wikipedia covers a wide range of topics, 30,000 English-language entries are dedicated to medical topics [
], which receive 10 million page-views per day across 286 languages [
]. Readers include patients and their families in search of information to prepare for health provider appointments or to understand information provided in such visits [
]. Many health professionals and students also consult Wikipedia, whether to provide patient care or study [
The nature of Wikipedia, which allows anyone to edit its entries, has raised concerns about health professionals and students using Wikipedia, non-medical professionals relying on it for medical information, and the potential impact on public health [
]. To address these concerns, health professions schools have launched courses to train learners how to critically appraise and edit Wikipedia. For example, at the University of California, San Francisco, faculty offer medical and pharmacy students courses on improving Wikipedia’s quality [
]. At Queen’s University, all first-year medical students are required to take the course,
Critical Appraisal, Research and Lifelong Learning,
a major component of which trains students to critically assess and contribute to Wikipedia’s medical content [
]. The efforts of the Queen’s University students in 2017 amounted to more than 1700 edits on 17 health topics, which were viewed over 3 million times in under a year.
At Queen’s University and other institutions, faculty have leveraged Wikipedia editing initiatives to support teaching evidence-based medicine (EBM) [
]. EBM is the “conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” [
]. Most health professions schools are required to train students in EBM, which includes teaching learners to ask clinical questions, acquire and critically appraise information, and apply the information to a patient’s care based on clinical expertise and patient preferences [
]. Wiki Education (Wiki Edu), a nonprofit started by the Wikimedia Foundation, provides support for Wikipedia educational initiatives across North America. To support initiatives across a variety of disciplines, Wiki Edu provides online dashboards that facilitate course management, track editing progress and impact, and provide access to interactive training materials.
A recent scoping review focused broadly on situating Wikipedia as a health information resource [
]. While valuable, this review provides limited information on educational methods utilized, content covered, and course evaluation techniques used in educational initiatives. Moreover, this review only includes educational initiatives described in the peer reviewed literature, which has been suggested as a narrow approach [
]. Therefore, health professions instructors interested in integrating Wikipedia into their teaching have limited synthesized evidence to guide their training, which could lead to wasted instructor effort and suboptimal learning experiences. Thus, this study reviews online curricular materials and the relevant literature with an eye to provide educators with a detailed and holistic view of current approaches to using Wikipedia in health professions education (HPE) and to identify directions for future educational initiatives and research.
Data collection: Curricular inventory
On December 4, 2019, JC scanned Wiki Edu’s dashboards to identify courses offered to HPE students [
]. A dashboard is a website provided to instructors by Wiki Edu. Dashboards are designed to serve as an information and communication hub for faculty, students, and Wiki Edu support staff engaged in courses. Dashboards, which instructors can tailor, provide tools to coordinate class assignments, track student edits, and serve as a platform to host course instructor and Wiki Edu educational materials (see Fig. 1 in the Electronic Supplementary Material).
JC reviewed all the dashboard titles indexed as:
Communicating Science and
Students in the Health Professions to determine course audience. If the audience was unclear, he reviewed the dashboard’s content. If still unclear, the instructor, institution, and course name were googled to identify the course’s audience.
A dashboard was included if it reported that participating HPE students made at least one edit to a Wikipedia article. Dashboards also had to feature key course details, such as those found on the course timeline, to enable the extraction of relevant course data. We excluded dashboards that only used the dashboard’s tracking features (e.g., counts of student edits), but did not include curricular materials.
In the dashboard review, JC identified those courses that had been offered multiple times (e.g., a course offered in 2018 and 2019). In these cases, we included only the most recent dashboard, rationalizing that the latest iteration potentially reflected iterative lessons learned.
Data collection: Literature review
To identify relevant articles, LM and JC, both with backgrounds in information science, met three times to design search strategies optimized for multiple databases. On December 18, 2019, JC used the collaboratively designed searches to systematically search MEDLINE via Ovid, Embase, CINAHL, ERIC, and Web of Science. Searches combined controlled vocabulary terms and keywords (see Appendix A, in the Electronic Supplementary Material, for searches). Searches were limited to English. No date restrictions were applied. We also handsearched the references of included articles. Citations were managed in EndNote.
For full-text review, we included those articles describing curricular initiatives offered in health professions education (HPE) that were defined as programs leading towards careers in patient care. Articles were excluded if they described interventions using wiki software, but not specifically Wikipedia. We also excluded those using Wikipedia, but not as part of a curriculum or course.
JC and NS independently reviewed article titles and abstracts. For articles that were unclear, LM served as a tiebreaker.
We designed our data extraction tool by drawing on our backgrounds and experiences, which include training in information science and teaching Wikipedia to medical students (JC, LM) and experience in education and community health and prevention research (PM). The tool captured details including, but not limited to, student and instructor type, course content, educational methods, and student outcomes. See
for the data extraction tool. JC and LM independently extracted data from journal articles. JC and PM independently extracted dashboard data. LM, JC, and PM met via conference call to discuss and resolve coding differences.
Synthesis and analysis
We used Google Sheets to generate summary reports and descriptive statistics. We then collectively discussed similarities and differences in the initiatives and brainstormed implications for future educational initiatives and research.
Our findings indicate that Wikipedia editing courses introduce students to EBM skills and provide opportunities to apply these skills. Currently many learners initially encounter EBM in preclinical training [
], which can make it difficult to appreciate its relevance to their future clinician roles [
]. For example, critical appraisal, the process of systematically judging research for trustworthiness and relevance to a particular context [
], is often taught using traditional approaches, such as journal clubs, that can seem disconnected from patient care. In contrast, the authentic experience of editing pages to be viewed by potentially thousands of readers is an intrinsically motivating approach to EBM [
]. However, a minority of courses explicitly referenced EBM, while the vast majority taught one or more EBM skills, speaking to a missed opportunity to make a direct link. Moreover, instructors should consider longitudinally assessing the integration of EBM in HPE such that learners’ knowledge and behaviors are evaluated not only following the Wikipedia course, but also when students engage in clinical practice.
Editing Wikipedia’s medical content requires editors to first interpret biomedical literature and second translate it into plain language interpretable across a spectrum of health literacy levels. These communication skills are similar to those that health professionals must use when sharing information with patients, skills which when effectively deployed have been shown to impact medical outcomes, safety, and patient satisfaction [
]. Several courses embedded health communication content. One course evaluated students’ experience in attempting to simplify their language [
]. Future instructors might draw upon these courses and their targeted efforts on health literacy to further develop HPE students’ communication skills, which have been documented as deficient upon graduation [
In contributing to the formulation and sourcing of Wikipedia entries, HPE students are learning about medical epistemology (how health knowledge is assembled, verified, and effectively communicated). This may guide Wikipedia use in their own practice and better position them to support patients searching and interpreting online medical information. In this process, students attain a greater degree of learner autonomy by gaining an ability to critically assess Wikipedia’s strengths and limits, with a new appreciation for the value of its sources. Moreover, students may gain an ability to rectify and enhance this public source as they have the potential for occasional edits and corrections of Wikipedia. Future directions include examining the extent to which the students’ experiences with Wikipedia have contributed to their learner autonomy, especially around delving into Wikipedia’s source materials, their guidance of patient’s use of Wikipedia, and their contributions to it. A longitudinal study examining the effect different educational methods have on editor and instructor retention (students and professors who continue to improve Wikipedia or continue to teach new course iterations) is also warranted.
Inherent in these courses is the empowering and positioning of students as critics and moderators of knowledge. Indeed, all courses culminated in students critically appraising Wikipedia topics. However, there was variation in how much agency the students received toward that goal. For example, while many courses included peer-evaluation activities, which allowed for epistemic agency and shared evaluation [
], only half offered students opportunities to select the article they edited. Wikipedia offers great potential to instructors seeking methods of engaging learner autonomy in their courses and to medical practitioners who take Wikipedia courses to participate more readily in current medical discourse and to apply current medical knowledge in their clinical practice.
For patients, the principal practical implications of these courses are that the student involvement, with medical professional oversight, extends Wikipedia coverage of medical topics in range and depth, while adhering to standards for format, topics, and evidence. This means that patients have a greater chance of finding their conditions covered and in more detail. By having students follow a standard format, patients will also be better able to navigate the edited articles and other medical entries more readily. Moreover, these articles will present a high level of evidence that has had professional oversight in its selection and presentation. On a smaller scale, certainly, patients have a great chance of encountering a health professional, experienced with and knowledgeable about Wikipedia, who can encourage and guide them in taking advantage of Wikipedia. We propose that this would enable them to be more informed patients, ready to participate in decision-making and other aspects of their health.
While we systematically searched the literature and Wiki Edu’s dashboards, we possibly missed initiatives, including those that may not have used a Wiki Edu dashboard. While the dashboards provided substantial course information, we were unable to capture some materials that were presented as external links or always identify instructors’ professions. We were also unable to extract information on the quality of the final article edits, including how many met Wikipedia’s guidelines.
We believe we identified the most robust sources of information available on these courses. However, we feel instructors would be challenged to adopt a course based on the identified dashboard or articles alone. We recommend course instructors consider making their materials freely available, perhaps by crafting MedEdPORTAL articles or sharing resources via FOAMed, which allow for sharing of detailed curricular materials and would enable instructors to readily examine available materials and make informed judgment on implementing those they deem appropriate. Additionally, future researchers might consider more in-depth qualitative study by interviewing course instructors to better understand the facilitators and barriers to integrating Wikipedia in HPE.
Beyond using Wikipedia as a teaching platform and learning tool, these initiatives aimed to improve the quality of Wikipedia’s medical pages; many of these are heavily accessed, which suggests their impact on public health. This speaks to the social contract between health professionals and society whereby providers have a responsibility to provide accurate scientific evidence and communicate effectively. Thus, ensuring edits are based on high-quality evidence presented in an unbiased neutral manner is a critical outcome for these courses. This public use of student-created health information has implications for Wikipedia and its editors, who also play a role in ensuring Wikipedia’s quality through their edits and by providing quality assessments on each medical topic’s talk page. As HPE instructors continue to offer courses and the volume of student edits increases, editors might consider partnering with courses to collaboratively assess pages, thus easing the workflow for editors and exposing learners to Wiki Project Medicine’s robust assessment system [
We identified a single educational initiative that included a final step of an instructor or expert verifying and adjusting the final product to ensure that the Wikipedia articles were left with high quality contributions [
]. Our results indicate that some instructors included proactive measures to moderate the quality of the final edits, such as encouraging students to interact with and work with Wikipedia’s medical editors, drafting contributions in My Sandbox, assessing contributions for readability and clear language, and peer reviewing suggested edits. Increased implementation of these measures, along with a final review of the live Wikipedia article following the editing initiative by a content expert, can help ensure that students are properly interpreting Wikipedia’s guideline for reliable medical sources. This focus on quality will also improve student experiences and interactions with the Wikipedia volunteer community.
What appears to be missing from the use of Wikipedia editing in HPE is systematic forms of assessment and evaluation to substantiate both the gains in learning and in the quality of Wikipedia entries, and to assess learning in EBM and health communication. This study should inspire current and new instructors using Wikipedia to add these important elements to the curriculum. Through the Wiki Edu dashboard, there is already a common set of measures for words and references added among the HPE courses. To this, common strategies and instruments could be established among the instructors to establish a body of finding across different settings and cohorts, not so much as a means of grading the students, but as a way of comparing the effectiveness of teaching strategies among different populations of students. These might include pre-test and post-test exercises to furnish the students with a means of assessing changes, for example, in (a) how they would approach the use of Wikipedia by professionals and patients; (b) identifying key factors in constructing and communicating medical knowledge; (c) ways of learning a new topic; and (d) expertise on the topic on which they have contributed to Wikipedia, and (e) means of engaging in evidence-based practices. These assessments might also involve an extension of Azzam’s deployment of outside experts [
] to assess a sample of editing changes for their quality, perhaps along dimensions of accuracy, evidence, and clarity. Such an approach, especially with some coordination among initiatives, might add to the evidence-based pedagogical practices behind HPE, which would be consistent with its own educational goals along EBM lines. To facilitate this goal, we encourage the formation of a special interest group in health professions education around the use of Wikipedia, which could lead to activities such as sharing assessment tools and other strategies.