Integrating Wikipedia editing into health professions education: a curricular inventory and review of the literature

Instructors included physicians, librarians, nurses, and pharmacists. In two instances, students served as instructors [28, 44]. While we limited our analysis to the most recent dashboard iterations, 16 instructors had previously offered courses. For example, Lebowitz offered a course in 2017, 2018, and 2019 [36]. All courses were supported by Wiki Edu staff members, which was associated with the use of the dashboard.

Wiki Edu dashboards are intended to support courses for training students to edit Wikipedia’s health pages. However, several courses integrated Wikipedia assignments as an approach for students to apply their learning of medical topics. For example, the course, Foundations of Clinical Trauma Psychology, which focused on trauma disorders and treatments, incorporated a Wikipedia assignment to allow students to apply and broadly share their newly acquired knowledge [26]. In a course on medical-related linguistic translation, Wikipedia was specifically used to provide students opportunities to apply their medical translation skills and contribute to Wikipedia’s Spanish-language version [38]. In this regard, several instructors highlighted Wikipedia as a conduit for students’ work to have impact beyond the classroom. To facilitate this goal, three instructors addressed health literacy in their courses to ensure students’ Wikipedia edits would be appropriate for its public audience [23, 24, 28]. For example, one course contained the lecture, “Writing for the Public, Considerations About Health Literacy,” and provided learners a readability calculator and guide for avoiding medical jargon [24].

Instructors delivered content through videos, live lectures, guest speakers, and Wiki Edu modules. For example, four instructors incorporated guest lecturers with three being librarian-led [24, 28, 29, 36]. All course dashboards included Wiki Edu modules, which were primarily presented as text documents, but also included videos and quizzes. We identified 44 distinct Wiki Edu modules across the dashboards, including those focused on the mechanics of editing Wikipedia, such as “Contributing Images and Media Files” (included in 25 courses) and “Editing” and “Drafting in the Sandbox” (14), while other modules covered broader topics, such as “Peer Review” (21). (See Appendix B, in the Electronic Supplementary Material, for the modules included.) All courses include the “Wikipedia Policies” module, which introduced the overall ethos of Wikipedia and its “five pillars,” which encourages editing from a neutral point of view and treating all editors with respect and civility [50].

Seventeen instructors customized their dashboards to orient students to the course. For example, five instructors supplemented Wiki Edu modules in their orientation materials with video and audio files, which often featured the instructors themselves [23, 24, 28, 31, 36]. Four instructors required full-day, on-site orientation sessions [23, 24, 28, 36].

Two dashboards specifically mentioned EBM [27, 34]. However, through Wiki Edu modules and additional instructor efforts, such as guest speakers and library resource guides, learners were trained in EBM skills, including to identify and articulate knowledge gaps (23 courses) and acquire (8) [23‐25, 28, 31, 32, 34, 36], appraise (26), and apply (26) information.

Four instructors in five courses linked or embedded additional resources in the dashboards, such as cloud apps (e.g., GoogleDocs) for document sharing [21‐23, 28, 31], while five instructors utilized video conferencing tools (e.g., Zoom) to host synchronous online sessions and digital office hours [23, 24, 28, 31, 36]. One instructor linked to their institution’s learning management system [47]. All courses utilized the Wikipedia-specific “My Sandbox” feature, which is a tool offered by Wikipedia that allows students to practice drafting edits in a simulated Wikipedia environment, prior to making live changes to Wikipedia. In addition to providing a low-stakes environment for practice editing, instructors encouraged or in some cases required students to request peer review of their edits while they were in My Sandbox.

All courses assigned students to edit a Wikipedia article required the application of EBM skills to identify gaps in a page’s content, acquire and appraise information to fill the gap, and then apply that information to make Wikipedia edits. Students primarily completed assignments independently. However, in nine courses students could work independently or in groups [21, 22, 27, 32, 36, 41, 42, 45, 47] and in four, instructors had students work in groups [25, 30, 33, 34]. In two courses we were unable to make such a determination [26, 44]. In 12 courses, students self-selected their article to edit. To guide their choice, seven instructors assigned the Wiki Edu modules “Finding Your Article” [21, 23, 28, 31, 38, 39, 44] and four assigned “Choose Your Article” [21, 38, 39, 47]. Fourteen instructors required students to select articles based on the course topic and/or select from an instructor-provided list. A single instructor assigned students an article [25].

Students were also frequently assigned to peer review fellow students’ work (24 courses) and to respond to peer comments (21), which was supported by a Wiki Edu module (21) and/or Wiki Edu’s Peer Review Rubric [23, 28, 31, 37, 51]. Fourteen courses also required that students make final presentations describing their edits. Less common assignments included: reflective essays [27, 30, 32, 39, 43], work-in-progress reports [23, 24, 28, 31, 36, 42], and requesting students nominate work for the “Did you know?” feature on Wikipedia [22, 42, 45, 46].

Beyond assignments, dashboards provided limited details on assessments of student learning (apart from self-assessments in course evaluations). However, there was evidence of overall course evaluations and evaluations of each course’s impact on Wikipedia. For example, five courses included end-of-course feedback sessions [23, 24, 28, 31, 36] and in one course students completed a final survey [32]. The 27 dashboards reflecting the work of 737 HPE students, indicated that 15,613 edits had been made on 477 articles, including the addition of 4,682 references and 42 images, which had been viewed 42.2 million times as of January 23, 2020.

All instructors blended in-person and online elements (e.g., hands-on skills labs, recorded lectures). For example, the three courses reported by Joshi used teleconference software to enable students to remotely attend weekly meetings [48]. Except for Badgett’s course, which was offered before Wiki Edu’s founding, all courses utilized Wiki Edu modules and dashboards. Evenstein’s course also described using the institution’s learning management system, Moodle, to organize course material. Of note, all courses focused on the English-language version of Wikipedia, except for Evenstein’s, which edited the Hebrew-language Wikipedia [49].

Instructors required students to complete a Wikipedia editing assignment (Tab. 3). Students were provided varying levels of autonomy in selecting a Wikipedia article to edit. For those that were prescriptive, article selection was commonly related to the course’s topic. As a component of the editing assignment, three courses required students to peer review each other’s edits [13, 48, 49]. We propose that by completing these assignments students had the opportunity to practice EBM skills.

Instructors analyzed student Wikipedia edits to assess student learning, while utilizing course surveys, focus groups, and interviews to evaluate their students and courses (which included student self-assessments of learning) (Tab. 3). Primarily surveys queried students’ satisfaction, but also self-perceived knowledge of Wikipedia and intentions to continue editing. Apollonio surveyed students at three points in the course using knowledge questions to objectively gauge increases in students’ knowledge of Wikipedia’s editing processes [12].

Students reported overall satisfaction with the courses and self-perceived improvements in their knowledge and skills. For example, the majority of students in the three courses described by Joshi self-reported increases in their ability to practice EBM [48]. Apollonio, based on pre-post surveys, concluded that her students’ knowledge of Wikipedia editing improved by course end [12]. Authors also analyzed students’ survey free-text responses. For example, by analyzing write-in comments, Murray identified course elements that the students appreciated and those identified as barriers, such as difficulties engaging with the broader Wikipedia editor community and students’ hesitancy to make live edits [13]. Additionally, Evenstein’s students described appreciating the opportunity to give back to the community via their edits, but also expressed that certain course elements (e.g., insufficient time dedicated to learning particular tasks) hindered the experience [49]. Interview and focus group data presented a similar mixed picture. Azzam’s students described difficulties in ensuring the readability of their edits and lamented that editing was more work than expected, but also highlighted that “it feels good to contribute to Wikipedia” and that they appreciated the potential global significance of their edits [11].

Four instructors analyzed the number of student edits made and references added or deleted as a measure of the quality of their learning, as well as their potential impact on Wikipedia [11‐14]. Instructors also tracked the durability of student edits with regard to whether the edits were reversed and used this as an indicator of quality edits. In two notable instances, the instructors went further. Azzam utilized natural language processing software to compute a “readability” score of Wikipedia entries before and after student edits to evaluate students’ ability to apply lessons provided on health literacy and communication skills [11]. Badgett evaluated student edits based on the publication types of the sources (e.g., systematic reviews) the students used to support their edits [14].

Beyond using Wikipedia as a teaching platform and learning tool, these initiatives aimed to improve the quality of Wikipedia’s medical pages; many of these are heavily accessed, which suggests their impact on public health. This speaks to the social contract between health professionals and society whereby providers have a responsibility to provide accurate scientific evidence and communicate effectively. Thus, ensuring edits are based on high-quality evidence presented in an unbiased neutral manner is a critical outcome for these courses. This public use of student-created health information has implications for Wikipedia and its editors, who also play a role in ensuring Wikipedia’s quality through their edits and by providing quality assessments on each medical topic’s talk page. As HPE instructors continue to offer courses and the volume of student edits increases, editors might consider partnering with courses to collaboratively assess pages, thus easing the workflow for editors and exposing learners to Wiki Project Medicine’s robust assessment system [63].

We identified a single educational initiative that included a final step of an instructor or expert verifying and adjusting the final product to ensure that the Wikipedia articles were left with high quality contributions [11]. Our results indicate that some instructors included proactive measures to moderate the quality of the final edits, such as encouraging students to interact with and work with Wikipedia’s medical editors, drafting contributions in My Sandbox, assessing contributions for readability and clear language, and peer reviewing suggested edits. Increased implementation of these measures, along with a final review of the live Wikipedia article following the editing initiative by a content expert, can help ensure that students are properly interpreting Wikipedia’s guideline for reliable medical sources. This focus on quality will also improve student experiences and interactions with the Wikipedia volunteer community.

What appears to be missing from the use of Wikipedia editing in HPE is systematic forms of assessment and evaluation to substantiate both the gains in learning and in the quality of Wikipedia entries, and to assess learning in EBM and health communication. This study should inspire current and new instructors using Wikipedia to add these important elements to the curriculum. Through the Wiki Edu dashboard, there is already a common set of measures for words and references added among the HPE courses. To this, common strategies and instruments could be established among the instructors to establish a body of finding across different settings and cohorts, not so much as a means of grading the students, but as a way of comparing the effectiveness of teaching strategies among different populations of students. These might include pre-test and post-test exercises to furnish the students with a means of assessing changes, for example, in (a) how they would approach the use of Wikipedia by professionals and patients; (b) identifying key factors in constructing and communicating medical knowledge; (c) ways of learning a new topic; and (d) expertise on the topic on which they have contributed to Wikipedia, and (e) means of engaging in evidence-based practices. These assessments might also involve an extension of Azzam’s deployment of outside experts [11] to assess a sample of editing changes for their quality, perhaps along dimensions of accuracy, evidence, and clarity. Such an approach, especially with some coordination among initiatives, might add to the evidence-based pedagogical practices behind HPE, which would be consistent with its own educational goals along EBM lines. To facilitate this goal, we encourage the formation of a special interest group in health professions education around the use of Wikipedia, which could lead to activities such as sharing assessment tools and other strategies.