Wikipedia and Academic Libraries: A Global Project/Chapter 10
Author Note
Funding details: This work was supported by the National Library of Medicine (NLM), National Institutes of Health (NIH) under cooperative agreement numbers UG4LM012342 with the University of Pittsburgh, Health Sciences Library System; UG4LM012340 with the University of Maryland, Baltimore; and UG4LM012344 with the University of Utah Spencer S. Eccles Health Sciences Library. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Abstract
Although academic skepticism of Wikipedia’s value as an information resource is widespread, the collaboratively created online encyclopedia is in fact one of the most frequently used health information resources in the world, including among students and professionals. As a result, the U.S.-based Network of the National Library of Medicine (NNLM) has, since 2018, organized biannual “#CiteNLM” edit-a-thons aimed at strengthening Wikipedia’s health pages by adding content and citations to trusted sources of information.
The first #CiteNLM edit-a-thon was a one-day virtual event in April 2018; since then NNLM’s edit-a-thons have evolved into month-long campaigns engaging primarily academic libraries with in-person edit-a-thons as well as virtual events. Hundreds of students, faculty, and library staff across the country (many of whom were new to Wikipedia editing) have collaborated in NNLM’s e orts to support universal access to high-quality health information. To date, over 600 health articles have been edited by over 400 editors. The current #CiteNLM campaign structure makes it easy for either individuals or groups to contribute or host affiliated events, which can include classroom exercises, citizen science projects, or library engagement efforts.
Introduction
Despite widespread skepticism of the value of Wikipedia as an information resource both within and outside academia, the collaboratively created online encyclopedia is one of the most frequently used health information resources in the world (Heilman & West, 2015). is includes students and professionals and also holds true in the medical field, with over 90 percent of medical students (Heilman & West, 2015; Metcalfe & Powell, 2011) and 50–70 percent of physicians (Heilman et al., 2011) using Wikipedia to find health information. Wikipedia’s health articles receive over 6.5 billion page views annually and are written in over 255 languages (Heilman & West, 2015).
As a means of improving this popular and accessible health information resource, the Network of the National Library of Medicine (NNLM) has, since 2018, organized biannual “#CiteNLM” edit-athons focused on using trusted sources of health information to add citations and content. The National Library of Medicine (NLM), which is part of the U.S. National Institutes of Health (NIH), is the world’s largest biomedical library and repository of digital and print health information resources. NNLM serves as an outreach arm of NLM focused on consistent and innovative engagement with both individuals and organizations, delivery of specialized trainings, and facilitation of health-focused events.
NNLM’s #CiteNLM campaigns dovetail nicely with two goals of NNLM and NLM. First, part of NNLM’s mission is to improve the public’s access to information to enable them to make informed decisions about their health (NNLM, Mission and Goals, para 1, n.d.-b). Although NLM is known for its digital resources (e.g., PubMed and MedlinePlus), a huge proportion of Internet users seek health information on websites like Wikipedia, making it an important resource for libraries providing health information to be aware of and familiar with. Much of NNLM’s work aims to promote health equity and informed decision-making by increasing digital health literacy. Improving the reliability of Wikipedia’s health pages is one way to meet Internet users where they are already comfortable looking for health information online for themselves and their families. NNLM believes that if it applies its human resources to enhancing Wikipedia articles, new partners will engage with NNLM’s work and more of the public will be reached by the broad spectrum of NLM’s health information resources.
Second, NNLM and NLM also have a formal commitment to promoting and supporting citizen science and crowdsourcing activities (NNLM, Crowdsourcing, para 1, n.d.-a). NNLM works to encourage community participation in the entire scientific research process via training for information professionals and collaboration with organizations working in this eld. While these activities have reached a wide variety of audiences, academic libraries have been particularly strong partners, especially in #CiteNLM campaigns. Wikipedia editing is a form of crowdsourcing and an avenue for public participation in the dissemination of high-quality research and health information. As of December 2020, over 400 individuals across the country have participated in #CiteNLM by editing over 600 health articles. Seven academic libraries have also contributed by hosting affiliated events.
Campaign History and Evolution
Background
The initial idea for implementing a series of online edit-a-thons began at NLM in a conversation with Wikipedian Ashleigh Coren during fall 2017, at a time when NNLM leadership was seeking innovative methods of engagement to expand beyond one-way communications like email newsletters. Edit-a-thon campaigns presented a creative opportunity to engage members through health topics of broad interest. After a call for interested parties in January 2018, a project planning team, the Wikipedia Working Group (WWG), was formed of librarians from NNLM’s regional and national offices. The WWG held a series of conversations with the Wikipedia medical community about their needs. Notably, trailblazer Dr. James Heilman, the founder of WikiProject:Medicine, consulted and later provided training. Due to its adaptability and simplicity, Wikipedia Library’s already established and widely recognized #1Lib1Ref campaign, which encourages librarians to improve Wikipedia by adding citations, served as a model (“The Wikipedia,” 2020).
2018 Campaigns
The topic chosen for the inaugural NNLM edit-a-thon in spring 2018, a one-day virtual event, was rare diseases. The WWG, which at the time included just one experienced Wikipedia editor, quickly learned the basics of editing biomedical articles in order to teach others. The WWG also created a project page in Wikipedia and a Wikimedia Labs Programs & Events Dashboard (NNLM Edit-a-thon, 2018), so participants could easily locate suggested articles and other information about the project and the WWG could quantitatively track event participation.
Once issues like time zone differences and platform choices had been ironed out, the next major challenge was to communicate plans for a virtual edit-a-thon to potential participants who were likely either unfamiliar with Wikipedia editing or accustomed to in-person editing events. Fortunately, since NNLM is an outreach-focused organization, various channels for project promotion already existed and creation of the #CiteNLM hashtag allowed participants to ‘gather’ on social media, both synchronously and asynchronously, to follow the work of their fellow editors across the country.
Ahead of the one-day edit-a-thon, Dr. James Heilman presented a well-attended customized training on behalf of the WWG which subsequently formed the basis for all future #CiteNLM trainings. Although the inaugural #CiteNLM event lasted only one day, it ran for twelve hours to give participants across the country ample time to join. WWG members took two-hour shifts staffing a virtual meeting room that participants could visit at any time to edit, ask questions, and connect with fellow participants and campaign organizers.
To evaluate the first campaign, the WWG gathered quantitative data from the Event Dashboard (NNLM Edit-a-thon, 2018), from Wikipedia itself, and from NNLM social media accounts. At least thirty-two editors edited at least 111 articles. Notably, Dashboard statistics likely underestimated true participation because the importance of Dashboard registration was not heavily emphasized during training and promotion for the first event. Participants came predominantly from the library community, which reflected the audiences reached most effectively by existing NNLM communication channels. Although overall engagement was lower than the WWG had hoped given the number of attendees at the training session, and it became evident that quantitatively measuring the impact of edit-a-thons can be complicated, this campaign proved that #CiteNLM could be a creative and successful way to engage member organizations and individuals.
Ready to test an improved campaign structure and grow #CiteNLM, the WWG soon began planning the Fall 2018 campaign focused on women’s health. The broader topic of the second campaign raised a new question: with so many women’s health-related topics, how should an editor choose an in-scope article to edit? A simple way is to focus on articles that, according to Wikipedia’s quality scale, rank lower: C-class, Start-class, and Stub-class articles typically have lots of room for improvement, while B-class and above articles are already fairly complete (“Wikipedia Assessment,” 2020). Lower-ranked articles were selected for recommendation to participants as editing targets.
The WWG planned wider promotional activities and more robust training. Three training sessions, each hosted by two NNLM staff, included an overview of Wikipedia, editing basics, and NLM resources for improving articles. Lasting around thirty minutes, these were intentionally brief to ensure ease of repeatability and minimize the time commitment needed from participants, were offered well in advance of the one-day edit-a-thon, and were recorded for asynchronous viewing. The virtual editing event utilized the same all-day drop-in structure as the spring 2018 event.
The second campaign, as expected, went more smoothly than the first. Dashboard numbers were more robust, owing partly to greater participation and partly to an increased proportion of participants registering so that their contributions could be tracked: at least fifty editors edited at least 204 articles. Following the now-repeated success of #CiteNLM, WWG members began to give presentations and poster sessions about the campaigns at a variety of conferences around the country. Presentations remain an important component of #CiteNLM outreach and have led to new professional connections and more detailed self-reflection. The WWG also remained committed to continual improvement and internally evaluated each aspect of the fall 2018 campaign by assessing content and processes from multiple perspectives to identify potential modifications for the next campaign.
2019 Campaigns
As an example of the WWG’s willingness to try new approaches, the WWG chose to hold its spring 2019 event at the 2019 Medical Library Association (MLA) annual meeting. This in-person session, sponsored by the MLA Health Disparities Special Interest Group, focused on the topic of health equity. Because the event occurred at the MLA conference, promotion was primarily aimed at health sciences librarians through channels like NNLM regional blogs and social media, MLA chapter email lists, and the MLA newsletter. Though the WWG had successfully held virtual events in the past, it was gratifying to work with fifty-four volunteer editors in person to improve content related to health disparities. The collaborative atmosphere and success of this event inspired the WWG to explore ways to incorporate in-person events for future campaigns, leading to significant structural innovations for the fall 2019 campaign.
In advance of the fall 2019 campaign, the WWG focused on enhancing the #CiteNLM experience with a more user-friendly web presence and a longer campaign period—two full months—coupled with support for institutions to host affiliated local events at any time during this period. is campaign did not o er a dedicated one-day virtual edit-a-thon; the WWG instead invited individual participants to attend a kickoff training session and then edit independently throughout October and November. The WWG built a new web page (https://nnlm.gov/national/guides/ccs/wikipedia-edit-thon) with everything needed to host affiliated events or participate as an individual. A new #CiteNLM Guide for Organizers, adapted from #1Lib1Ref, laid out the steps for hosting independent events in a straightforward PDF format. This campaign was the WWG’s most successful yet, with at least 108 editors and 9 affiliated events, several of which are detailed below.
2020 Campaigns
The WWG planned for spring 2020 largely by looking to build on the successes of fall 2019; the only major change was a reduction in the campaign period from two months to one to reduce the amount of staff time required. Unfortunately, April 2020 proved to be an extremely difficult time for libraries to offer programming as institutions scrambled to shift operations from in-person to virtual in the midst of a global pandemic. It is also likely that individual participation was reduced by the dramatically altered circumstances as potential participants focused on core job responsibilities and adapting to the rapidly evolving situation. As a result, overall participation was fairly low: forty-nine editors participated and only two institutions held affiliated editing events. The WWG viewed the challenges of the spring 2020 #CiteNLM campaign not as failures but as informative setbacks that provided motivation to make #CiteNLM more flexible and better able to meet the ongoing need for virtual programming opportunities.
Acknowledging the altered working conditions due to the COVID-19 pandemic, the WWG approached the fall 2020 campaign with renewed energy and developed a new hybrid campaign structure combining a month-long campaign period for affiliated events with a scheduled two-hour live collaborative edit-a-thon for individuals. The campaign training webinar featured strategies and resources for institutions to host their own virtual #CiteNLM edit-a-thon any time during the month of October. At the live event, participants received a brief training and then joined breakout sessions to edit articles related to the campaign topic of maternal and child health, chat and ask questions about Wikipedia, and connect with the #CiteNLM community. Over thirty editors participated in this virtual event and many have expressed excitement about making their first Wikipedia edit during the session.
Leading up to the fall 2020 event, a new outreach campaign was launched aimed at reaching library and information science (LIS) students. is group is an intended audience for NNLM outreach and LIS students are well-situated to be excellent Wikipedia contributors due to their likely interest in information literacy, research, and reference. The WWG invited students to join the two-hour live editing event, pitching it as an opportunity to meet practicing information professionals and network with other LIS students from around the country while also learning about Wikipedia editing and Wikipedia’s health information. Messaging was sent to MLIS programs promoting the #CiteNLM edit-a-thon as an opportunity to build reference skills, health information literacy, familiarity with trustworthy health information resources, and an understanding of how to utilize Wikipedia responsibly. As a result of this simple campaign, LIS students from at least six different programs across the country joined the virtual editing event. The WWG hopes that these new editors will continue to contribute to Wikipedia and pass information about the importance and ease of doing so along to their colleagues as they finish their degrees and enter the field.
To reach a broader audience, training opportunities were expanded to include a four-week asynchronous online course entitled Wikipedia + Libraries: NNLM (https://nnlm.gov/wikipedia-libraries-nnlm), which ran concurrently with the campaign and was taught by members of the WWG. Tailored to a librarian audience, the course introduced over thirty participants to the importance of Wikipedia’s health and medical information, the basics of Wikipedia editing, and suggestions for incorporating Wikipedia into their library’s programming and instruction. In total, between the virtual event, Wikipedia + Libraries students, and other participants, eighty-six editors edited seventy-seven Wikipedia articles for the fall 2020 campaign.
Academic Libraries as Partners
Through the six #CiteNLM campaigns to date, academic librarians and libraries have been vital partners in NNLM’s work to improve health information on Wikipedia. Academic libraries are well-situated to do this work since they tend to have existing information or health literacy programming that #CiteNLM can be used alongside; experience with instruction; topical expertise; and a connection with academic infrastructure that places value on dissemination of high-quality health information and research. Examining academic libraries’ motivations for participating in #CiteNLM and their experiences hosting campaign-a liated editing events provides valuable insight into the campaigns’ successes and challenges. ese events are also informative case studies for incorporation of Wikipedia editing into academic library programming and instruction. rough the spring 2020 #CiteNLM campaign, seven academic libraries hosted a liated editing events, and four of these responded to a survey about their experiences. e host libraries were Indiana University Ruth Lilly Medical Library, Massachusetts College of Liberal Arts Freel Library, Radford University McConnell Library, University of California, Los Angeles Biomedical Library (UCLA), University of Maryland, Baltimore Health Sciences and Human Services Library (UMB), University of Massachusetts Lamar Soutter Library (UMass), and University of Pennsylvania Biomedical Library (UPenn).
Choosing to Host
The Wikipedia Working Group employed a suite of engagement tools to reach academic partners and raise awareness of #CiteNLM. Several libraries became involved with #CiteNLM through their established connection with NNLM as hosts of NNLM Regional Medical Libraries (UCLA, UMass, UMB). UPenn, which does not host a Regional Medical Library, learned of #CiteNLM through individual librarians’ participation in the 2019 MLA immersion session. e remaining partner libraries learned of #CiteNLM via online trainings and promotional messages distributed via social media and email.
Regardless of how they learned about #CiteNLM, academic library partners ultimately chose to host #CiteNLM events for several reasons. UMB noted that holding an in-person #CiteNLM event simply allowed for a fun, collaborative library program to engage faculty and students from across campus without much staff time needed for development. Several institutions cited the high number of residents and medical students utilizing Wikipedia on a daily basis as a driving factor. Participating in a #CiteNLM event allows medical students to learn more about this information resource and have a hand in directly improving health information widely utilized by providers and patients. In alignment with the core mission of the #CiteNLM campaigns, the desire to make Wikipedia a more credible and reliable information resource was also an important factor in why libraries decided to host #CiteNLM events. As UMass noted, engaging knowledgeable people to edit Wikipedia allows subject matter experts to share their expertise on a topic and provides the public with a more credible source of health information.
Implementation and Collaboration
UCLA and UPenn collaborated creatively with other university departments to position #CiteNLM alongside Health Literacy Month and Open Access Week. UCLA also hosted their event in conjunction with UCLA’s Centennial Initiative, pulling in faculty and audiences from on campus and from the wider University of California system. Others libraries opted to hold stand-alone events that better suited their programming needs. Some partner libraries hosted multiple sessions (e.g., a training preceding the actual edit-a-thon or several scheduled editing events to allow varied audiences to participate), while others held one event combining training and editing. The range of ways in which academic library partners have tailored #CiteNLM to t their programming needs demonstrates the campaign’s adaptability to various contexts even within the academic library.
To facilitate connections between planned library programming and #CiteNLM, academic library partners effectively utilized both NLM’s wide array of digital health information resources (e.g., MedlinePlus) and NNLM Wikipedia editing resources (e.g., the Organizers’ Guide and online trainings). All of the WWG’s academic library host partners stated that they utilized the Organizers’ Guide to help plan their event and then adapted NNLM’s materials in various ways to fit their local needs. For example, UPenn created training materials based on NNLM’s but with additional information about their locally available databases. UMB used NNLM’s trainings as a basis for the training offered at their in-person event and utilized linked resources from the #CiteNLM web page to identify articles in need of editing. UMass relied on NNLM’s promotional materials to publicize their event and on NNLM’s recorded trainings to introduce remote editors to Wikipedia editing. Partnering with NNLM’s Pacific Southwest Regional office, librarians from the UCLA Biomedical Library created training materials specific to the campus. They promoted directly to the UCLA MLIS program, recruiting students to participate in the campaign and even help with creating the event Dashboard. The team also heavily promoted the event through the OpenUCLA Centennial Initiative, which was open to the entire campus as well as the public. Finally, many institutions diverged from the topic of the current #CiteNLM campaign, choosing instead to adapt the materials to a topic of greater interest to their local audience.
Evaluation
Each partner library learned valuable lessons and identified opportunities for improvement of future #CiteNLM events. Most partners identified a desire for a greater number of participants at their events and plan to address this issue in several ways. Since upcoming events will most likely be virtual for the foreseeable future, some institutions are exploring the possibility of holding edit-a-thons as asynchronous events similar to the main #CiteNLM month-long campaigns to enable more students, staff, and faculty to participate as their schedules allow. Partner libraries also mentioned plans to expand promotion to wider audiences (e.g., through posted advertisements in spaces frequented by students and hospital staff), advertise earlier, and collaborate with university health sciences departments to draw in more editors by emphasizing the educational potential of edit-a-thon participation.
In some cases, event location may have contributed to low participation. Due to space restrictions, UMB held their event in library classrooms located in the basement away from the usual library traffic. They hypothesized that moving future in-person events to a central location on campus or in the library may pique the curiosity of passers-by and lead to increased attendance. UMass experienced difficulties tracking participation because not all participants joined the event dashboard, so their edits were not captured. In spite of reported lower-than-desired attendance in some cases, all responding partner libraries agreed that taking part in #CiteNLM helped transform Wikipedia users into contributors by changing their perceptions of Wikipedia. Participants and hosts both learned that they could make an impactful contribution to Wikipedia by becoming editors and improving the encyclopedia’s health information.
The NNLM WWG is extremely appreciative of the work their academic library partners have done to bolster the success of #CiteNLM and hopes that these collaborations will grow even stronger for future editing campaigns. Partner feedback has consistently proved essential for shaping the future direction of #CiteNLM and academic libraries will continue to be a core audience for the WWG’s efforts to improve health information on Wikipedia, even as the WWG continues to think about ways to effectively tailor the campaigns for new audiences.
Looking Forward
Looking to the future, the WWG has identified several avenues to expand the reach and impact of #CiteNLM campaigns. Given the success of the fall 2020 virtual edit-a-thon, this format will be repeated in spring 2021. The Wikipedia + Libraries course will also be offered again alongside the spring 2021 campaign. The WWG plans to continue and expand its outreach to LIS students and programs by repeating fall 2020’s outreach campaign and pilot testing a program-in-a-box designed to make it easy for LIS faculty to teach about evaluating and contributing to Wikipedia’s health information in any class where it would be relevant. Finally, the WWG hopes to partner more closely with health sciences students and faculty for future campaigns, as their topical expertise is a largely untapped but promising resource.
The WWG continually takes qualitative and quantitative feedback into account in improving each successive campaign in order to better reach and serve its audiences (see Cowles et al., 2020 for greater detail). It is their hope that as a result, #CiteNLM will continue to engage new editors and empower them to contribute further to the important and ongoing work of improving the reliability of Wikipedia’s health information.
References
Cowles K., Sheppard M., Waltman E., & Wilson T. K. (2020). Crowdsourcing and collaboration from coast to coast: NNLM’s #CiteNLM Wikipedia edit-a-thons. Journal of Electronic Resources Librarianship, 32(4), 267–75. https://doi.org/10.1080/1941126X.2020.1821991
Heilman, J. M., Kemmann, E., Bonert, M., Chatterjee, A., Ragar, B., Beards, G. M., Iberri, D. J., Harvey, M., omas, B., Stomp, W., Martone, M. F., Lodge, D. J., Vondracek, A., de Wol , J. F., Liber, C., Grover, S. C., Vickers, T. J., Meskó, B., & Laurent, M. R. (2011). Wikipedia: A key tool for global public health promotion. Journal of Medical Internet Research, 13(1), e14. https://doi.org/10.2196/jmir.1589
Heilman J. M., & West A. G. (2015). Wikipedia and medicine: Quantifying readership, editors, and the signi cance of natural language. Journal of Medical Internet Research, 17(3): e62. https://doi.org/10.2196/jmir.4069.
Metcalfe, D., & Powell, J. (2011). Should doctors spurn Wikipedia? Journal of the Royal Society of Medicine, 104(12), 488–89. https://doi.org/10.1258/jrsm.2011.110227.
Network of the National Library of Medicine. (n.d.-a). Crowdsourcing and citizen science. https://nnlm.gov/national/guides/ccs.
Network of the National Library of Medicine. (n.d.-b). Mission and goals. https://nnlm.gov/mar/about/mission.
NNLM Edit-a-thon. (2018, April 30). In Wikipedia Outreach Dashboard. https://outreachdashboard.wmflabs.org/courses/National_Network_of_Libraries_of_Medicine/NNLM_Edit-a-thon_-_spring_2018/home.
The Wikipedia Library/1Lib1Ref. (2020, December 10). In Wikipedia. https://meta.wikimedia.org/wiki/The_Wikipedia_Library/1Lib1Ref.