Contributions to conservation outcomes by natural history museum-led citizen science: Examining evidence and next steps

Orchid Observers Publications

30 May, 2017 / Alyson Slade

The Orchid Observers project has been cited in a journal article and book chapter, both co-authored by Dr John Tweddle, Head of the Angela Marmont Centre for UK Biodiversity, Natural History Museum London.
Orchid Observers is one of the projects analysed in Contributions to Conservation Outcomes by Natural History Museum-led Citizen Science: Examining Evidence and Next Steps. The article was published in Biological Conservation, Volume 208, April 2017 pages 87-97 and is available to read here.
Orchid Observers is cited as an innovation case study in the chapter Citizen Science: Authentic Science Research at the Natural History Museum, In: Museum Participation: New Directions ForAudience Collaboration, edited by Kayte McSweeney and Jen Kavanagh.  (Published April 2016).

Biological Conservation

Volume 208, April 2017, Pages 87–97

• Heidi L. Ballard^{a, ,} ,
• Lucy D. Robinson^b,
• Alison N. Young^c,
• Gregory B. Pauly^d,
• Lila M. Higgins^d,
• Rebecca F. Johnson^c,
• John C. Tweddle^b

 Show more

https://doi.org/10.1016/j.biocon.2016.08.040

Get rights and content

Under a Creative Commons license

  Open Access

---

Highlights

•

We examined 44 citizen science programs at 3 natural history museums in US and UK

•

Key types identified: bioblitz, monitoring, bounded research, digitized collections

•

26 programs contributed to conservation research, management, education and policy

•

Collaborate among researchers, educators at NHMs to ensure conservation outcomes

---

Abstract

Through their unique combination of specimen collections, scientific and public education expertise, and wide audience reach and trust, natural history museums (NHMs) are obvious settings for bridging conservation science and education through citizen science. Building on over 100 years of amateur naturalist contributions to biodiversity science, a wide range of NHM-based citizen science programs have emerged recently. Yet no comparative studies of the conservation outcomes of this work exist. Here we ask, what is the evidence that NHM citizen science contributes to conservation, what kinds of programs and strategies do so, and how could this approach be better realized for conservation goals? We analyzed 44 citizen science programs across three museums (one U.K., two U.S.) to assess whether and how they contribute to conservation-relevant outcomes. We found evidence that they support conservation both directly, through site and species management, and indirectly through research, education and policy impacts. This study has implications for understanding the role NHMs can play in maximizing the socio-ecological impacts of citizen science, including bringing citizen science to new audiences, mobilizing volunteers to collect and analyze data to study species invasions and impacts of global changes, and conducting locally-relevant research in urban systems. NHM citizen science can provide multiple entry-points and levels of engagement for participants in science and access to new means of studying biodiversity, both in the field and virtually. From our findings we recommend collaboration among the research and education staff within NHMs and other similar conservation organizations, as well as partnerships with external organizations to successfully contribute to conservation outcomes.

Keywords

• citizen science;
• natural history museums;
• biodiversity;
• digitization;
• conservation education;
• natural resource management

---

1. Introduction

Natural history museums (NHMs) and the field of citizen science share the dual goals of education and generating new scientific knowledge. Through their unique combination of specimen collections, scientific and public education expertise, and wide audience reach and trust, NHMs are obvious settings for bridging conservation science and education through citizen science. Building on over 100 years of amateur naturalist contributions to biodiversity science, a wide range of NHM-based citizen science projects have recently emerged. Yet, no comparative studies of the outcomes of this work exist, particularly with respect to conservation outcomes. Analyzing past and current citizen science programs at three high-profile natural history museums in the U.S. and U.K., we examined the evidence of whether and how NHM-led citizen science contributes to conservation, and how this approach could further advance conservation goals. The implications of these findings apply not just to NHMs, but also provide a lens through which a broader range of conservation organizations can examine how citizen science may or may not contribute to conservation outcomes such as education, research, and species and land management.

Citizen science has been defined in recent years, with slight variation, as members of the public collaborating with professional scientists to collect, transcribe, categorize, and/or analyze data that contributes to our understanding or management of the natural world (Bonney et al., 2009; Bonney et al., 2014 ;  Gura, 2013). We see citizen science as an inherently interdisciplinary field encompassing the range of natural and social sciences, including education, psychology, and sociology among others. While Sullivan et al. (2014) have recently noted the effectiveness of interdisciplinary approaches to citizen science across disciplines like biology and informatics, this merely scratches the surface of the collaborations potentially involved in effective programs. A variety of typologies for sorting and categorizing different citizen science programs exist in the recent literature that help illuminate differences in level of community or public involvement and program structure (Haklay, 2013 ;  Shirk et al., 2012), impacts on public understanding of science (Bonney et al., 2015), or primary organizational goals (Wiggins and Crowston, 2011). For our purposes of exploring, characterizing, and analyzing citizen science that particularly involves NHMs, we define NHM-led citizen science as citizen science programs that are initiated or coordinated by NHM staff and/or involve the specimen collections and curatorial research of an NHM. We are interested in what is unique about NHM-led citizen science for conservation, as opposed to what NHMs or citizen science programs might do separately, and also the generalizable lessons that can be applied to conservation practice more broadly.

Conservation organizations and museums alike often struggle to evaluate the conservation impacts of their work (Miller et al., 2004 ;  Spooner et al., 2015). To address this need, the Cambridge Conservation Forum (CCF) developed a conceptual framework to evaluate conservation activities and to help organizations clarify their objectives, based on a review of current conservation research and the input of 36 conservation organizations. They delineated seven categories that directly or indirectly lead to targeted improvements in the status of species, ecosystems or landscapes: Species Management and Site Management (Direct), and Research, Education, Policy, Livelihood, and Capacity-Building activities (Indirect) ( Kapos et al., 2008). Rather than limit to only direct conservation activities, the CCF framework included the myriad activities that contribute to conservation indirectly, such as public education programs that influence individual conservation behaviors.

Evidence that citizen science contributes to these conservation activities has only recently begun to be examined (Conrad and Hilchey, 2011), despite the recent surge in citizen science projects globally ( Bonney et al., 2014). We adapted the CCF framework for evaluating conservation effectiveness to align with the NHM and citizen science fields (Table 1), and then asked whether NHM-led citizen science efforts contribute to conservation outcomes. Importantly, NHMs and citizen science programs each have a variety of goals in addition to conservation; conversely, not all conservation activities can or should be expected of them. For the purposes of our analysis, we adapted the CCF framework by combining species management and site management into a single category. Further, the CCF category “capacity-building”, which Kapos et al. (2008) defined as “actions to enhance specific skills among those directly involved in conservation” was not a goal for any of the projects considered in this analysis, nor is it a common goal of NHM or citizen science efforts individually; thus, we excluded this category. For the remaining five categories, in this article we review the existing empirical or theoretical research on how citizen science and NHMs independently have or might contribute to these conservation outcomes. We then analyzed citizen science projects at three NHMs to determine to what extent, and under what circumstances, NHM-led citizen science projects contribute to these outcomes.

Table 1. Definitions of conservation activities (adapted from Kapos et al., 2008).

Conservation Activity Type	Definition and examples
Species and Site Management	Managing species and populations, (e.g., captive breeding), and managing sites, habitats, landscapes and ecosystems.
Research	Research aimed at improving the information base on which conservation decisions are made (e.g., surveys, inventories, monitoring, and mapping).
Education	Education and awareness-raising to improve understanding and influence behavior among people (e.g., campaigns, lobbying, and educational programs).
Policy	Developing, adopting or implementing policy or legislation (e.g., management plans, trade regulations, and actions that make conservation goals official).
Livelihoods	Enhancing and/or providing alternative livelihoods to improve the well-being of people that are impacting the species/habitats of conservation interest, such as through sustainable resource management, income-generating activities, and others.

Table options

1.1. Species and Site Management

NHMs contribute to conservation through species and site management primarily through their collections, which can both inform conservation assessments and practical management. As more museums digitize their collections, land managers can increasingly access high-quality, voucher-referenced information crucial for species conservation (Drew, 2011). Furthermore, specimen and observational data combined with environmental data lead to applied biodiversity informatics such as species distribution modeling that can inform management and conservation (Anderson, 2012 ;  Gaubert et al., 2006). Similarly, for citizen science, McKinley et al. (2017) found evidence that citizen science has contributed to natural resource management and policy by providing high quality information and through public engagement. Further, collaborative monitoring can help land managers work with local communities to monitor the effects of resource management practices (Fernandez-Gimenez et al., 2008). Sullivan et al. (2017) also note the value of timely spatial and temporal data generated through the citizen science program eBird for informing species management.

1.2. Research

NHMs have a long history in both the U.S. and the U.K. for contributing to biodiversity research and conservation education. NHMs are particularly well-positioned to answer some of the grand research challenges in biodiversity conservation in the 21st Century: species' response to habitat loss and fragmentation, biological invasions, and the effects of climate change (Drew, 2011; Krishtalka and Humphrey, 2000; Suarez and Tsutsui, 2004 ;  Winker, 2004). Specifically, because of the historical record provided by specimens, museums can study the effects of environmental and human-related change on the distribution and abundance of species, phenology, and pollination rates, over long time periods (Hoeksema et al., 2011; Johnson et al., 2011; Robbirt et al., 2010 ;  Shaffer et al., 1998). This vast stored potential, however, presents two challenges: the need to digitize historical and current biodiversity data, and to acquire modern records for comparison. Citizen science can provide a means to address both challenges. Notes from Nature and other crowdsourcing initiatives are liberating vast quantities of historical data from museum specimens and catalogs (Hill et al., 2012), and citizen scientists are also gathering vast datasets of contemporary biodiversity and environmental records, contributing extensively to biodiversity research as evidenced by hundreds of peer-review journal articles (Sullivan et al., 2017 ;  Theobald et al., 2015).

1.3. Conservation Education

In a time where biodiversity is highly threatened, the most pressing issues require scientific literacy and conservation action (Hacker and Harris, 1992); yet society as a whole has become more and more disconnected from the natural world (McKee, 2005). NHMs are located primarily in urban settings and have an opportunity to link urban populations to their own biodiversity, to help people understand it, feel a connection to their place, and a desire to conserve it. As informal science education institutions, NHMs have the goal of increasing public understanding of science as well as appreciation for the natural world (Miller et al., 2004). Research reviews in environmental and museum education confirm that NHMs reach a wide range of public audiences with free-choice learning opportunities (Dillon, 2003 ;  Falk, 2005), and also reach schools and youth through intensive schools programming. Citizen science programs also have evidence of conservation education outcomes, such as increasing participants' knowledge of target taxa and their understanding of the scientific process (Brossard et al., 2009; Bonney et al., 2009 ;  Crall et al., 2013). Furthermore, evidence has been found in some cases that volunteers in some monitoring programs change their behavior based on what they have learned (Evans et al., 2005), and tend to disseminate the information they gain, thereby increasing the profile of the issues being researched (Couvet et al., 2008). Additionally, some citizen science programs foster a sense of place in participants (Haywood, 2014) and influence future conservation behaviors (Krasny et al., 2014).

1.4. Policy

While conservation outcomes at NHMs tend to center on Research and Education activities, the common focus on sustaining and increasing our understanding of our natural world means many NHMs may also impact conservation policy. Citizen science can contribute to conservation and natural resource management policy through providing better data for resource decision-making and through broadening participation and engagement in the science that informs management (McKinley et al., 2017).

1.5. Livelihoods

We have long had evidence that participatory and community-based approaches like citizen science contribute to community resilience and conservation-related livelihoods (Western and Wright, 1994), but evidence that other forms of citizen science impact livelihood activities is more tenuous (Jordan et al., 2012). In addition, citizen science faces challenges with reaching historically underrepresented groups, and calls to better connect community priorities and research objectives (Evans et al., 2005; Pandya, 2012 ;  Trumbull et al., 2000) could be addressed by partnering with NHMs with close ties to the local urban communities they often serve, though it should be noted that cultural institutions such as museums can also struggle to bring in diverse audiences that are fully representative of the cities in which they are located (Warwick Commission, 2015).

Combined, citizen science and NHMs have the potential to impact conservation activities in synergistic ways. Many museums have realized that for members of the public to fully grasp the scientific process and its role in conservation, they need to be more engaged in science and conservation activities in their everyday lives, beyond the museum walls. Citizen science is one effective way to accomplish this goal. Combining two forms of informal science education, citizen science and museum education, can provide participants with understanding of conservation science and the impacts of human actions on ecosystems, as well as supporting the development of skills and dispositions that facilitate conservation behaviors, all key conservation education outcomes (Jacobson et al., 2006 ;  Monroe, 2003). Furthermore, as a means to help provide the information necessary to meet today's biodiversity and global environmental change challenges, several NHMs have developed citizen science projects focusing on the digitization of biodiversity data and acquiring modern records for comparison to historical museum records (Ellwood et al., 2015). NHM citizen science has the potential to produce valuable biodiversity data for research and management from under-studied areas while simultaneously engaging people with and educating them about nature and conservation. With such great potential for NHM citizen science to impact conservation in these ways, we investigated to what extent, and in what ways, does the reality live up to this promise? We did this by comparing the evidence from three active citizen science programs at NHMs. Specifically, we asked:

•

Is there evidence that NHM-led citizen science initiatives produced conservation outcomes?

•

What types of programs and strategies led to conservation outcomes?

•

What are the key characteristics and innovations of NHM-led citizen science programs that resulted in conservation outcomes?

2. Methods

2.1. Study Sites

The California Academy of Sciences (CAS), Natural History Museum of Los Angeles County (NHM LA) and Natural History Museum in London (NHM London) were the focus of this study. These institutions were selected purposively for their high number of active programs and participants and the duration of their citizen science efforts (Table 2). All are located in large metropolitan areas and receive local, national, and international visitors. They are all research institutions whose scientists conduct research locally and around the world, and they have exhibits and educational programs that are experienced by hundreds of thousands of visitors every year. All three have broadly similar goals for their citizen science programs. Yet, the very nature of citizen science is to intertwine public engagement and the scientific research of the museum; hence, we report here the specific features that provided context for each NHM's approach to the structure and implementation of their citizen science programs, particularly with respect to whether programs were initiated or coordinated by the public education and outreach unit, or the research and curation unit, or both (Table 2).¹

Table 2. Organizational context for citizen science programs in three NHMs.

	NHM London	CAS	NHM LA
Location	London, UK	San Francisco, CA	Los Angeles, CA
No. of annual visitors	5.4 million	1.4 million	791,000
Stated goals for Citizen Science Programs	Establish a collaborative center of excellence in citizen science, and deliver an ambitious program of innovative research projects that combine broad public participation and quality learning experiences with high-impact science.	Building community in person and through technology and mobilizing people and data to make a difference and connecting people to their local nature by providing support, opportunities, education, resources and tools to document biodiversity. Strategically designing programs and collecting biodiversity data toward biodiversity conservation through research, management, education, and stewardship.	To engage the public in citizen science programs and projects focused on collecting biodiversity data in the greater L.A. region and beyond.
Institutional location of CS Program coordination	Angela Marmont Centre for UK Biodiversity in Science Group (which encompasses Research and Curation teams)	Citizen Science Department in the Institute for Biodiversity Science and Sustainability (which includes Research and Collections)	Joint coordination by staff in Education & Exhibits and Research & Collections
Supporting depts.	Life Sciences, Earth Sciences, Public Engagement Group	iNaturalist, Botany, Invertebrate Zoology & Geology, Teacher & Youth Education, Communications	Marketing & Communications, and Advancement
No. of key personnel in CS program	3 full-time, 4 part time, 9 scientist leads	6 full-time, 5 scientist leads	3 full-time, 6 part-time, 6 scientist leads
Key roles of personnel involved in CS FT = full time PT = part time	Two FT staff work across a range of projects, coordinating the program, developing new projects and sharing knowledge and good practice via conferences and publications. Four PT staff deliver specific citizen science projects. The Head of the Angela Marmont Centre for UK Biodiversity leads the Program.	Two FT staff lead and deliver the citizen science program, develop new projects, build partnerships, communicate about citizen science and lead the Bay Area Citizen Science Coalition. The Director of Citizen Science serves as the representative for the program in division planning. Three FT staff work with Science Action Clubs developing citizen science modules for use in after- school settings.	Three FT staff coordinate citizen science efforts across departments and support a range of projects. An additional three education staff help to support citizen scientists, public and school programs, and staff training. All the above staff plus one scientist and two communications staff sit on the museum's Citizen Science Working Group.
Start of earliest citizen science (CS) programs,2No. participants to date	1996 64,500 total participants	2000 10,000 total participants	1994 4200 total participants
No. active CS programs, No. participants Aug. 2014–15	9 programs 4000 participants	10 programs 1200 participants	12 programs 1500 participants
No. of museum scientists in CS programs	33	25	11

2

Earliest program dates exclude Cetacean Stranding Programme and Marine Mammal Stranding Program, which began many decades ago, because these are broad national programs not initiated as citizen science programs by the museum originally.

Table options

2.2. Data Collection and Analysis

In order to evaluate the conservation outcomes for the three NHM's citizen science programs, we compiled documentation about all past and current citizen science programs at each NHM, including but not limited to each project's inception, duration, number of participants annually and in total, goals of the project, target audience participating and ways that participants were, or are, involved in the scientific process, focal taxa if appropriate, and research questions addressed (See Appendix A for summary of all programs analyzed). We collected evidence of contributions to the five conservation activities defined above. Evidence for contributions to Site or Species Management activities were evidence-based internal reports, reports from other organizations, and news media reports that discussed the use of data or findings from the citizen science program for changing or supporting species, resource, or land management decisions on a local scale. Evidence for contributions to Policy was the same documentation requirement as for Site and Species Management, but applied to changes or support of government policies on a regional or national scale. Evidence for contributions to Research was assessed via peer-reviewed journal publications based on data collected or processed by the citizen science program. Evidence for contributions to Education was program evaluation data or reports about science or environmental learning outcomes for participants (we did not include evaluations around program satisfaction or initial motivations for participating). Evidence for contribution to Livelihoods was reports from program evaluations, news media, external community-based organizations or government. We also conducted approximately 10 unstructured interviews with additional NHM staff to fill any gaps on program information and source applicable evidence.

Data analysis was conducted in two stages. First, all co-authors collaboratively sorted all museums' programs using the information (reported in Appendix A). We clustered programs with similar features and categorized them into four program types that would allow for systematic analysis of evidence of conservation outcomes. Second, for the analysis of evidence of contribution to the five conservation outcome categories, the lead author (unaffiliated with any museum) qualitatively coded all of the (independently verifiable) evidence documents described above, to control for bias and provide consistency in coding. She used the CCF Framework to analyze the evidence for each program for each conservation outcome category. Specifically she determined: Were there official reports from land managers showing evidence of the programs' citizen science data being used for management decisions? Did published peer-reviewed journal articles result from citizen scientist-collected data? Did educational evaluation of the program, if conducted, report a positive change in participants' understanding of conservation science, attitudes, or behaviors? If so, what were the specific changes? Were there any reports with verifiable evidence of changes in regional or national policy explicitly stating the use of the citizen science program's data? Were there any government or media reports stating that the citizen science program had provided participants with new livelihoods or income-generating sources, or otherwise improved the well-being of the community involved in the program? These produced our findings regarding evidence in each of the four program types for each of the five conservation outcome categories.

3. Results

All three NHMs have been engaging in projects that could be defined as citizen science since their inception, but growth of programs designed specifically as citizen science programs, especially of technology-enabled projects that are fueling the expansion of this field, started in 2012 for CAS, 2010 for NHM LA, and 2006 for NHM London. We found that despite the relatively recent emergence of citizen science as an intentional component of these natural history museums' activities, there was substantial evidence that many programs were contributing to several different conservation activities. Beyond that, the important analysis for the fields of conservation, citizen science and science education was: which types of programs contributed to which conservation activities, and how? To answer these questions, we report here: 1) the four main program types into which we categorized all NHM citizen science programs; 2) for each program type, the evidence we found for impacts on, or contributions to, the five conservation activities outlined above, with summarized findings from examples of programs for which we have evidence (not all programs have evidence of impacts, which we address in the Discussion). Notably, we found no evidence of conservation-related livelihood outcomes from any of the programs to date. We discuss the implications of this with respect to our findings and the need for evaluation in the Discussion.

3.1. Main Types and Key Features of Museum-Led Citizen Science

The three NHMs had 44 different citizen science programs, including 15 at NHM London, 16 at the CAS, and 13 at NHM LA; 31 of these are currently active (Supplemental Appendix A). We first inventoried and characterized the range and scope of citizen science programs at NHMs. Analyzing the characteristics of each program, we developed four categories that represent the main types of NHM citizen science programs and used the distinguishing characteristics to define the key features of each type (Table 3). All programs fell into one of these types and only one program fell into two categories (Orchid Observers, an intentionally hybrid program). The key features that separated these four program types were duration, research approach, and ways in which citizen scientists participate. This functional scheme reflects the programs within this study, and can be applied to NHM citizen science programs more broadly, but is meant to complement rather than replace existing typologies for the citizen science field as a whole.

Table 3. Four main types of NHM Citizen Science programs and their key features.

Key Features	Bioblitzes and other Citizen Science Events	Ongoing Monitoring Programs	Bounded Field Research and Inventory Projects	Data Processing of Digitized Collections
Overall design and goals	Snapshot inventory of specific place in short span of time	Ongoing monitoring of change in abundance or distribution of species over time	Design and methods driven by one research question	Enhance museum digitized specimens by crowd-sourcing data entry
Duration	One-day or very short-term	More than 1 year	May be less than 1 year or several years	Any duration, from a minute to years
Main aspects of scientific process that public participates in	Data collection of observations and/or samples	Data collection of observations and/or samples	Data collection of observations and/or samples	Transcription of collections-based natural history data and/or classification of digitized photos of specimens, entirely online
Type and intensity of training	Short on-site training	Online instructions or in-person trainings	Online instructions, kits with equipment mailed, or in-person trainings	Online instructions
Geographic scope	Bounded specific site	From local to nationwide	From local to nationwide	All online so worldwide
Main audience	General public including families, amateur experts, youth and adults	General public, special interest groups	General public, amateur experts	General public, amateur experts

Table options

3.1.1. Bioblitz and Other Citizen Science Events

These programs were designed to take a biodiversity snapshot in one particular location over a short time frame (usually 2–24 h). Volunteers collected geo-located, time-stamped photo observations and recorded field observations or specimens of as many species as possible. Some Bioblitzes focused on one taxonomic group (e.g. plants or insects), while others aimed for as complete an inventory as possible. Other short-term events, such as festivals, involved similar data collection activities and may be singular, seasonal, or annual events.

3.1.2. Ongoing Monitoring Programs

These programs were designed for volunteer collection of observational field data that could be used for a variety of research or monitoring questions and typically operated indefinitely. The main goal was to track change over time, usually for species distribution or abundance, often focused on a particular taxonomic group or species. Data might be geo-located, time-stamped photos, recorded field observations, or occasionally, specimens.

3.1.3. Bounded Field Research and Inventory Projects

Curators and other NHM researchers identified and developed research questions that require, or can benefit from, a citizen science approach, such that the design and methods by which citizen scientists collect data was driven by that research question. This type of program had a start and an end date, closing when the question was answered (i.e. enough data have been gathered). Duration varied from less than a year to several years, with the duration being set based on the research question. This contrasts with Ongoing Monitoring Programs that could be much longer-term or indefinite.

3.1.4. Data Processing of Digitized Collections

Specimen digitization and crowdsourcing initiatives are rapidly emerging at NHMs, providing a crucial enhancement of their collections that allows unprecedented access to specimen data for in-house curators, external researchers, and the public. Citizen scientists across the globe transcribed collections data and extracted other scientific data (e.g. phenological information) from photos of specimens, specimen labels, or collections registers via a website, feeding into a database ready for analysis.

3.2. Evidence of Contributions to Conservation Activities

For each conservation activity, we analyzed which museum programs and which program types had evidence of contributing to that activity, and if so then what the specific contribution was (Table 4). Overall, 26 of the 44 programs (59%) had independently assessable evidence of contributing to one or more of the five conservation activities. Across NHMs, fifteen of these had evidence in two or more categories, and four of these had evidence of contributing to three categories. We provide example citations for evidence of outcomes with references in Appendix B. Eighteen programs had no evidence of contributing to conservation outcomes; however, it is important to note that eight of these are less than a year old and may not have produced assessable evidence yet, such as peer-reviewed journal articles or policy reports. It is important to note that the presence or absence of evidence of an outcome was influenced by whether the program leaders were able to document outcomes or evaluate the program impacts; a lack of evidence could be due to a lack of documentation, not necessarily a lack of impact.

Table 4. Evidence of conservation activities by program type at three natural history museums.

	Bioblitzes and other Citizen Science Events 6 total projects	Ongoing Monitoring Programs 21 total projects	Bounded Field Research and Inventory Projects 14 total projects	Data Processing of Digitized Collections 4 total projects
NHM London 15 projects	*Alexandra Palace Park Bioblitz (M, R) *Wembury and other Bioblitzes	Bugs Count (R, E) Open Air Laboratories (OPAL) Programme (R, E) Riverfly Partnership/ Anglers′ Monitoring Initiative (M, P) The Big Seaweed Search Cetacean Strandings Information Programme (R, P) Chinese Mitten Crab Survey (M, R, P) Urban Tree Survey/LeafSnapUK	~ Microverse (E) Decoding Nature (M, R, E) ~Orchid Observers (R) The Bluebell Survey	~ Orchid Observers Herbaria@Home Notes from Nature (Zooniverse) – Ornithological
California Academy of Sciences 16 projects	*Pillar Point Intertidal Bioblitzes (M, E) ~Grassroots Bioblitzes in SF Bay Area (M, R)	Science Action Club (E) Living Roof Monitoring Pillar Point Intertidal Monitoring (M, R) Careers in Science Interns Summer program - LiMPETS (M, E) ~BioCaching app  ~ Marin MPAWatch Marine Mammal Stranding Network (R, P)	Marin Municipal Water District (MMWD) Plant Inventory (M, R, E) *SF Bay 2 K (R) * Mountain Lake Biological Monitoring *Bay Area Ant Survey (R) *Most Wanted Spider (R) *Rocky Shore Partnership Program	Notes from Nature (Zooniverse) - CalBug
NHM Los Angeles 13 projects	~ Citizen Science Meet-Ups  ~ BioblitzLA	Reptiles and Amphibians of Southern California (RASCals) (R, E, P) Southern California Squirrel Survey (R) Nature Garden Survey L. A. Nature Map L. A. Spider Survey (R) ~Snails and Slugs Living in Metropolitan Environments (SLIME) Marine Mammal Stranding Network (R, P)	*California Parrot Project (R) GeckoWatch (R) Biodiversity Science: City and Nature (BioSCAN) (R) ~*ButterflySCAN

Letters indicate whether we found evidence that the program contributed to Site and/or Species Management (M), Research (R), Education (E), Policy (P), Communities and Livelihoods (L). ~ = projects begun within the last year, * = projects that have ended. Orchid Observers is a hybrid project and therefore listed both as a Bounded Research and a Digitization project; however, counted only once towards total number of projects.

Table options

Three of the four types of programs had evidence of outcomes for at least one conservation activity (Table 5). Ongoing Monitoring Programs, by far the most common type of citizen science program offered by the three museums, had evidence of contributing to four of the conservation activity categories (only lacking evidence for impacts on Livelihoods). Bioblitzes and Bounded Field Research and Inventory Projects had evidence for three conservation activities (Site and Species Management, Research and Education). Only for Data Processing of Digitized Collections did we find no evidence of contributing to conservation outcomes. We discuss the possible reasons for this and the indirect contributions of this program type to conservation Research in the Discussion.

Table 5. Number of programs with evidence by type of conservation activity and type of program.^⁎

Type of program	Total no. of programs with evidence (26 of 44)	Site and Species Management (9; 20% of programs)	Research (21; 48% of programs)	Education (9; 20% of programs)	Policy (6; 14% of programs)	Livelihoods (0)
Bioblitz and other events	3 of 6	3	2	1	0	0
Ongoing monitoring	13 of 21	4	10	5	6	0
Bounded research	10 of 14	2	9	3	0	0
Data processing	0 of 4	0	0	0	0	0

⁎

Orchid Observers is a hybrid project and therefore listed as both as a Bounded Research and a Digitization project; however, counted only once towards total number of projects.

Table options

Below we present a summary of the evidence available and provide selected examples of the projects that contribute to each conservation activity, grouped by program type, as indicated in Table 4.

3.2.1. Evidence of Species and Site Management outcomes for conservation

All three museums had programs with Site and Species Management outcomes for conservation, across three program types—Bioblitz Events, Ongoing Monitoring, and Bounded Research.

3.2.1.1. Bioblitz Events

Three Bioblitz programs had evidence for Site and Species Management outcomes for conservation (Table 4 ;  Table 5). For example, the NHM London led a series of 24-h Bioblitz events involving amateur naturalists, adults, families and schools. After the Alexandra Palace Park Bioblitz, the local Council used the resulting data to help designate the park as a local nature reserve (granted in 2013). In addition, the management regime was modified based on Bioblitz data to better conserve locally rare acid grassland (Alexandra Palace Park Board 2012). Also, the Grassroots Bioblitz program at CAS partnered with Nerds for Nature (an all-volunteer conservation group) and local land managers; 17 Bioblitz events had resulted in 15,000 biological observations submitted through iNaturalist. Resulting data were used by the land managers for building species lists and to inform habitat management decisions at under-resourced parks; for example, one bioblitz found marine organisms after recent restoration re-connecting Lake Merritt in Oakland, CA to the San Francisco Bay that provided evidence that restoration tactics were proving successful.

3.2.1.2. Ongoing Monitoring

Four Ongoing Monitoring programs had evidence for Site and Species Management outcomes for conservation (Table 4 ;  Table 5). One example was the Riverfly Partnership/Anglers' Monitoring Initiative started at NHM London in 2004 in partnership with angler groups. It has become a collaborative network of trained volunteers across the UK that monitor river water quality by recording riverfly larvae. As well as supporting active conservation management and habitat restoration, the program served as an indicator of river health and early warning system for pollution incidents, and fed directly into reporting by the UK Environment Agency. Pollution detection by anglers had resulted in prosecutions and subsequent river cleaning (Environment Agency Wales, 2009).

3.2.1.3. Bounded Research

Two Bounded Research programs had evidence for Site and Species Management outcomes for conservation (Table 4 ;  Table 5). One of these, the Plant Inventory, was established in 2012 by CAS and the Marin Municipal Water District (MMWD) to document and collect every plant species on the highly biodiverse 18,000 + acres the MMWD manages. All data provided the MMWD with a baseline to help manage the watershed. One inventory event located a new population of the invasive Harding Grass (Phalaris aquatica) early enough to allow low-cost rapid eradication in sensitive habitat.

3.2.2. Evidence of Research outcomes for conservation

NHM citizen science programs contributed to Research outcomes more than any other type of conservation outcome: 21 of the 35 programs that had existed for more than a year showed evidence for this type of outcome, across all program types (Table 4 ;  Table 5). Ongoing Monitoring and Bounded Research projects most often contributed to these outcomes; approximately half of all of these program types had Research outcomes. Most of these outcomes involved finding species new to science, finding new species records for particular locations, and increasing understanding of species distributions and abundance. We provide examples here of significant research contributions, and a list of all key publications from these programs in Supplemental Appendix B.

3.2.2.1. Bioblitz Events

Two Bioblitz programs had evidence of contributions to biodiversity research. As an example, the NHM London's Alexandra Palace Park Bioblitz resulted in the 4th UK record for a rare Bolbitius fungus leading to ongoing research to determine if it should be classed as a species new to science. Such Research informed understanding of the distribution and abundance of rare or infrequently encountered species, in particular for under-recorded groups such as fungi and feeds into species conservation assessments.

3.2.2.2. Ongoing Monitoring

Ten Ongoing Monitoring programs had evidence of Research impacts for conservation. Three such programs at NHM LA were the Reptiles and Amphibians of Southern California (RASCals) project, the Southern California Squirrel Survey, and the L.A. Spider Survey, all of which compared historical museum records to modern observations submitted by volunteers. These projects had all resulted in multiple new locality records including new county and/or state records (e.g. Pauly et al., 2015).

3.2.2.3. Bounded Research

Nine Bounded Research projects had evidence of contributions to biodiversity research that informs conservation. For example, Orchid Observers at NHM London began in April 2015 and brought together biological recording of orchid flowering times and online crowdsourcing of digitized plant specimens to investigate phenological responses to climate change. Participants had already identified 200 new localities, some for sensitive and declining UK orchid species. In addition, Biodiversity Science: City and Nature (BioSCAN), started in 2012, was an urban insect survey at NHM LA for which volunteers hosted insect traps in their backyards and sent specimens to the museum. The project had produced five peer-reviewed publications at time of publication, including the description of 30 fly species new to science (Hartop et al., 2015).

3.2.3. Evidence of Education outcomes for conservation

All three museums had programs with conservation education outcomes, with evidence largely coming from participant surveys (questionnaires or interviews). Education outcomes occurred across three program types - Bioblitz Events, Ongoing Monitoring, and Bounded Research. A range of conservation education outcomes were represented, including increase in knowledge about the natural history of the site or the science process, interest or self-efficacy toward environmental science and science in general.

3.2.3.1. Bioblitz Events

Pillar Point Intertidal Bioblitz from CAS was the only Bioblitz that has evidence of contributing to conservation education of participants. Follow-up email questionnaire surveys sent to 30 participants with 24 responses showed that 82% reported an increase in their knowledge about the biodiversity of the site, 41% reported an increase in their understanding of the natural history of Pillar Point and 67% reported an increase in their understanding of their role in the scientific process.

3.2.3.2. Ongoing Monitoring

Five Ongoing Monitoring programs had evidence of impacts on the education outcomes for participants. Here we give one example from each museum. At NHM London, evaluation of participants in the OPAL program revealed that 50% of respondents stated this was their first time carrying out a biological survey, 90% said they had learned something new about their place, and 83% said they had developed new skills (OPAL Community Environment Report 2013). For the Careers in Science Interns at CAS doing the LiMPETS project, interviews revealed youth participants develop a sense of agency toward science and environmental science, and develop a stronger identity with respect to their own understanding and role in science (Ballard et al. this issue). At the NHM LA, RASCals participants (n = 123) were given an online questionnaire in 2013, with 30 respondents (24%). As a reflection of self-efficacy toward science, 80% of respondents stated that they felt they had gained understanding in science topics and confidence in explaining science topics to others.

3.2.3.3. Bounded Research

Three Bounded Research projects had evidence of impacts on the education outcomes for participants. One example comes from NHM London's Decoding NAture project, which administered evaluation questionnaires to over 600 participating students. Over 95% had changed their perception of scientists. Pre- and post-session evaluation questionnaires given to 200 of these students provided evidence that 95% of students increase their knowledge of DNA and improved their identification skills. In addition, follow-up email questionnaires sent by CAS to 73 MMWD Plant Inventory participants had 25 responses and show that 96% reported an increase in their knowledge about the biodiversity of the site, 83% reported an increase in their understanding of local natural history, and 74% reported an increase in their understanding of their role in the scientific process.

3.2.4. Evidence of Policy outcomes for conservation

Only Ongoing Monitoring projects had evidence of conservation policy outcomes, but a surprising number were in this category (six of the 21 projects) (Table 4 ;  Table 5). As one example, the Riverfly Partnership developed a Biodiversity Action Plan which was included as part of the UK Government's response to the Convention on Biological Diversity. This plan included recovery strategies for threatened species and habitats based on the Riverfly Partnership's work to improve knowledge of eight species of aquatic invertebrates to inform conservation actions by government actors (Kindemba, 2010).

3.2.5. Evidence of Livelihood outcomes for conservation

None of the analyzed programs had gathered evidence that their programs resulted in livelihood outcomes as defined above.

4. Discussion

4.1. What kinds of programs and strategies of NHM citizen science particularly contribute to conservation activities?

Our findings indicate some key ways NHM citizen science contributes to conservation outcomes, yet we acknowledge that absence of evidence doesn't necessarily mean there are not contributions to an area of conservation, and may only reflect that contributions have not been measured or documented in a way that can be critically and independently assessed. While not a focus of this study, we did see that the ways and extent to which the public participated in programs, a major focus of the broader field of citizen science (Shirk et al., 2012), may have played a role in the ability of programs to contribute to conservation outcomes. We found that Ongoing Monitoring programs, in which members of the public often participated for many years and sometimes took on leadership roles, contributed most frequently to a range of conservation outcomes (Table 5). While not “co-created” as defined by Shirk et al. (2012), this more intensive participation might have facilitated stronger impacts on education and policy as well as research outcomes.

4.1.1. Site and Species Management

Half of all bioblitz programs contributed to site and species management, proportionally more than other program types. This was primarily by providing partners with data necessary for better management of the surveyed site. Regardless of program type, having a land management partner was a key factor for producing site and species management outcomes.

4.1.2. Research

Ongoing Monitoring programs contributed more frequently to conservation research than the other programs types. The success of these programs reflects their alignment with the primary type of science done by NHMs: documenting and describing biodiversity. Data Processing of Digitized Collections programs did not at time of publication have evidence of Research outcomes; however, all of the programs in this category were recently established. There is increasing evidence that citizen science-based specimen digitization programs frequently contribute to research in an indirect way, by liberating large datasets and making these freely available online for researchers worldwide to use (Ellwood et al., 2015). An emerging challenge is how we can track and assess the onward use of these data, including their value for conservation-related activity.

4.1.3. Education

Ongoing Monitoring and Bounded Research were the program types most likely to have education outcomes for conservation, and most were regarding knowledge gains in understanding science and an increased connection to place. These findings are consistent with other biologically-focused citizen science programs (Brossard et al., 2009; Cooper et al., 2009 ;  Crall et al., 2013). This is not surprising, as these types of citizen science projects usually engage volunteers beyond a single event, giving participants more exposure to scientific content and process, local conservation issues, and programs that have had more time for evaluation; this is also consistent with Bonney et al.’s (2015) review of citizen science's impacts on public understanding of science. While we found much evidence for impacts on science education outcomes, we found little evidence that participants were or intended to engage in additional conservation activities. This may again be a reflection of the focus of the program evaluations, which are often determined by funding sources targeting science education, rather than lack of impact. If programs have the goal of increasing conservation actions by participants as suggested by Silvertown et al. (2013), this needs to be inherent in the design and evaluation of the program.

4.1.4. Policy

We found that only Ongoing Monitoring programs had contributed to policy. Importantly, most had been monitoring for longer than the other types of programs and all had partnerships with, or work in support of, government agencies. As non-profits and non-partisan organizations that are often government funded, most NHMs don't have a mandate or mission to directly influence policy, though NHM London's Science Strategy includes supporting policy development through evidence-based science. However, museums can provide invaluable scientific expertise - both with researcher-collected and citizen scientist-collected data - that can inform policy decisions through partnerships with government agencies and conservation organizations. There is growing evidence that citizen science can inform policy throughout most stages of the policy process (e.g. McKinley et al., 2017) and we predict that this will be a growth area for NHMs and other publicly-funded organizations that are increasingly required to actively demonstrate their societal relevance and value.

4.1.5. Livelihoods

Our findings showed a distinct lack of evidence of impacts on livelihoods; however NHM citizen science is ripe with possibilities in this area. Citizen science as a field struggles with strategies for documenting evidence of impacts on communities and socio-ecological systems' resilience. Because the field is moving towards being able to measure community outcomes such as social capital and community capacity (Jordan et al., 2012), and connection to place (Haywood, 2014), livelihoods is an area in which museums could have a big effect. NHMs are institutions rooted in place and could have strong connections to their local neighborhoods if programs are designed for this.

4.2. What influences the ability of NHM citizen science to impact conservation?: Best Practices

4.2.1. Collaboration across traditional museum departments

Most of the programs that had two or more different conservation activities were the result of a close collaboration between NHM staff from different departments. For example, all four programs that had evidence of contributions to three different conservation activities had similar implementation strategies involving collaborations between research and education departments. In the case of RASCals at NHM LA, the curator developed the research questions and educators were heavily involved in recruitment and training. For Decoding NAture, RASCals and the MMWD Plant Inventory, a tight collaboration between the curators and the education experts on the Citizen Science Team allowed strategic evaluation of participants, which impacted study design and recruitment efforts as well as documented educational outcomes. This finding is relevant to other conservation organizations that have similar dual goals of research and education, where both sides bring skills and expertise, rather than seeing citizen science as an either/or endeavour as it is often portrayed (Bonney et al., 2014).

4.2.2. Form and Maintain Partnerships

Examining the structure of the programs that had evidence for conservation outcomes, we found that the majority were structured as collaborations across multiple institutions (Table 4 and Appendix A). Regardless of program type, all the programs with Site and Species Management impacts were built on partnerships with local resource management or parks organizations. Museums are highly visible partners and can garner large participation for small organizations. Partnering organizations can also help shape program goals and design to maximize the usefulness of the data to inform future management decisions. While many of the programs with Research outcomes were led by museum scientists, some were authored in collaboration with external scientists – allowing more scientists to leverage the wide reach of NHMs to broad audiences. For Education outcomes, programs involving youth (Microverse, Careers in Science – LiMPETS, SAC, Decoding NAture) required partnering with schools and after-school programs and, consistent with the research on school-community partnerships (Monroe et al., 2015 ;  Uzzell, 1999), can provide youth with opportunities to do authentic science and interact with community members and scientists in an unprecedented way.

4.3. Gaps Revealed: a Need to Maximize the Conservation Impacts of NHM Citizen Science

4.3.1. Reaching diverse audiences in urban areas

Because NHM citizen science targets biodiversity in urban ecosystems in particular, these programs potentially provide significant opportunities to engage urban audiences in a unique way. Evidence for whether museums successfully engage audiences that mirror the full diversity of local urban populations is mixed, however. For example, within the UK, recent reports show that science-based museums are successfully engaging more ethnically diverse audiences (UK Association for Science and Discovery Centres, 2014), but that cultural organizations, including museums, often have narrower social, economic, ethnic and educated visitor demographics than society as a whole (Warwick Commission, 2015). Because the field of citizen science struggles with the relatively narrow sector of society who typically participates (Evans et al., 2005; Pandya, 2012 ;  Trumbull et al., 2000), museums may be well-placed to support broadening of participation in citizen science. However, there is still some way to go. Our study found evidence that some NHM-led citizen science projects were successful in reaching more ethnically diverse audiences that represent the local population more equitably. For example, the RASCals program at NHM LA gained most data from participants in diverse urban neighborhoods, with the curator and museum staff engaging residents in their neighborhoods in person. Providing a way to participate in a project in one's own neighborhood can offer a locally-accessible and relevant gateway into conservation actions which, consistent with other research on participatory approaches to environmental education, provide participants with appreciation for the natural world and for their local place (Krasny et al., 2014). The lack of data on the demographics of participants across citizen science (Soleri et al., 2016), and particularly NHM citizen science programs, is a challenge for research and evaluation in the field as a whole.

4.3.2. Document who is using the citizen science-collected data submitted to the large national/international databases and how

In the cases where our findings revealed a lack of evidence for conservation research outcomes for programs, we see a common challenge for the field of citizen science in general. We need better ways to track how data submitted by volunteers are eventually contributing to advancing our knowledge of biodiversity through journal articles, resource management decisions, government policies and agency decisions, and climate change research. NHMs are the home for many national and international databases, and have a great potential for building systems to track use of citizen science data. Until the downstream use of such data is tracked effectively (e.g. via Digital Object Identifiers - unique serial codes assigned to datasets and other digital objects, that are referenced when a given dataset is used in research or publications) and attribution of such use is commonplace, the conservation benefits are likely to remain at best under-estimated, and at worst undetected.

4.3.3. Improve and expand evaluation of citizen science programs

The lack of evidence for many of these programs may simply be due to the lack of evaluation. Our findings indicated that projects designed with strategic conservation goals in mind and/or that included a well-planned evaluation program were most effective, reinforcing calls for careful design of programs (Shirk et al., 2012). The field of citizen science has only relatively recently begun to evaluate for education and other programmatic outcomes (Jordan et al., 2012 ;  Phillips et al., 2014). However museum educational research and evaluation have been developing for decades and possess useful evaluation tools that could be adapted to NHM citizen science (Dillon, 2003 ;  Falk and Dierking, 1992). Guidelines for evaluating citizen science programs have recently become widely available (Phillips et al., 2014), and the Citizen Science Association has formed a Research and Evaluation Working Group to link and leverage evaluations of projects across the hundreds of members.

4.3.4. Share learning and coordinate activities across institutions

Johnson et al. (2011) have previously recommended that priority be placed on identifying and digitizing collections holdings that are most suitable for climate change research, whilst Ellwood et al. (2015) identified the need to develop and share best practice and standards documentation for crowdsourced digitization projects. We additionally recommend that institutions increase efforts to share learning on successful (and less successful) approaches to NHM-based citizen science across institutions. This will both enable further consideration of the key factors that positively impact conservation outcomes and foster collaborations that address grand challenges in conservation. The growing number of national and international citizen science associations, journals (e.g. Citizen Science: Theory and Practice), and communities of practice provide ready-made avenues through which such discussions can progress (Citizen Science Association, European Citizen Science Association, Australian Citizen Science Association).

4.4. Innovations for citizen science by NHMs for conservation and biodiversity research

4.4.1. Studying impacts of environmental change and the spread of non-native species by bringing together historical collections and current field recording/monitoring

For two of the three programs with contributions to three different conservation outcomes, the Chinese Mitten Crab Survey at NHM London and RASCals at NHM LA, the lead scientists specifically used the data to ask conservation-relevant questions about invasive species that informed both research and policy. Tracking species invasions and range shifts, not just in urban areas, but globally, is a major contribution that citizen science has started making to conservation research and site and species management (Cooper et al., 2009). There is huge potential for the long time-series data contained within museum collections to support contemporary conservation activity (e.g. through understanding species responses to climate change; Johnson et al., 2011), and crowd-sourced digitization efforts clearly have a large part to play in this process (e.g. Ellwood et al., 2015). NHM citizen science could particularly contribute to conservation through programs that leverage historical collections combined with current field-based citizen science to more accurately understand change that has already occurred and better-predict future change, such as the Orchid Observers project which provided an online platform where citizen scientists extracted flowering dates from both historical collections and contemporary photographs of orchids, feeding into a single database for analysis of a 180 year time-series.

4.4.2. Broadening participation in science

We found CAS and NHM LA citizen science programs had begun to address the challenge of broadening participation and mobilizing new audiences with innovative new forms of Bioblitzes and other citizen science events. These programs represent innovative ways of reaching audiences who haven't traditionally participated in citizen science nor even felt “welcome” in the outdoor recreational or environmental movements. For example, Latino families went to citizen science camp-outs in Los Angeles for the first time. Grassroots Bioblitzes from CAS met people in their own neighborhoods and parks and were organized by local managers and educators. Further, there is some evidence that those who uploaded observations to iNaturalist had not used iNaturalist prior to their first bioblitz, and about one third of those continued to make iNaturalist observations afterward. These opportunities introduced people to the natural world and facilitated participation in science using strategies that draw from informal science and environmental education research, specifically working with neighborhood and community-based groups, and offering bilingual materials and experiences (Bell et al., 2009; Krasny et al., 2014; Pandya, 2012 ;  Stern et al., 2010).

4.4.3. Integrating multi-taxa projects, multiple entry-points and tiered levels of engagement

Museums are uniquely positioned to layer the different types of citizen science projects described above to design more impactful programs. NHMs have diverse scientific and collections expertise united in a single institution. If integrated and designed well, people can participate in multiple ways, intensively or not, from short-term bioblitzes to committed long-term monitoring of a site. For example, NHM LA's new SuperProject allowed researchers to examine factors structuring biodiversity across the urban landscape by recruiting hundreds of site hosts who will participate in up to six individual citizen science projects for their site. This kind of program integration, internally at one NHM or through collaboration across institutions can increase participation in multiple citizen science projects and generate sites with biodiversity data for multiple taxonomic groups.

5. Conclusions

The citizen science programs at these three museums had a measurable impact on several aspects of conservation. Their combined programs had reached 78,000 participants over the last 20 years and contributed to at least 30 publications. Evidence of the impacts of NHM citizen science to inform biodiversity research is significant. NHMs have researchers in-house with the goal of contributing to our understanding of the natural world; as long as NHM citizen science involves those researchers, it will likely be a fruitful source of research to inform conservation. Evidence of what those participants have learned or gained, and whether they have subsequently changed their behavior to better conserve biodiversity, is lacking, but that is true for much of the field of environmental education. Better evaluation and research on conservation education outcomes of NHM citizen science is clearly needed.

What is unique about NHM citizen science for conservation is the built-in mission of NHMs to contribute to society's understanding of biodiversity both through research and through educational programming, with which many NHMs have over 100 years of experience. Thus, NHM citizen science is a natural outgrowth of this history, but at the same time is a key way that NHMs can evolve and transform to better meet the 21st century challenges of global environmental change, digital divides, concerns about public scientific literacy, and society's disconnection from nature (Winker, 2004). At the same time, the field of citizen science is increasingly impacting conservation research and decision-making; however, its full potential can't be realized until it is more inclusive of a broader audience. Everyone should feel welcome and have the choice to participate in science. Millions of visitors of all ages, classes, races, and cultures visit NHMs every year, with the goal of having fun and learning more about the natural world. Leveraging this broad appeal can help a vast new constituency feel welcome to participate in biodiversity research programs that support conservation science. It also has the potential to inspire new conservation-minded behaviors within broad and diverse demographics of society, a critical and growing challenge for global biodiversity conservation.

Role of funding sources

This research was not conducted with the support of any funding from organizations other than authors' institutions.

Acknowledgements

We want to acknowledge and thank all of the citizen science volunteers and researchers who have donated their time and energy in all three NHMs programs. We also thank the three anonymous reviewers for their constructive suggestions which improved the manuscript.

Appendix A. Supplementary data

Supplementary material Appendix A. Program Information for citizen science programs for NHM London, California Academy of Sciences and NHM Los Angeles (Dates, location, participants, scientific goals and means of participation for volunteers). Appendix B. References used as evidence of contributions to conservation outcomes.

Help with PDF files

Options

References

1. • Anderson, 2012
   • R.P. Anderson
   • Harnessing the world's biodiversity data: promise and peril in ecological niche modeling of species distributions
   • Ann. N. Y. Acad. Sci., 1260 (2012), pp. 66–80
   • 
2. • Bell et al., 2009
   • P. Bell, B. Lewenstein, A.W. Shouse, M.A. Feder
   • Learning Science in Informal Environments: People, Places, and Pursuits
   • National Academies Press (2009)
   • 
3. • Bonney et al., 2009
   • R. Bonney, C.B. Cooper, J. Dickinson, S. Kelling, T. Phillips, K.V. Rosenberg, J. Shirk
   • Citizen science: a developing tool for expanding science knowledge and scientific literacy
   • Bioscience, 59 (11) (2009), pp. 977–984
   • 
4. • Bonney et al., 2014
   • R. Bonney, J.L. Shirk, T.B. Phillips, A. Wiggins, H.L. Ballard, A.J. Miller-Rushing, J.K. Parrish
   • Next steps for citizen science
   • Science, 343 (6178) (2014), pp. 1436–1437
   • 
5. • Bonney et al., 2015
   • R. Bonney, T. Phillips, H. Ballard, J. Enck
   • Can citizen science enghance public understanding of science?
   • Public Underst. Sci. (2015)
   • 
6. • Brossard et al., 2009
   • D. Brossard, B. Lewenstein, R. Bonney
   • Scientific knowledge and attitude change: the impact of a citizen science project
   • Int. J. Sci. Educ., 27 (2009), pp. 1099–1121
   • 
7. • Conrad and Hilchey, 2011
   • C.C. Conrad, K.G. Hilchey
   • A review of citizen science and community-based environmental monitoring: issues and opportunities
   • Environ. Monit. Assess., 176 (1–4) (2011), pp. 273–291
   • 
8. • Cooper et al., 2009
   • C.B. Cooper, J. Dickinson, T. Phillips, R. Bonney
   • Citizen science as a tool for conservation in residential ecosystems
   • Ecol. Soc., 12 (2) (2009), p. 11
   • 
9. • Couvet et al., 2008
   • D. Couvet, F. Jiguet, R. Julliard, H. Levrel, A. Teyssedre
   • Enhancing citizen contributions to biodiversity science and public policy
   • Interdiscip. Sci. Rev., 33 (2008), pp. 95–103
   • 
10. • Crall et al., 2013
    • A.W. Crall, R. Jordan, K. Holfelder, G.J. Newman, J. Graham, D.M. Waller
    • The impacts of an invasive species citizen science training program on participant attitudes, behavior, and science literacy
    • Public Underst. Sci., 22 (6) (2013), pp. 745–764
    • 
11. • Dillon, 2003
    • J. Dillon
    • On learners and learning in environmental education: missing theories, ignored communities
    • Environ. Educ. Res., 9 (2) (2003), pp. 215–226
    • 
12. • Drew, 2011
    • J. Drew
    • The Role of Natural History Institutions and Bioinformatics in Conservation Biology
    • Conserv. Biol., 25 (6) (2011), pp. 1250–1252
    • 
13. • Ellwood et al., 2015
    • E.R. Ellwood, B.A. Dunckel, P. Flemons, R. Guralnick, G. Nelson, G. Newman, S. Newman, D. Paul, G. Riccardi, N. Rios, K.C. Seltmann, A.R. Mast
    • Accelerating the Digitization of Biodiversity Research Specimens through Online Public Participation
    • Bioscience, 65 (4) (2015), pp. 383–396
    • 
14. • Environment Agency Wales, 2009
    • Environment Agency Wales
    • News Release 12 May 2009: Partnership proves successful in reducing pollution
    • http://www.riverflies.org/sites/172.16.0.99.riverflies.local/files/Press%20release.pdf (2009) (Accessed 04 Sept 2015)
    • 
15. • Evans et al., 2005
    • C. Evans, E. Abrams, R. Reitsma, K. Roux, L. Salmonsen, P.P. Marra
    • The Neighborhood Nestwatch Program: Participant Outcomes of a Citizen-Science Ecological Research Project
    • Conserv. Biol., 19 (3) (2005), pp. 589–594
    • 
16. • Falk, 2005
    • J.H. Falk
    • Free-choice science learning: framing the discussion
    • Environ. Educ. Res., 11 (3) (2005), pp. 265–280
    • 
17. • Falk and Dierking, 1992
    • J.H. Falk, L.D. Dierking
    • The museum experience
    • Howells House, Washington, DC (1992)
    • 
18. • Fernandez-Gimenez et al., 2008
    • M.E. Fernandez-Gimenez, H. Ballard, V.E. Sturtevant
    • Adaptive Management and Social Learning in Collaborative and Community-Based Monitoring: A Study of Five Community-Based Forestry Organizations in the western USA
    • Ecol. Soc., 13 (2) (2008), p. 4
    • 
19. • Gaubert et al., 2006
    • P. Gaubert, M. Papes, A.T. Peterson
    • Natural history collections and the conservation of poorly known taxa: Ecological niche modeling in central African rainforest genets (Genetta spp.)
    • Biol. Conserv., 130 (2006), pp. 106–117
    • 
20. • Gura, 2013
    • T. Gura
    • Citizen science: amateur experts
    • Nature, 496 (7444) (2013), pp. 259–261
    • 

1. • Hacker and Harris, 1992
   • G.G. Hacker, M. Harris
   • Adult learning of science for scientific literacy: some theoretical and methodological perspectives
   • Stud. Educ. Adults, 24 (1992), pp. 217–225
   • 
2. • Haklay, 2013
   • M. Haklay
   • Citizen science and volunteered geographic information: Overview and typology of participation
   • Crowdsourcing geographic knowledge, Springer, Netherlands (2013), pp. 105–122
   • 
3. • Hartop et al., 2015
   • E.A. Hartop, B.V. Brown, R.H.L. Disney
   • Opportunity in our Ignorance: Urban Biodiversity Study Reveals 30 New Species and One New Nearctic Record for Megaselia (Diptera: Phoridae) in Los Angeles (California, USA)
   • Zootaxa, 3941 (4) (2015), pp. 451–484
   • 
4. • Haywood, 2014
   • B.K. Haywood
   • A “sense of place” in public participation in scientific research
   • Sci. Educ., 98 (1) (2014), pp. 64–83
   • 
5. • Hill et al., 2012
   • A. Hill, R. Guralnick, A. Smith, A. Sallans, R. Gillespie, M. Denslow, J. Gross, Z. Murrell, T. Conyers, P. Oboyski, J. Ball, A. Thomer, R. Prys-Jones, J. de la Torre, P. Kociolek, L. Fortson
   • The notes from nature tool for unlocking biodiversity records from museum records through citizen science
   • ,in: V. Blagoderov, V.S. Smith (Eds.), No specimen left behind: mass digitization of natural history collections, ZooKeys, vol. 209 (2012), pp. 219–233 http://dx.doi.org/10.3897/zookeys.209.3472
   • 
6. • Hoeksema et al., 2011
   • B.W. Hoeksema, J. van der Land, S.E.T. van der Meij, L.P. van Ofwegen, B.T. Reijnen, R.W.M. van Soest, N.J. de Voogd
   • Unforeseen importance of historical collections as baselines to determine biotic change of coral reefs: the Saba Bank case
   • Mar. Ecol., 32 (2011), pp. 135–141
   • 
7. • Jacobson et al., 2006
   • S.K. Jacobson, M.D. McDuff, M.C. Monroe
   • Conservation education and outreach techniques
   • Oxford, New York (2006)
   • 
8. • Johnson et al., 2011
   • K.G. Johnson, S.J. Brooks, P.B. Fenberg, A.G. Glover, K.E. James, A.M. Lister, E. Michel, M. Spencer, J.A. Todd, E. Valsami-Jones, J.R. Young, J.R. Stewart
   • Climate Change and Biosphere Response: Unlocking the Collections Vault
   • Bioscience, 61 (2) (2011), pp. 147–153
   • 
9. • Jordan et al., 2012
   • R.C. Jordan, H. Ballard, T.B. Phillips
   • Key issues and new approaches for evaluating citizen-science learning outcomes
   • Front. Ecol. Environ., 10 (2012), pp. 307–309
   • 
10. • Kapos et al., 2008
    • V. Kapos, A. Balmford, R. Aveling, P. Bubb, P. Carey, A. Entwistle, J. Hopkins, T. Mulliken, R. Safford, A. Stattersfield, M. Walpole, A. Manica
    • Calibrating conservation: new tools for measuring success
    • Conserv. Lett., 1 (2008), pp. 155–164
    • 
11. • Kindemba, 2010
    • V. Kindemba
    • Rescuing our riverflies
    • Biodiversity News, 49 (22) (2010) United Kingdom Department for Environment, Food and Rural Affairs (Accessed 04 Sept 2015) http://jncc.defra.gov.uk/pdf/ukbap_biodiversitynews-49.pdf
    • 
12. • Krasny et al., 2014
    • M.E. Krasny, A. Russ, K.G. Tidball, T. Elmqvist
    • Civic ecology practices: Participatory approaches to generating and measuring ecosystem services in cities
    • Ecosyst. Serv., 7 (2014), pp. 177–186
    • 
13. • Krishtalka and Humphrey, 2000
    • L. Krishtalka, P.S. Humphrey
    • Can natural history museums capture the future?
    • Bioscience, 50 (7) (2000), pp. 611–617
    • 
14. • McKee, 2005
    • B. McKee
    • Growing up Denatured
    • The New York Times (2005), p. F1 (April 28)
    • 
15. • McKinley et al., 2017
    • D.C. McKinley, A.J. Miller-Rushing, H.L. Ballard, R. Bonney, H. Brown, S. Cook-Patton, D.M. Evans, R.A. French, J.K. Parrish, T.B. Phillips, S.F. Ryan, L.A. Shanley, J.L. Shirk, K.F. Stepenuck, J.F. Weltzin, A. Wiggins, O.D. Boyle, R.D. Briggs, S.F. Chapin III, D.A. Hewitt, P.W. Preuss, M.A. Soukup
    • Citizen Science Can Improve Conservation Science, Natural Resource Management, and Environmental Protection
    • Biol. Conserv., 208 (2017), pp. 15–28
    • 
16. • Miller et al., 2004
    • B. Miller, W. Conway, R.P. Reading, C. Wemmer, D. Wildt, D. Kleiman, S. Monfort, A. Rabinowitz, B. Armstrong, M. Hutchins
    • Evaluating the conservation mission of zoos, aquariums, botanical gardens, and natural history museums
    • Conserv. Biol., 18 (1) (2004), pp. 86–93
    • 
17. • Monroe, 2003
    • M.C. Monroe
    • Two avenues for encouraging conservation behaviors
    • Hum. Ecol. Rev., 10 (2) (2003), pp. 113–125
    • 
18. • Monroe et al., 2015
    • M.C. Monroe, H.L. Ballard, A. Oxarart, V.E. Sturtevant, P.J. Jakes, E.R. Evans
    • Agencies, educators, communities and wildfire: partnerships to enhance environmental education for youth
    • Environ. Educ. Res. (2015), pp. 1–17
    • 
19. • Pandya, 2012
    • R.E. Pandya
    • A framework for engaging diverse communities in citizen science in the US
    • Front. Ecol. Environ., 10 (6) (2012), pp. 314–317
    • 
20. • Pauly et al., 2015
    • G.B. Pauly, G.S. Yoshida, R. Worrell
    • Geographic distribution: USA, California: Hemidactylus garnotii
    • Herpetol. Rev., 46 (2015), p. 569
    • 

1. • Phillips et al., 2014
   • T.B. Phillips, M. Ferguson, M. Minarchek, N. Porticella, R. Bonney
   • User's Guide for Evaluating Learning Outcomes in Citizen Science
   • Cornell Lab of Ornithology, Ithaca, NY (2014)
   • 
2. • Robbirt et al., 2010
   • K.M. Robbirt, A.J. Davy, M.J. Hutchings, D.L. Roberts
   • Validation of biological collections as a source of phenological data for use in climate change studies: a case study with the orchid Ophrys sphegodes
   • J. Ecol., 99 (1) (2010), pp. 235–241
   • 
3. • Shaffer et al., 1998
   • H.B. Shaffer, R.N. Fisher, C. Davidson
   • The role of natural history collections in documenting species declines
   • Trends Ecol. Evol., 13 (1) (1998), pp. 27–30
   • 
4. • Shirk et al., 2012
   • J.L. Shirk, H.L. Ballard, C.C. Wilderman, T. Phillips, A. Wiggins, R. Jordan, E. McCallie, M. Minarchek, B.V. Lewenstein, M.E. Krasny, R. Bonney
   • Public participation in scientific research: a framework for deliberate design
   • Ecol. Soc., 17 (2) (2012), p. 29
   • 
5. • Silvertown et al., 2013
   • J. Silvertown, C.D. Buesching, S. Jacobson, T. Rebelo
   • Citizen science and nature conservation
   • D.W. Macdonald, K.J. Willis (Eds.), Key Topics in Conservation Biology 2, John Wiley & Sons, Ltd. Published (2013), pp. 124–142 (2013)
   • 
6. • Soleri et al., 2016
   • D. Soleri, J. Long, M. Ramirez-Andreotta, R. Eitemiller, R. Pandya
   • Finding Pathways to More Equitable and Meaningful Public-Scientist Partnerships
   • Citizen Science: Theory and Practice, 1 (1) (2016)
   • 
7. • Spooner et al., 2015
   • F. Spooner, R.K. Smith, W.J. Sutherland
   • Trends, biases and effectiveness in reported conservation interventions
   • Conserv. Evid., 12 (2015), pp. 2–7
   • 
8. • Stern et al., 2010
   • M.J. Stern, R.B. Powell, N.M. Ardoin
   • Evaluating a constructivist and culturally responsive approach to environmental education for diverse audiences
   • J. Environ. Educ., 42 (2) (2010), pp. 109–122
   • 
9. • Suarez and Tsutsui, 2004
   • A.V. Suarez, N.D. Tsutsui
   • The Value of Museum Collections for Research and Society
   • Bioscience, 54 (1) (2004), pp. 66–74
   • 
10. • Sullivan et al., 2014
    • B.L. Sullivan, J.L. Aycrigg, J.H. Barry, R.E. Bonney, N. Bruns, C.B. Cooper, T. Damoulas, A.A. Dhondt, T. Dietterich, A. Farnsworth, D. Fink, et al.
    • The eBird enterprise: an integrated approach to development and application of citizen science
    • Biol. Conserv., 169 (2014), pp. 31–40
    • 
11. • Sullivan et al., 2017
    • B.L. Sullivan, T. Phillips, A.A. Dayer, C.L. Wood, A. Farnsworth, M.J. Iliff, I.J. Davies, A. Wiggins, D. Fink, W. Hochachka, A.D. Rodewald, K.V. Rosenberg, R. Bonney, S. Kelling
    • Using open access observational data for conservation action: a case study for birds
    • Biol. Conserv., 208 (2017), pp. 5–14
    • 
12. • Theobald et al., 2015
    • E.J. Theobald, A.K. Ettinger, H.K. Burgess, L.B. Debey, N.R. Schmidt, H.E. Froehlich, C. Wagner, J. HilleRisLambers, J. Tewksbury, M.A. Harsch, J.K. Parrish
    • Global change and local solutions: Tapping the unrealized potential of citizen science for biodiversity research
    • Biol. Conserv., 181 (2015), pp. 236–244
    • 
13. • Trumbull et al., 2000
    • D.J. Trumbull, R. Bonney, D. Bascom, A. Cabral
    • Thinking scientifically during participation in a citizen-science project
    • Sci. Educ., 84 (2000), pp. 265–275
    • 
14. • UK Association for Science and Discovery Centres, 2014
    • UK Association for Science and Discovery Centres
    • UK Science and Discovery Centres: Effectively engaging under-represented groups. May 7, 2014 Report
    • http://sciencecentres.org.uk/reports/underserved/UK%20Science%20and%20Discovery%20Centres%3B%20Effectively%20engaging%20under-represented%20groups%20(May%207%202014).pdf (2014)
    • 
15. • Uzzell, 1999
    • D. Uzzell
    • Education for Environmental Action in the Community: New Roles and Relationships
    • Camb. J. Educ., 29 (3) (1999), pp. 397–413
    • 
16. • Warwick Commission, 2015
    • Warwick Commission
    • Enriching Britain: Culture, Creativity and Growth. The 2015 Report of the Warwick Commission on the Future of Cultural Value
    • http://www2.warwick.ac.uk/research/warwickcommission/futureculture/finalreport/warwick_commission_report_2015.pdf (2015)
    • 
17. • Western and Wright, 1994
    • D. Western, M. Wright
    • Natural connections: perspectives in community-based conservation
    • Island Press (1994)
    • 
18. • Wiggins and Crowston, 2011
    • A. Wiggins, K. Crowston
    • From conservation to crowdsourcing: A typology of citizen science. System Sciences (HICSS)
    • 2011 44th Hawaii international conference on, IEEE (2011)
    • 
19. • Winker, 2004
    • K. Winker
    • Natural History Museums in a Post-biodiversity Era
    • Bioscience, 54 (5) (2004), pp. 455–459
    • 

Corresponding author.

1

CAS has recently acquired iNaturalist, an online website and app that allows users to submit and share species observations and crowdsource their identification. Although CAS is supporting and has made a significant investment in iNaturalist, it is not included in our analysis because iNaturalist is a platform and community that hosts numerous projects, but it is not a citizen science project itself.

© 2016 The Authors. Published by Elsevier Ltd.