Eleanor D. Brown|Byron K. Williams
The potential for citizen science to produce reliable and useful information in ecology
United States Geological Survey|The Wildlife Society|United States Geological Survey
examined features citizen science influence data quality inferential power usefulness ecology background context examination considered topics ecological sampling probability based purposive opportunistic linkage sampling technique statistical inference design based model based scientific paradigms confirmatory exploratory distinguished several types citizen science investigations intensive research rigorous protocols targeting clearly articulated questions massparticipation internetbased projects opportunistic data collection lacking sampling design examined overarching objectives design analysis volunteer training performance identified key features influence data quality project objectives design analysis volunteer training performance projects good designs trained volunteers professional oversight meet statistical criteria produce highquality data strong inferential power therefore well suited ecological research objectives projects opportunistic data collection little sampling design minimal volunteer training better suited general objectives related public education data exploration reliable statistical estimation difficult impossible cases statistically robust analytical methods external data may increase inferential power certain opportunistically collected data ecological management especially government agencies frequently requires data suitable reliable inference standardized protocols stateoftheart analytical methods wellsupervised programs citizen science make valuable contributions conservation increasing scope species monitoring efforts data quality improved adhering basic principles data collection analysis designing studies provide data quality required including suitable statistical expertise thereby strengthening science aspect citizen science enhancing acceptance scientific community decision makers potencial de la ciencia ciudadana para producir informacin til confiable en la ecologa examinamos las caractersticas de la ciencia ciudadana que influyen sobre la calidad de datos el poder inferencial la utilidad en la ecologa consideramos temas como el muestreo ecolgico basado en probabilidad deliberado oportunista la conexin entre la tcnica de muestreo la inferencia estadstica basada en diseo basada en modelo los paradigmas cientficos confirmatorio exploratorio como trasfondo contextual para nuestra evaluacin distinguimos varios tipos de investigacin de ciencia ciudadana desde investigacin intensiva con protocolos rigurosos enfocados en preguntas claramente articuladas hasta proyectos de participacin masiva en plataformas de internet con recoleccin de datos oportunistas carentes de un diseo de muestreo examinamos los objetivos generales el diseo el anlisis la preparacin de los voluntarios el desempeo identificamos caractersticas clave que influyen sobre la calidad de los datos los objetivos del proyecto el diseo el anlisis la preparacin el desempeo de los voluntarios los proyectos con buenos diseos voluntarios preparados supervisin profesional pueden cumplir con criterios estadsticos para producir datos de alta calidad con un fuerte poder inferencial por lo tanto son muy adecuados para los objetivos de investigacin ecolgica los proyectos con una recoleccin oportunista de datos un diseo de muestreo nfimo nulo una preparacin mnima de los voluntarios son ms adecuados para los objetivos generales relacionados con la educacin pblica la exploracin de datos ya que la estimacin estadstica confiable puede ser complicada imposible en algunos casos los mtodos analticos estadsticamente slidos los datos externos ambos pueden incrementar el poder inferencial de ciertos datos recolectados de manera oportunista el manejo ecolgico en especial el que realizan las agencias gubernamentales requiere frecuentemente de datos apropiados para una inferencia confiable con protocolos estandarizados mtodos analticos modernos programas supervisados correctamente la ciencia ciudadana puede contribuir de forma valiosa la conservacin al incrementar el alcance de los esfuerzos de monitoreo para una especie la calidad de datos puede mejorarse si se adhiere los principios bsicos de la recoleccin anlisis de datos se disean los estudios para que proporcionen la calidad requerida de datos si se incluye una pericia estadstica adecuada fortaleciendo el aspecto cientfico de la ciencia ciudadana aumentando su aceptacin dentro de la comunidad cientfica con quienes toman las decisiones citizen science entails participation nonprofessional volunteers scientific investigation disciplines including archaeology astronomy biology long involved amateurs volunteers may considered citizen science broadest sense silvertown 2009 follett strezov 2015 biological projects range nationalscale projects us audubon societys centuryold christmas bird count dunn et al 2005 uk open air laboratory opal surveys trees insects biodiversity opal 2018 indepth research small teams sponsored earthwatch institute brightsmith et al 2008 earthwatch 2018 many ecological projects use data collected volunteers eg devictor et al 2010 tulloch et al 2013 pocock et al 2017 name citizen science concerns scientific investigation citizen engagement ecological science usually involves collection data comparison predictions generation new hypotheses estimation ecological attributes often projects specifically designed give amateurs role working professionals silvertown 2009 citizen science enthusiastically promoted increasing public participation science satisfying scientific objectives silvertown 2009 dickinson et al 2012 pushing envelope ecologists achieve dickinson et al 2012 bonney et al 2009 argue projects simultaneously fulfill goals recruitment research conservation education authors bonney et al 2009 tulloch et al 2013 hold citizen science enhances geographical scope data collection provides source skills free labor silvertown 2009 also benefits public increased scientific literacy bonney et al 2009 nevertheless full scientific potential citizen science remains unrealized tulloch et al 2013 theobald et al 2015 often viewed skeptically scientists bonney et al 2014 used extensively mainstream ecological research theobald et al 2015 management decision making conrad hilchey 2011 skepticism scientists decision makers due partly technical problems inadequate consideration sample size experimental design conrad hilchey 2011 key factor increasing scientific credibility well influence applicability citizen science data sets enhancement data quality ruizgutierrez et al 2016 sought highlight features citizen science influence reliability inferential power usefulness management conservation biological diversity view citizen science perspective scientific information focus biological populations monitoring estimation ecological attributes illustrative examples rather comprehensive review scientific aspects public involvement ecological research inadequately treated literature key scientific practice deserve richer discussion emphasis participation group ie 1 nonprofessional volunteers scientistdirected scientific investigation involves data collection framing citizen science way comprehensive many variants information collection analysis described citizen science said focused projects linkage scientific process public engagement inferential context considering public participation ecological research includes sampling designs protocols strength inferences obtained analytic methods research paradigms investigating distribution abundance organisms require specialized sampling techniques distinguish real change natural variation sampling variability sampling probabilistic ie random nonprobabilistic probabilitybased sampling every unit population interest known nonzero probability selected observations inherit randomness random survey design nonprobabilistic sampling population units selected without probabilitybased sampling framework one type nonprobabilistic sampling purposive deliberate sampling sampling design targets explanatory factors statistical model another type opportunistic sampling chance observations recorded without specific sampling design two common problematic issues nonrepresentative sampling imperfect detectability yoccoz et al 2001 nonrepresentative sampling occurs sampled units representative population inference made cochran 1977 thompson 2012 example organisms sampled along roadsides estimate density area larger roadsides imperfect detectability concerns lack detection organisms collecting field data failing account organisms actually present detected obvious consequences estimating population size density distribution nonrepresentative sampling imperfect detectability combine sampling observer variability severely weaken inferential linkage ecological attributes data collected classical framework statistical inference survey sampling designbased inference involving probabilitybased survey designs cochran 1977 thompson 2012 possibility unit population may sampled establishes inferential linkage units selected design allows statistical inferences made entire population including unselected units targets inference probabilitybased sampling include population parameters population totals means ratios estimated observations maashebner et al 2015 strength designbased inference statistical assessment rely assumed models data structures main limitations restriction inference population sampling units nonzero sampling probabilities limited ability address analytical causal hypotheses sterba 2009 alternative framework modelbased inference uses statistical models mimic randomness absence probabilitybased sampling gregoire 1998 lenhard 2006 models typically include parameters hypothesized factors eg variable observer effort nondetectability habitat variation thought influence observations assumed applicability models potential observations allows inference extended beyond sample population important limitation data collected may fully represent population interest survey design probability based maashebner et al 2015 modelbased inference potentially applied data collected purposive opportunistic sampling purposive sampling linked directly model data collected deliberately strengthen inferences population parameters opportunistic sampling linkage missing data may may prove useful reliably estimating parameters either case models include covariates hypothesized influence observations analysis involves estimating covariate parameters sample data approach may produce reliable inferences relevant environmental observer factors accounted model necessary covariate information collected sterba 2009 however opportunistically collected data joint requirements model specification data collection rarely met information metadata external data sources followup sampling needed improve statistical reliability much knowledge critical conservation absolute abundance density obtained probabilitybased sampling design standardized protocols inference population interest addition important information rate parameters reproduction mortality rates obtained modelbased inference data collected purposively approaches contrast opportunistic data collection without sampling design unlikely result reliable inferences target population without information beyond survey john tukey 1977 1980 pioneer statistics distinguished exploratory confirmatory research paradigms affects hypothesis formation study design data collection analytical methods confirmatory paradigm hypotheses generated priori data collection whereas exploratory paradigm hypotheses generated posteriori collection confirmatory analysis provide reliable scientific information investigation welldefined priori hypotheses statistical analysis data collected according designs target hypotheses alternatively exploratory analysis data useful formulating hypotheses posteriori 2step process exploration data often collected opportunistically patterns lead hypotheses design new investigations test whether patterns genuinely express underlying ecological structure statistical reliability results cannot stand exploration alone without form followup study ecology involves exploratory confirmatory analysis depending circumstances nonetheless important confuse one distinction particularly relevant citizen science often involves opportunistic collection large amounts data exploratory analysis new web 20 technologies goodchild 2007 several factors influence quality usefulness data estimation inference particularly population research monitoring typology overall design projects ranges rigorous protocols designed clearly articulated questions opportunistic data collection without protocols grouping builds pyramid framework us geological surveys amphibian research monitoring initiative corn et al 2005 intensive research handpicked sites top standardized monitoring rigorous sampling design broad area middle coarse measurements checklist approach national scale bottom used similar typology recognizing considerable variation within among groups pocock et al 2017 intensive scientific research projects volunteers work small teams close supervision lead scientists volunteers may perform complex tasks collect data according explicit protocols often allow confirmatory hypothesis testing statistical inference examples include projects sponsored earthwatch institute earthwatch 2018 nongovernmental organization connects ecotourists conservationoriented research eg peruvian rainforest birds brightsmith et al 2008 longitudinal monitoring studies consist standardized repeat monitoring often many participants high degree institutional coordination data species occurrence relative abundance may collected volunteers sampling protocols involve specified time intervals locations tulloch et al 2013 allowing confirmatory hypothesis testing statistical inference level biological expertise frequently prerequisite category includes surveys british newson et al 2005 swiss kry schmidt 2004 north american sauer et al 2013 breeding bird surveys atlastype monitoring studies involve substantial number participants rudimentary sampling scheme entailing data collection defined time period grid broadly defined spatial cells within volunteers choose locations tulloch et al 2013 data may consist lists species occurrence checklists sometimes designed yield relative abundance within grid cells atlas single survey species spatial distribution include repeat sampling repeatvisit data statistical adjustments may allow modelbased inferences unbiased estimates quantitative population trends eg link et al 2006 one example repeatvisit atlas us audubon societys annual christmas bird count dunn et al 2005 among many atlases tulloch et al 2013 finally internetbased projects accept opportunistic data openaccess web portals may involve sampling scheme datacollection protocols little training mass participation typically presenceonly observational data location occurrence species collected opportunistically recorded via web portals often covering broad areas potential bias frequently induced volunteer skill differences spatial variation factors isaac et al 2014 analysis often requires additional statistical modeling assumptions constrain use data rarely produces reliable estimates without followup investigation category includes swedish species gateway birdmonitoring project snll et al 2011 country branch worldbirds roberts et al 2005 projects led cornell lab ornithology bonney et al 2009 dickinson et al 2010 hochachka et al 2012 australiabased questagame project questagame 2018 objectives guide project design data collection influence data quality objectives applied ecological research pursued projects strong designs expertise training volunteers professional oversight projects may produce longterm data usable reliable estimation involving multiple research questions example north american breeding bird survey longitudinal survey targets various information needs species conservation concern neotropical migrants grassland birds sauer et al 2013 however many largescale internetbased projects opportunistic data collection better suited general objectives related public education recreation data exploration eg bonney et al 2009 tulloch et al 2013 purposes unnecessary meet temporal replication design requirements needed distinguish real ecological patterns error natural variation largescale projects opportunistic data collection often involve surveillance monitoring numerous species broad area specific priori research questions dickinson et al 2010 unlike targeted monitoring designed investigate particular questions models nichols williams 2006 rather hypothesis testing statistical estimation data exploration often used search potential patterns species occurrence habitat associations hochachka et al 2012 investigations design govern questions addressed data collected analytical methods williams et al 2002 bird et al 2014 survey monitoring design general factors including number taxa number sites sampling variability sampling frequency duration monitoring major determinants analysis techniques used reliable inference many respects gold standard survey design entails data presence absence abundance stratified random sampling across geographic area standardized protocols control sources bias metadata allow quantification error yoccoz et al 2001 magurran et al 2010 isaac et al 2014 designed approaches inference used hypothesisdriven projects longitudinal surveys accomodate foregoing standards particularly basic applied research requiring highquality data eg williams et al 2002 good design account 2 main sources variation monitoring data yoccoz et al 2001 spatial sampling allows reliable representation area inference made temporal sampling allows quantitative estimation detection probability yoccoz et al 2001 buckland et al 2005 magurran et al 2010 databases projects often usable address wide range questions many atlastype projects projects openaccess internet contribution opportunistically collected data reliable estimation confirmatory testing priori hypotheses may difficult impossible projects may produce sample data needed address high rates variation visit frequency observer effort species misidentification isaac et al 2014 common sources variation result nonrepresentative sampling geographic areas additional statistical assumptions data constraints limit range research questions addressed data sets nichols et al 2012 opportunistically collected data statistically robust analytical methods external data may increase inferential power modelbased approach statistical adjustments sometimes reduce variability bias compensate data features violate statistical assumptions example occupancy modeling mackenzie et al 2017 provide unbiased information temporal trends opportunistic data sets meet certain conditions temporal replication consistent datacollection methods covariate information kry et al 2010 van strien et al 2013 isaac et al 2014 examples kindberg et al 2009 szabo et al 2010 used visit duration list length respectively proxies survey effort monitoring large mammals birds maes et al 2015 discussed methods estimating species distribution geographic range opportunistic data sets international union conservation nature iucn red list purposes bird et al 2014 reviewed modeling techniques although exploratory analysis patterns data collected opportunistically always possible still necessary minimum deal survey effort detection issues otherwise patterns discovered may confound sampling error ecological processes factor influencing data quality volunteer training performance tracking volunteers amateur otherwise may already subjectmatter experts sufficient training many perform tasks well professionals however complex task greater investment needed training professional supervision fostersmith evans 2003 newman et al 2003 handson training oversight offered intensive research teams necessary teaching complicated tasks eg trapping point counts woodland mammal monitoring newman et al 2003 global positioning systems forest inventory plots invasive plant monitoring crall et al 2011 taxonomy hymenoptera pollinator identification kremen et al 2011 limited training needed broadscale internetbased projects involving simple tasks identifying small number species counting eggs may provided online instructional materials quizzes eg bonney et al 2009 sometimes given feedback conjunction data entry screening data obvious errors opportunistic monitoring projects may provide training kind eg roberts et al 2005 observer bias citizen science projects investigated experimentally comparing data collection volunteers professionals data reliability varies depending volunteer group species ecosystem task steger et al 2017 many studies volunteers comparable professionals straightforward identification species higherlevel taxonomic groups insect pollinators kremen et al 2011 amphibians genet sargent 2003 woodland mammals newman et al 2003 see roy et al 2016 although making difficult taxonomic identifications coccinellid lady beetles gardiner et al 2012 calling amphibians genet sargent 2003 insect pollinators kremen et al 2011 detecting organisms volunteers comparable cases mammals territories newman et al 2003 expertise comes lengthy experience cryptic marine invertebrates fostersmith evans 2003 nocturnal mammals sunde jessen 2013 african ungulates herds steger et al 2017 course professionals frequently fail detect organisms solution survey design allowing quantitative estimates detection error williams et al 2002 generally addressing observer bias requires monitoring performance accounting data variability metadata crall et al 2011 milnergulland shea 2017 project design rather citizen involvement per se fundamental constraint limiting use citizen science data ecological research issue data collected professionals volunteers intensive research longitudinal monitoring studies designed around priori hypotheses defined questions often involve expense labor investment training supervising volunteers allow stronger inference ensure data collected wellsuited intended use nichols et al 2012 emphasize testing ecological hypotheses confirmatory paradigm entirely possible largescale internetbased citizen science projects specifically designed purpose although projects rare examples include studies silvertown et al 2011 evolution snail color morphs pocock evans 2014 parasitism invasive leafminer projects involving simple protocols suitable mass participation contrast observational approaches lacking necessary design features strong inference may less expensive require data adjustments additional analytical assumptions limit inferences nichols et al 2012 internetbased projects involving mass contribution opportunistic data participants skill levels eg bonney et al 2009 often produce data lack strong inferential power require exploratory searching patterns order generate hypotheses followup investigation mentioned combined requirements model specification data coverage modelbased approach limit use opportunistic data sets reliable inferences although external covariate information may compensate kry et al 2010 van strien et al 2013 isaac et al 2014 similarly conditions may allow atlastype studies produce reliable inferences eg dunn et al 2005 link et al 2006 tulloch et al 2013 many citizen science participants aspire actionable science decision makers use management planning stewardship conrad hilchey 2011 ganzevoort et al 2017 project planners goals must take account importance data quality decision makers especially government much ecological research carried agencies eg us department interior 2018 applied research meet ecological management policy objectives including conservation frequently requires highquality data suitable reliable estimation inference managementrelated ecological hypotheses tested effects management actions distinguished sampling error natural variation two conservationrelated examples management context species recovery us endangered species act invasive species control costly regulatory responsibilities may exacting needs data quality johnson et al 2015 land managers need assurance quality embark expensive programs newman et al 2010 meet needs citizen science project methods must chosen carefully regarding level accuracy access statistical expertise specific research questions crall et al 2011 welldesigned intensive research studies longitudinal monitoring studies usually best suited volunteer training regular performance monitoring essential establishing record reliability crall et al 2011 accounting sources data variability pocock evans 2014 steger et al 2017 biodiversity conservation 1 way citizen science potentially make valuable contributions increasing number observers geographic scope species monitoring efforts standardized protocols stateoftheart analytical methods wellsupervised program data nonprofessionals provide reliable results magurran et al 2010 schmeller et al 2009 tulloch et al 2013 citizenscience investigations could used conservation biogeography examine largescale patterns processes example impacts climate change biodiversity devictor et al 2010 buckland et al 2005 suggest designing longitudinal monitoring schemes common across global regions surveys entered various levels modified lower sampling rates simpler methods nations fewer resources could participate standardized citizen science monitoring schemes could also used population trend estimates necessary iucn red list assessments rare species maes et al 2015 track spatial dynamics invasive species eraud et al 2007 volunteers participation foster new skills enthusiasm science devictor et al 2010 example earthwatch project surveying british woodland mammals 30 volunteers joined conservation groups 5 actually changed careers biology newman et al 2003 instances historical opportunistic data sets yield reliable analysis temporal trends long robust methods used modelbased approaches increase inferential power opportunistic data van strien et al 2013 isaac et al 2014 important source information changes species relative abundance distribution example historical opportunistic data sets used van strien et al 2018 examine whether woodland fungi benefited policies reduced atmospheric nitrogen sparks et al 2005 exam