Elizabeth Spier|Pia Rebello Britto|Terri Pigott|Eugene C. Roehlkepartain|Michael McCarthy|Yael Kidron|Jan Glover|Daniel A. Wagner|Julia Lane|Mengli Song
PROTOCOL: Parental, Familial and Community Support Interventions to Improve Children's Literacy in Developing Countries: A Systematic Review
American Institutes for Research|United Nations Children's Fund|Loyola University Chicago|Search Institute|Yale University|American Institutes for Research|Yale University|Institute of International Education|American Institutes for Research|American Institutes for Research
majority worlds children despite substantial increases access primary school academic learning neither occurring expected rates supplying basic foundational skills necessary succeed 21st century 2010 approximately 61 million primary schoolage children worldwide attending school among attending school academic learning far assured example 46 percent children nicaragua achieve grade 4 learning standards figure drops less 5 percent malawi ghana 2008 four five young women completed grade 6 still illiterate partially literate unesco 2012 significant lag academic achievement tells us schools alone fully meet childrens needs literacy development many reasons exist challenges providing adequate literacy instruction within school context example world bank study found average 19 percent teacher absence rate across bangladesh ecuador india indonesia peru uganda many teachers physically present spending time teaching classroom chaudhury hammer kremer muralidharan rogers 2006 even children teachers classroom student learning significantly hampered unfamiliarity language instruction altinok 2013 large class sizes insufficient number teachers teacher assignment practices disproportionally allocate lowestperforming teachers communities highest needs unesco 2014 despite efforts innovations many countries problems solved quickly alternative approaches needed support childrens literacy development proximal contexts ie childs home community direct influence literacy development numerous initiatives underway globally try improve childrens literacy development including interventions work parents families communities initiatives intended supplement childrens schoolbased learning provide alternatives children access preprimary primary education examples interventions include tutoring peerassisted learning mobile libraries programs build parental knowledge support childrens literacy literacy instruction outside regular schools eg context religious education provision educational media use outside regular classroom instruction numerous interventions low middleincome countries lmics little information regarding interventions evidence effectiveness evidence reveals therefore review focused evidence works improve childrens literacy development lmics interventions focused children 3 12 years old work parents families communities draw two dimensions learning 1 contexts support literacy learning 2 learning outcomes areas preliteracy literacy ecological models demonstrated proximal contextsparticularly school home community among strongest influences learning christenson reschly 2010 dickinson neuman 2006 within contexts influences categorized human eg families nonhuman eg print studies human influences parent child shared book reading peertopeer learning community volunteers britto brooksgunn griffin 2006 britto oketch weisner 2012 nonhuman influences include access print environmental print learning resources mobile libraries interpersonal support parents tutors community members access print learning support digital means educational radio television technologies doiron 2011 influences operate naturally occurring interactions daily life whereas others operate programs services human nonhuman influences intersect support childrens learning example reading interventions may rely community parental engagement support implementation lancy bock gaskins 2010 often support international organizations national nongovernmental organizations ngos community basedorganizations hoppers 2006 literacy skills acquired progressively study use developmental lens emergent preliteracy skills reading writing therefore review includes interventions intended improve childrens literacy development point preprimary period middle childhood ie 3 12 years old conceptual framework study draws contextual pathways linked literacy developmental perspective five features characterize model see figure 1 family community contexts one pathways shown figure 1 may weak nonexistent reducing likelihood child reach full potential regard literacy development interventions considered current review expected act one weak missing pathways leading improvements childrens literacy development intervention review act one pathways include one preliteracy literacy outcomes children nonschool contextual pathways literacy learning policymakers practitioners individual country level multinational organizations increasingly want select interventions documented reliable evidence effectiveness discussed previously poor literacy development persistent significant concern many countries however available published research literature outside industrialized nations quite limited albeit growing dearth highquality quantitative studies also lack balance quality quantity evidence intervention effectiveness across different contexts eg literature expected available middleincome countries lowincome countries despite high level investment interventions lowincome countries field benefit knowing types interventions subject rigorous evaluation well evidence produced evaluations already high level interest results review among large international development organizations us agency international development usaid unicef well smaller ngos academics across developing countries review also expected provide comprehensive information models approaches evidence supporting refuting effectiveness models used absence evidence objective work provide evidence needed ngos governments others make informed decisions interventions improve childrens literacy outcomes lmics also include information help end users understand extent evidence grounded studies carried similar contexts eg level parental literacy family reading initiatives successful advisory panel indicated contextual information vital effective decision making expect find significant gaps availability evidence works improve childrens literacy development lmics outside formal education system highlighting availability evidence expect review provide information help funders learn invest evaluation resources work products full literature review shorter briefs made available partner organizations website relevant information clearinghouses usaids development experience clearinghouse clearinghouse international developments child youth family policies columbia university build search institutes extensive network international countryspecific ngos save children world vision international ymca international international youth foundation contact numerous professional networks focused global child youth development including unicef alliance international youth development communication initiative network international society child indicators uns committee rights child objective systematic review examine availability evidence evidence says effectiveness interventions improve parental familial community support childrens literacy development developing countries review provide information contextual influences parental family community support childrens literacy development skills using interventions target influences explore following questions overarching goal review increase availability information evidencebased decision making international agencies ngos government policymakers whether intervention leads improved child literacy outcomes affected larger context even intervention effective strengthening one specific aspect model human material resources required interventions implemented fidelity example intervention shows parents engage shared book reading little impact childrens literacy development printed materials available likewise intervention engages parents shared book reading children improves childrens literacy may work well introduced place parents time literacy skills engage reading children plus print materials available addition home community contexts replicability scalability intervention depend availability resources required introduce implement intervention example successful mobile library initiative may require acquisition maintenance one vehicles staff basic training managing library therefore extent information available frame results terms contextual variables may affect replicability scalability intervention approach well resources necessary introduce deliver intervention information essential stakeholders considering interventions may right given local context resources available section addresses criteria study inclusionexclusion search strategy methods used primary research criteria determination independent findings study coding categories statistical procedures treatment qualitative research section address eligible designs types participants interventions outcome measures define criteria including excluding studies review population intervention comparison outcomes model petticrew roberts 2006 used categorize inclusion criteria literature population intervention comparison outcome criterion must met inclusion fifth supplemental categorycontextalso may considered population target population children 3 12 years old living lmics defined world banks country income classification countries many childrens births registered exact ages may unknown therefore absence information regarding exact ages accept studies children described preprimary primary school age studies focus children disabilities included even though disabilities distinct topic interest intervention studies reviewed must primary studies interventions literature reviews metaanalyses intervention address literacy preliteracy skills delivered family community members eg volunteers interventions delivered within school setting acceptable delivery mechanism parent community members addition different types interventions eg cash transfers vouchers libraries included purpose address literacy outcomes delivered within formal schooling intervention could program product policy practice however primary focus study aligned topic area literacy comparison eligible studies must include treatment comparison group include studies use randomized control trials rcts regression discontinuity designs also include quasiexperimental studies provided baseline serious pretest differences outcomes interest mean baseline differences must 25 exclude singlegroup prepost designs weak internal validity quasiexperimental studies included extent methods exist computing appropriate measure effect size studies use complex research designs propensity score matching regression discontinuity designs report complex statistical analyses hierarchical linear models may excluded agreedon effect sizes methodological literature synthesized standard effect sizes thus quasiexperimental studies evidence pretest differences excluded primary criterion article selection must contain evaluation valid comparison group valid comparison group must 1 drawn population intervention group 2 baseline data available demonstrate preexisting differences outcomes interest descriptive studies included outcomes eligible preliteracy literacy outcomes include full range skills including phonemic awareness listening vocabulary speaking pronunciation print concepts knowledge alphabet reading comprehension fluency sight reading words writing spelling narrative development preliteracy literacy outcomes must assessed standardized measures countryspecific locally used assessments assessments developed evaluation extent overaligned contents intervention include secondary measures school dropout grade progression long also direct measure childrens literacy development relevant literature included published 2003 later studies 10 years ago high likelihood assessing interventions longer use longer implemented context eg childrens access primary education parental literacy use firstlanguage instruction increased many lmics last decade unpublished studies eligible interventions dissertations research reports government agencies ngos included documents powerpoint presentations internal agency memos editorials notes student term papers advertisements promotional materials editorials letters case series personal communication notes eligible review sufficient information included screen literature abstracts published english spanish french study passes title abstract screening main text another language identify qualified reviewers literate language review study studies included systematic review obtained electronic academic literature grey literature ie unpublished reports key informant solicitation search online databases across diverse disciplines see table 1 work advisors networks identify relevant literature examine reference sections relevant work agency websites searched grey literature include united nations international development banks aid groups ngos foundations international research institutes centers expertise project advisory panel also asked use professional networks assist search grey literature list search terms used search electronic databases agency websites informant solicitation see table 2 search terms necessarily must adapted database although concepts phases human development early middle childhood home communitybased learning remain constant also use core set search terms common databases read lit obtain grey literature publications reports documents archives agency websites searched using electronic database search terms vocabulary agency vary based disciplinary affiliation agencyspecific language therefore although terms academic search relevant new vocabulary may need inserted agency search searching publications agency websites may may include following search fields title keyword isbn subjects series departments publishing agencies languages yearrelease date publication lists regioncountry forth agencies reports accessed online search agency librarian contact sent request assist searching agency publications initial screening use titles abstracts determine whether study meets defined inclusion criteria obtain fulltexts articles pass initial screening given multiple sources searched possibility obtaining duplicate articles high duplicates removed articles identified review double coded address possible selection issues completed coding forms reconciled cases disagreement two reviewers third reviewer consulted render decision copies code books available appendix literature screened selected based prisma preferred reporting items systematic reviews metaanalyses guidelines illustrated figure 2 moher liberati tetzlaff altman 2009 prisma guidelines mendeley software used manage title abstract library store fulltext documents coding completed using microsoft excel least rigorous research lmics addresses effects interventions intended improve childrens preliteracy literacy development parents families communities research conducted area falls two general types first studies communitylevel interventions primarily carried auspices large organizations unicef studies cover lmics second smallerscale studies interventions delivered individual children families literature search expected yield information populations interventions outcomes studied lmics following information two studies eligible inclusion review one study reported unicef initiative facilitated transition young children primary school unicef 2010 evaluation consisted multiple countrylevel rcts interest common evaluation framework tools across diverse countries fact schoolage children community members served informal teachers groups preschool children deliver intervention aim intervention increase school readiness preschool children including area preliteracy although several countries included study countries implemented trial sufficient controls thus countries evidence use randomization information group equivalence pretest included another study banerjee cole duflo linden 2007 measured impacts two remedial education interventions compared costeffectiveness short medium term across initial abilities students intervention interest review hired young women community teach basic literacy skills children lagging behind government schools included intervention outside formal school setting based community although study examined relevant intervention analysis strategy included use instrumental variables adjust differences among treatment control groups study may included authors provide unadjusted treatment control group means calculate effect size several reports study examine treat study unit analysis anticipate studies report number outcomes conduct metaanalysis outcome separately study contributing one effect size analysis may consider using robust standard errors hedges tipton johnson 2010 synthesizing results studies include outcomes anticipate obtaining enough studies able use technique code studies based characteristics participants interventions study design participant characteristics include country age gender intended beneficiaries native language schooling experience school level intervention characteristics include focus intervention eg language skills print awareness characteristics intervention providers duration intensity intervention materials used setting research design coded along potential sources bias attrition randomization problems preexisting differences experimental control groups employ research quality scale instead separately analyze rcts quasiexperimental studies major statistical analyses used study coded measures effect size relevant outcomes reported codes used possible examination moderators effect size heterogeneity coding form provided appendix review include study designs ranging rcts various types quasiexperiments excluding onegroup prepost studies studies use either standardized mean difference logodds ratio effect size comparing outcomes two independent groups analyze different types effect sizes separately transform effect sizes metric studies may employ complex quasiexperimental designs regression discontinuity propensity score matching combine effect sizes different study designs necessarily comparable example combine standardized mean effect size computed study using covariateadjusted means standardized mean effect size computed unadjusted means effect sizes clustered randomized trials adjusted using approaches suggested hedges 2007 pigott 2011 possible combine study results using metaanalysis conduct analyses outcome intervention type using separate analyses outcomes direct measures literacy versus successful grade progression example anticipate several studies using peer teaching model increase school readiness skills included review begin analysis examining similar outcomes similar interventions examine heterogeneity effect sizes outcome within intervention types across studies using i2 index test variance component random effects model provide 95 percent confidence interval effect use sensitivity analyses examine impact effect size outliers reporting results outlier included excluded anticipate wide range interventions implemented number lmic contexts use random effects model accurately reflects heterogeneity expect sample studies use moderator analysis explore heterogeneity sufficient numbers studies indicated power guidelines hedges pigott 2004 able conduct moderator analyses use conceptual model guide examining whether effect size heterogeneity related example characteristics participants age children income level community child disability status able conduct moderator analyses use common measure random variance component anticipate finding small numbers studies metaanalysis procedures weighted inverse variance effect size use stata r software conduct metaanalysis use recent techniques computing effect sizes synthesizing results publication bias examined using funnel plots eggers test also produce forest plots relevant analyses effect sizes appropriate attempt contact authors obtain missing data within studies successful obtaining important information least two contact attempts may need drop study particular analysis explicitly searching literature purely qualitative assessments interventions however eligible studies include qualitative evidence examine studies insight interventions quantitative outcomes worked given setting lead review author lead author person develops coordinates review team discusses assigns roles individual members review team liaises editorial base takes responsibility ongoing updates review combined team brings substantial expertise areas required complete review high level quality work led air collaboration search institute yale university well expert consultants team led principal investigator elizabeth spier extensive experience conducting international education evaluations team members include pia britto expert child development education eugene roehlkepartain expert parent family community engagement improve child outcomes terri pigott yael kidron experts systematic review methods janis glover yale university senior reference librarian extensive expertise information retrieval michael mccarthy assist daniel wagner expert literacy multiple international contexts serve senior advisor advisory board composed academic researchers developing countries provide input literature context help dissemination networks quality assurance provided julia lane brings set skills including statistics education context airs professional editors mengli song contributed substantial methodological statistical expertise works clearinghouse summary review conducted auspices 3ie international initiative impact evaluation funded grant provided usaid air also providing financial support facilitate completion work staff involved project published research papers area review arranged multiple reviewers several organizations take part work ensuring areas review carried individuals organizations conflict interest example relevant literature published yale university would reviewed personnel organizations systematic review completed june 30 2014 june 30 2014 august 31 2014 develop work products help disseminate results review journal article special topic papers powerpoint presentation findings summaries results referenced publications given rapid increase use rigorous evaluation field would highly beneficial update review within next years authors review would pleased complete update funding become available completing form accept responsibility preparing maintaining updating review accordance campbell collaboration policy campbell collaboration provide much support possible assist preparation review draft review must submitted relevant coordinating group within two years protocol publication drafts submitted agreed deadlines unable contact extended period relevant coordinating group right deregister title transfer title alternative authors coordinating group also right deregister transfer title meet standards coordinating group andor campbell collaboration accept responsibility maintaining review light new evidence comments criticisms developments updating review least every five years requested transferring responsibility maintaining review others agreed coordinating group support coordinating group preparing review conditional upon agreement publish protocol finished review subsequent updates campbell library campbell collaboration places restrictions publication findings campbell systematic review abbreviated form journal article either publication monograph version campbell systematic reviews journals however restrictions preclude publication findings reported elsewhere authors considering publication journal aware possible conflict publication monograph version campbell systematic reviews publication journal publication press status campbell systematic reviews acknowledge campbell version include citation note systematic reviews published campbell systematic reviews coregistered cochrane collaboration may additional requirements restrictions copublication review authors accept responsibility meeting copublication requirements understand commitment required undertake campbell review agree publish campbell library signed behalf authors form completed elizabeth spier phd date 21 july 2014