Trine Filges|Carole Torgerson|Louise Gascoine|Jens Dietrichson|Camilla Palmhøj Nielsen|Bjørn A. Viinholt
Effectiveness of continuing professional development training of welfare professionals on outcomes for children and young people: A systematic review
Danish Center for Social Science Research|Durham University|Durham University|Danish Center for Social Science Research|Danish Center for Social Science Research|Danish Center for Social Science Research
cpd aims improve outcomes children young people educational welfare professionals work clear evidence cpd education improves student academic outcomes cpd delivered variety settings different kinds trainers educators differing lengths time differing intensity many methods delivery coaching sessions feedback based observations videotapes classroom practice feedback reflection workshops review looked effects cpd approaches education welfare practitioners preschool teachers pedagogues school teachers social workers psychologists police officers educational social crime justice outcomes children young people andas secondary outcomesany effects professional practice practitioners fields purposes review cpd must involve development core professional skills aim review campbell systematic review sr examines effects cpd approaches education welfare practitioners educational social outcomes children young people outcomes practitioners review summarises evidence 51 moderatequality studies including 48 randomised controlled trials rcts three quasiexperiments review includes studies evaluate effects cpd childrens young peoples professionals outcomes fiftyone studies identified related education eligible studies identified social welfare crime justice 51 education studies grouped three subtopic areas 12 studies reporting 10 trials considered cpd social emotional development interventions daycare kindergarten preschool school settings 38 studies reporting 33 trials dealt cpd language literacy development interventions one study looked cpd stress reduction 48 studies used experimental designs random assignment 26 51 studies included metaanalyses reduction caused studies reporting trial five studies insufficient reporting outcomes calculate effect size four studies studies rated high risk bias total 16 studies assessed sufficient methodological quality included metaanalyses studies spanned period 19992018 thirtythree trials undertaken united states two uk one following countries denmark ireland netherlands new zealand portugal australia chile germany social emotional development interventions nine studies small body evidence social emotional development interventions daycare kindergarten preschool school settings finds effect cpd student academic outcomes four studies results two individual studies could combined single metaanalysis student outcomes ie nonacademic teacher outcomes precluding conclusions concerning effectiveness ineffectiveness type cpd outcomes language literacy development interventions 17 studies moderate body evidence language literacy development interventions finds effect cpd student academic outcomes 13 studies results three individual studies could combined single metaanalysis teacher outcomes thus precluding conclusions concerning effectiveness ineffectiveness type cpd teacher outcomes stress reduction one study possible draw conclusions one study placed subtopic stress reduction insufficient evidence conclusions drawn exception language literacy development interventions type cpd seems effect student academic outcomes dominance united states main country types cpd interventions covered review evaluated clearly limits generalisability findings moreover limited number studies means possible conduct analysis specific cpdapproaches across cultures professionsservicedeliverer types organisations servicereceiver types agencies consider conducting large rct series large rcts evaluating effectiveness cpd intervention countries outside united states review authors searched studies december 2018 quality cpd education welfare professionals working children young people key importance policy makers practitioners fields order inform education welfare professions nature effectiveness diversity approaches cpd sr international literature undertaken western societies increasing acknowledgement value working evidenceinformed approaches methods therefore results sr utmost relevance review aimed systematically search locate quality appraise synthesise available effectiveness studies evaluated relevant interventions using rigorous designs effects cpd approaches education welfare practitioners preschool teachers pedagogues school teachers social workers psychologists police officers educational social crime justice outcomes children young people outcomes practitioners empirical evidence external validity specific cpdapproaches across cultures across professionsservicedeliverer types across organisations across servicereceiver types search concluded december 2018 relevant studies identified electronic searches bibliographic databases specific targeted relevant online repositories internet search engines searched identify published unpublished literature reference lists included studies reviews also searched design review full sr studies adequately address primary research question effectiveness question highquality evaluations cpd interventions improve educational social outcomes children young people professional practice outcomes practitioners using experimental designs rcts quasirandomised trials studies quasiexperimental designs qeds studies utilised approaches included review due absence adequate control group conditions studies included included least one valid reliable outcome standardised validated test standardised different population electronic searches identified 5146 potentially relevant studies screening titles abstracts full papers using inclusionexclusion criteria three stages independent double screening 51 studies included review area education studies could grouped three subtopic areas according focus professional development pd investigated although 50 two subtopic areas 38 studies dealt pd language literacy development interventions outcomes 12 studies investigated social emotional development interventions outcomes one study looked pd interventions related stress reduction social emotional development subtopic area two trials reported two papers thus number trials 10 language literacy number trials 33 two trials reported two papers one trial reported four papers thirtyfour trials conducted united states one study undertaken following countries australia chile denmark germany ireland netherlands new zealand portugal two trials undertaken uk professional participants evaluations pd interventions exclusively preschool teachers pedagogues teachers participants exclusively children young people attending preschool including day care kindergarten nursery school settings included studies met minimum threshold quality due inclusion criterion review metaanalyses focused social emotional development subtopic area language literacy subtopic area except three language literacy development area studies review rcts overall included studies varied risk bias judgements single study could characterised robust rct low risk bias assessed risk bias items total 17 studies one evaluating stress reduction remaining language literacy area given score 5 least one risk bias items corresponding risk bias high findings considered metaanalysis random effects models used pool data across studies used standardised mean difference smd hedges g used estimating smd applied small n correction pooled estimates weighted inverse variance methods 95 confidence intervals cis used funnel plots used assess possibility publication bias sensitivity analysis used evaluate whether pooled effect sizes robust cluster correction across study design components methodological quality used homogeneity professional student outcomes two subtopic areas basis metaanalyses control conditions similar tended comprise business usual pd except three studies language literacy development area one social emotional development area reported either student teacher outcomes enabled calculation smd standard error approximately end intervention twentysix studies left metaanalysis nine social emotional development area 17 language literacy area sample sizes reported studies used metaanalyses social emotional development topic area varied 99 students 1685 students average 914 students 22 classes 224 classes average 95 nine schools 58 schools average 26 schools four studies could combined metaanalysis student academic outcomes seems effect student academic outcomes weighted average smd 005 95 ci 007 016 statistically significant evidence heterogeneity studies results two individual studies could combined single metaanalysis student outcomes teacher outcomes weighted average smd student social competences 013 95 ci 003 024 022 95 ci 008 037 students socioemotional skills three studies reported outcomes various student measures different combined teacher outcomes reported three subscales class positive climate negative climate behavioural management weighted average smd positive climate 061 95 ci 008 114 negative climate 018 95 ci 073 108 behaviour management 030 95 ci 014 073 sample sizes reported studies used metaanalyses language literacy area varied 164 students 4078 students average 1632 students 24 classes 324 classes average 113 four schools 224 schools average 58 schools thirteen studies reported results student academic outcomes language literacy development topic area seems effect student academic outcomes weighted average smd 004 95 ci 001 010 result somewhat sensitive due removal studies scores 4 blinding component weighted average effect became larger statistically significant studies blinding scores 4 removed note however four studies contributed average evidence heterogeneity student outcomes reported results three individual studies could combined single metaanalysis teacher outcomes seem positive effect teacher outcomes measured early language literacy classroom observation ellco weighted average smd 045 95 ci 016 074 small amount heterogeneity studies also seems positive effect teacher outcomes measured three summary class measures emotional support instructional support classroom organisation weighted average smd emotional support 030 95 ci 011 049 classroom organisation 023 95 ci 004 043 instructional support 020 95 ci 001 039 evidence heterogeneity studies weighted average instructional support lost statistical significance sensitivity analysis cluster correction otherwise none results changed one study reported results two ellco subscales one study reported results mathematics teaching practices find adverse effects moderate body experimental evidence exists relation effect pd topic area education similar evidence appear exist topic areas social welfare crime justice small body evidence exists relation effect pd social emotional development interventions students teachers majority studies report student outcomes teacher outcomes reported exceptions different combined moderate number experimental evaluations pd language literacy undertaken mainly united states number studies used metaanalysis reduced 38 17 reduction caused studies reporting trial two studies insufficient reporting outcomes calculate effect size three studies studies rated high risk bias total 16 studies judged high risk bias 5 scale accordance protocol excluded metaanalysis basis would likely mislead inform short result review currently insufficient evidence conclusions drawn except students language literacy subtopic area seem effect student academic outcomes weighted average effect small statistically significant otherwise small number available studies reporting similar outcomes precludes conclusions concerning effectiveness ineffectiveness pd moreover limited number studies prevented analysis specific pdapproaches across cultures across professionsservicedeliverer types across organisations across servicereceiver types vast majority studies undertaken united states dominance united states main country pd interventions meeting criteria evaluated using rigorous methods within specific parameters clearly limits generalisability findings none studies however considered overall high quality risk bias assessment process excluding studies high risk bias metaanalysis applied review left us 17 total 33 possible studies synthesise language literacy area finding right entailing important information stakeholders degree confidence place expected gains pd language literacy area given limited number rigorous studies available countries united states would natural consider conducting large rct series large rcts evaluating effectiveness pd intervention topic area social emotional development languageliteracy development countries outside united states trials designed conducted reported according methodological criteria rigour respect internal external validity order achieve robust results quality professional development education welfare professionals working children young people eg preschool teachers pedagogues school teachers social workers psychologists police officers etc key importance policy makers practitioners fields general wellbeing countrys citizens provision better opportunities terms educational social welfare outcomes eg participation higher education reduction antisocial behaviour linked quality pd available welfare professionals conversely potential barrier achieving education welfare aspirations variable quality professional training delivered educational andor welfare practitioners due challenges designing implementing high quality pd could mean education training groups professionals may sometimes less optimal order inform education welfare professionspolicy makers practitionersabout nature effectiveness diversity approaches cpd sr international high quality causal literature undertaken following conceptualisation proposed buysse hollingsworth 2009 one think professional development programmes terms providers learners content organisation facilitation learning experiences relation present review learners consider recipients cpd professionals already completed initial training professionals thus fully qualified employment cpd thought specific type pd purposes review focus cpd use terms cpd pd interchangeably clear following found studies fulfilled inclusion criteria field education hence examples provided literature contextualisation section focuses topic area improve professionals ability provide children emotional support increase professionals awareness importance meeting students high expectations create positive teacherchild interactions individual level use positive behaviourmanagement strategies classroom level pd contect focuses developing teachers knowledge understanding substantive fields language literacy development numeracy skills development forth also relevant review beyond relevant ask cpd delivered many different ways buysse winton rous 2009 darlinghammond hyler gardner 2017 egert et al 2018 pianta et al 2006 argued effective programmes tend focus specific content instance new curriculum content based quality rating scale could take form example lesson plans unit plans sample student work observations peer teachers video written cases teaching thereby providing teachers clear vision best desired practices highlight benefits collaboration feedback fellow teachers since mode provision facilitate reflection help learning collborating colleagues moreover provide opportunities changing teacher practices organisational level buysse hollingsworth 2009 darlinghammond et al 2017 cpd provided different kinds trainers educators implemented variety settings differing lengths time differing intensity according buysse hollingsworth 2009 darlinghammond et al 2017 pianta et al 2006 programmes intensive short order facilitate reflection time retaining focus however may easily gap theoretically expected effectiveness particular design features practical reality kennedy 2016 characterised pd programmes terms theories actiondefined terms content teachers learnand programmes facilitate teachers enactment content according kennedys typology enactment facilitation pd programmes range highly prescriptive simply providing body knowledge teachers may choose react highly prescriptive programmes clearly limit teacher discretion may also tension prescription motivation effects pd programme depend teachers motivation learn change practice mandatory assignment teachers programmes may much effect learning kennedy 2016 clearly contextual aspects workplace environment organisational support may also moderate effects type pd egert et al 2018 kennedy 2016 individual teachers schools forced pd programme may provide enough personal engagement organisational support respectively change practices hence top alreadycomplex task teaching caring children young people undertake cpd may present professionals additional burden perception pd programme depend entirely receive deliver whether content relevant useful whether mode delivery suitable individual organisational context process conducting review become clear variation types pd provided professionals working children young people indeed large review aimed systematically search locate quality appraise synthesise available effectiveness studies evaluated relevant interventions using rigorous designs rigorous designs refer research designs establish causal link cpd interventions outcomes professionals children young people therefore included sr metaanalytic designs true experiments rcts quasiexperiments baseline equivalence demonstrated pretests outcomes interest excluding studies using instrumental variable approach see appendix including studies using regression discontinuity rd design ti teacher social worker police psychologist ti professional development continuing professional development cpd service training professional learning teacher learning training ab experiment quasi experiment qed control allocat randomied controlled trial rct regression discontinuity rdd scoping search produced 470 potentially relevant hits screening using preliminary inclusion criteria indicated range potentially relevant studies mainly topic area education also areas social welfare policing available systematically assembled also aware recently published metaanalysis specific area professional development professionals working childrens early language literacy development markussenbrown et al 2017 metaanalysis formed part basis electronic citation searching topic area education note search covered entire field education limited studies language literacy development review completed using sr design methods open scrutiny torgerson 2003 minimises bias increases confidence results education welfare professionals employees working directly indirectly children young people explicit purpose enhancing cognitive noncognitive development includes limited education welfare employees working towards goals settings nurseries day care child care institutions preschools schools different levels education welfare professionals either publicly privately employed receive salary work may fulltime parttime education11 tas included within although acknowledge relevance work terms welfare uk context role ta degree level profession although likely many tas working degree welfare professionals completed ordinary basic training higher education institute relevant professional degree degree varying international standard classification education iscedlevels eg diploma postgraduate certificate ba msc phd education welfare professionals recipients pd activities interventions evaluated examples education welfare professionals include teachers teaching assistants tas preschool teachers pedagogues care providers social workers paraprofessionals psychologists police officers family support providers disability specialists inclusion specialists roles education welfare professionals include planning developing delivering evaluating learning development opportunities children young people purpose review adopted following definitions inspired buysse et al 2010 cpd encompasses facilitated learning opportunities education welfare professionals completed ordinary basic training higher education institute relevant professional degree previous degree varying iscedlevels eg diploma ba msc phd cpd includes types facilitated learning opportunities types cpd shorter term informal situated practice lead credits diplomas degrees types cpd longer term involve formal coursework take place teachers colleges universities lead credits diplomas degrees aim cpd enhance professionals knowledge skills ways relevant application practice serve ultimate beneficiaries intervention children young people education welfare professionals work cpd delivered public private professional development professional training entities cpd delivered many less formal ways including coaching mentoring consultations established communities teams practice cases cpd must explicitly formulated content goals note informal allocation mentor purpose general collegial support included definition cpd cpd enhances professionals knowledge skills ways relevant better serve ultimate beneficiaries intervention children young people education welfare professionals work order inform education welfare professionspolicy makers practitionersabout nature effectiveness diversity approaches cpd important systematically search locate quality appraise synthesise available effectiveness studies two previous tertiary reviewsior reviews reviewsin field professional development educators undertaken dunst et al 201522 dunst et al 2015 located retrieved systematic searches cordingley et al 2015 metasynthesis 15 reviews dunst et al 2015 looked features pd terms delivery pedagogy etc associated positive teacher student outcomes included srs concluded range key pd characteristics led positive outcomes however reviews metasynthesis meet criteria inclusion basis key items reported article due variety factors review using sr metaanalytic design focusing pd defined example focusing induction beginning teachers sr included metasynthesis relevant review subsequently citation searched relevant empirical studies blank de las alas 2009 zaslow tout halle whittaker lavelle 2010 umbrella review cordingley et al 2015 included nine reviews international literature looking effective professional development relating findings reviews standards rigour one review identified electronic searching met inclusion criteria timperley wilson barrar fung 2007 judged consistently robust asp