Peter Tugwell|J. André Knottnerus
The need for consensus on consensus methods
science consensusorienteddecisionmaking methods still infancy despite increased emphasis patient multiple stakeholder engagement clinical research process used variety objectives including research terminology setting research priorities establishing core outcome domains instrument sets reporting guidelines major topic recent omeract outcomes meeting australia key elements approach agreement seeking collaborative cooperative egalitarian inclusive participatory many methods achieving consensus common one used health research delphi journal published number papers method 1verhagen ap de vet hc de bie ra kessels ag boers bouter lm et althe delphi list criteria list quality assessment randomized clinical trials conductingsystematic reviews developed delphi consensusj clin epidemiol 1998 51 12351241abstract full text full text pdf pubmed scopus 1702 google scholar 2maclennan kirkham j lam tbl williamson pr randomised trial comparing three delphi feedback strategies found evidence difference setting high initial agreementj clin epidemiol 2018 93 18abstract full text full text pdf pubmed scopus 30 google scholar 3diamond ir grant rc feldman bm pencharz pb ling sc moore et aldefining consensus systematic review recommends methodologic criteria reporting delphi studiesj clin epidemiol 2014 67 401409abstract full text full text pdf pubmed scopus 1092 google scholar 4dahmen r van der wilden gj lankhorst gj boers delphi process yielded consensus terminology research agenda therapeutic footwear neuropathic footj clin epidemiol 2008 61 819826abstract full text full text pdf pubmed scopus 13 google scholar however great variation implemented demonstrated paper grant et al analysed data eight online modifieddelphi panels used set priorities patientcentered outcomes research three topics weight management obesity congestive heart failure kawasaki disease tremendous variation lack consistency way analysis conducted round way decisions made achieve consensus concern insufficient safeguards unacceptable datamining need consensusmethods evidence base minimise substantive biases meanwhile important transparency method implementation used delphi studygrant et al recommend delphi researchers prospectively completely register protocol containing preanalysis plans publicly available independently controlled platform timestamps entries journals research sponsors require prospective registration delphi processes precondition publication funding improving participation rates topic two papers optimising study participation rates completing study questionnaires important internal external validity whether research design experimental observational previously focus controlled trials systematic reviews show successful responseinducing strategies included monetary incentives prenotification telephone reminders recorded delivery use shorter questionnaires van gelder et al show systematic review issue different approaches necessary increase rates questionnaire completion observational studiesespecially completion rates controls nonpatients among patients responseinducing strategies paperbased questionnaires included unconditional monetary incentives shorter questionnaires found others randomised controlled trials however controls additional strategies work include webbased questionnaires also increased conditional lottery ticketsit would important ensure additional incentives induce bias explaining randomisation patients recruited clinical trial one common forms informed patient engagementjepson et al conclude done poorly impeded clinical trial recruitment based qualitative analysis 73 audio recordings recruitment consultations five multicenter ukbased rctsin oncology orthopedics vascular surgery using gamblingrelated metaphors computerdriven explanations especially confusing patients call support training involved obtaining informed consent cross cultural adaptation questionnaires requires well validated instrument application cornucopia good clinimetrics psychometrics probably reserved popular outcome instruments yao et al provide good example looking rolandmorris disability questionnairethere 1000 papers use translated 31 instruments covering 26 languagescultures however minority versions met crosscultural adaptation criteria beaton et al 5beaton de bombardier c guillemin f ferraz mb guidelines process crosscultural adaptation selfreport measuresspine phila pa 1976 2000 25 31863191crossref pubmed scopus 6656 google scholar health status clinimetricpsychometrics methods terwee et al 6terwee cb bot sd de boer mr van der windt da knol dl dekker j et alquality criteria proposed measurement properties health status questionnairesj clin epidemiol 2007 60 3442abstract full text full text pdf pubmed scopus 6035 google scholar example six instruments assessed responsiveness change booth et al present claimed first known example patientcentred criterionbased approach inform selection qualitative evidence synthesis methods comprised seven domains nicely reflected acronym retreat review question epistemologytimetimescale resourcesexpertise audience purpose type data objective address current deficit understanding selection criteria important among many involved qualitative synthesis often leads mismatch aims quality evidence synthesis optimal methods address aims examples presented around experiences pain adolescence complex reality experienced patient wishes die suggest retreat framework extended documenting characteristics types evidence synthesis beyond involve synthesis qualitative research clinicians want recommendations argued recommendations unnecessary may even counterproductive genuine evidencebased practice others argue although recommendations useful context high certainty evidence may inappropriate context low low certainty evidence response controversy neumann et al conducted international multicenter randomized controlled trial determine whether clinicians prefer recommendations accompanying evidence summaries versus structured evidence summaries alone clinical questions informed low low certainty evidence multicenter study 200 practicing clinicians 10 countries compared evidence summaries without grade recommendations showed recommendations addition evidence summary preferred high low certainty evidence although main outcome worryingly less half participants interpreted evidence summaries correctly systematic literature searching hallmark systematic reviews treatment aspects clinical care indeed carried undertaking primary researchthe topic upcoming jce series however cooper et al review issue point consensus metrics systematic literature searches assess yield relevant articles criteria adequacy 50 studies six metrics identified sensitivity specificity precision accuracy number needed read yield almost used handsearching goldstandard systematic reviews report need consensus metric acceptable thresholds network metaanalyses diagnostic tests comparative effectiveness network metaanalyses combine direct indirect comparisons effectiveness one active intervention increasingly accepted clinical care policymaking encouraging see conceptual approach applied diagnostic tests challenge added complexity comparing sensitivity specificity differing decision thresholds owen et al issue review different proposed approaches demonstrate one prefer using markov chain monte carlo mcmc methods fit bivariate network metaanalysis model incorporating constraints increasing test threshold accounting correlations multiple test accuracy measures study motivating example comparing minimental state examination montreal cognitive assessment bias assessments focus four papers debate best instrument assessing risk bias systematic reviews continues evolve new instruments robinsi robis modifications existing ones amstar 2 rob 20 rcts banzi et al compare assessment 5 raters two original amstar robis applying convenience sample 31 systematic reviews peer review journals 5 cochrane reviews found amstar robis appear offer similar reliability fair correlation overall judgments mainly differences domains final scores defined robis takes longer calls fuller knowledge systematic review methodology statistics well clinical expertise usefulness applying prisma amstar systematic reviews types shown paper sharma oremus applied instruments 19 systematic reviews health literacy cancer screening found reviews good quality general deficits uncovered using scales included absence protocol registration publication lack dual screening data extraction absence gray literature searches failure provide lists excluded articles lack integration risk bias assessments interpretations efficacy bolzern et al make bold statement significant numbers recently published cluster trials major journals may fitforpurpose results invalidated frequent occurrence selection bias reflected large baseline differences age although sample size 23 cluster trials published major journals 2015 2017 large found previous time periods concern baseline selection bias continues present cluster trials especially showing statistically significant results published high impact journals whereas seen trials journals individual randomisation authors argue design necessary individuals consented randomization missing outcome data nonrandom bias invalidate third individual trial results metaanalyses even mentioned publications state unexplained loss followup methods available explicitly assess risk bias associated missing outcome data test whether significance effect estimates robust issue kahale et al tackle issue metaanalyses demonstrate lack adequate reporting cochrane noncochrane systematic reviews provide 10item categorisation reasons missing outcome data used systematic review authors two articles address scaling issues multimorbidity indices increasingly needed comorbid conditions increase increased survival populations globally ethnicity deprivation important factors received specific attention previously gurney et al demonstrate factors integrated m3 measuring multimorbidity index improved performance two commonly used indices charlson elixhauser indices predicting important adverse outcomes 200000 patients diabetes new zealand important see weights also work conditions countries knowledge translation research results practice policy severely hampered predilection researchers use different scales assess outcomes transformation results different scales common metric essential attempting draw conclusions sunderland et al demonstrate done methodologically robust fashion using item response theory applied rescoring observed scores common scale call equated score 11 selfreport social anxiety scales 5 50 items via fixed parameter item calibration method prognosis systematic reviews metaanalyses estimates prognostic association given priority cochrane collaboration complement systematic reviews metaanalyses treatment number challenges prez et al report finding alternative extracting estimates metaanalysis prognostic marker studies details provided insufficient lose good studies applied example recent 122 study systematic review association lipid markers cardiovascular events demonstrate use method based simulating individual patient data performed better commonly used generalized least squares trend estimation although healthcare claims data widely used medical research ability identify persons healthrelated conditions problematic especially perverse financial incentives occur making certain diagnoses hara et al provide example japan using diagnostic medication codebased claimsbased algorithms applied longitudinal database five million claims successfully diagnosed hypertension diabetes andor dyslipidemia applying health screening results gold standard authors suggest framework used assess validity claims based algorithms data finally thoughtful commentary sacristan dilla arguing applicability individualization generalisability contrary current thinking time walk back population average patient individual patient understanding populationoriented research actually exploratory individualoriented research confirmatory