Yasaman Jamshidi?Naeini|Lilian Golzarri?Arroyo|Colby J. Vorland|Andrew W. Brown|David B. Allison
Re-analysis of data from a cluster RCT entitled “health literacy and exercise-focused interventions on clinical measurements in Chinese diabetes patients”
Indiana University Bloomington|Indiana University Bloomington|Indiana University Bloomington|Indiana University Bloomington|Indiana University Bloomington
wang et al1wang l fang h xia q et alhealth literacy exercisefocused interventions clinical measurements chinese diabetes patients cluster randomized controlled trialeclin med 2019 17100211httpsdoiorg101016jeclinm201911004summary full text full text pdf scopus 10 google scholar examined effects three health literacy exercise interventions hba1c primary outcome fourarm cluster randomized trial crct account clustering nesting eight healthcare centers chcs randomly assigned four conditions two per condition repeated measurements occurred four timepoints enrollment nominally significant pvalues reported intervention effects hba1c reanalyzed data accounting clustering nesting present results herein assumption underlying validity typical inferential statistical methods independence observations outcome respondent related outcomes respondents crcts clusters randomized inferences intervention effects often intended individuals study individuals chc cluster expected similar different clusters resulting pattern correlated data errors model residuals independent across individual participants2murray dm varnell sp blitstein jl design analysis grouprandomized trials review recent methodological developmentsam j public health 2004 94 423432crossref pubmed scopus 478 google scholar3sainani k importance accounting correlated observationspmr 2010 2 858861crossref pubmed scopus 71 google scholar inherent correlation among observations cluster typically inflates type error rates accounting nonindependence within clusters lead incorrect estimation variance pvalues smaller valid analysis produce intervention effects thus invalid inferences intervention effects4brown aw li p bohan brown mm et albest oftforgotten practices designing analyzing reporting cluster randomized controlled trialsam j clin nutr 2015 102 241248crossref pubmed scopus 41 google scholar valid analysis mean analysis null hypothesis continuously distributed data continuously distributed teststatistic produces sampling distribution pvalues uniform interval 01 invalid analysis refers testing procedure produces sampling distribution pvalues study generalized estimating equations gee models random effect individual subjects used accounts repeated measurements nested within participants participants nested within chcs authors study collegially shared data us able reproduce published results per original methods1wang l fang h xia q et alhealth literacy exercisefocused interventions clinical measurements chinese diabetes patients cluster randomized controlled trialeclin med 2019 17100211httpsdoiorg101016jeclinm201911004summary full text full text pdf scopus 10 google scholar negligible differences pvalue change hdl within control group 24month followup pvalue 0042 pvalue reported study 005 differences may reasonably arise due rounding reanalyzed data intervention effects intervention compared control hba1c secondary outcomes baseline followup point herein present model accounts clustering nesting effects design within context methodologic choices original authors involved null hypothesis significance testing based pvalues 005 alpha level thus debate relative value pvalues frequentist testing within scope current work reanalyze data used linear mixed models lmm instead gee used study suggested gee unreliable type error rates crcts clusters avoided5li p redden dt small sample performance biascorrected sandwich estimators clusterrandomized trials binary outcomesstat med 2015 34 281296crossref pubmed scopus 87 google scholar gee populationaveraged approach6zeger sl liang ky albert ps models longitudinal data generalized estimating equation approachbiometrics 1988 44 10491060crossref pubmed scopus 3380 google scholar using asymptotic z test assumes large sample sizes specifically gee based methods use empiricalsandwich estimation standard errors degrees freedom limited empiricalsandwich estimation leads unreliable type error rates hypothesis testing number clusters per condition small increased variability sandwich variance estimator substantially inflates type error7murray dm design analysis grouprandomized trials oxford university press usa1998google scholar 8murray dm hannan pj wolfinger rd baker wl dwyer jh analysis data grouprandomized trials repeat observations groupsstat med 1998 17 15811600crossref pubmed scopus 99 google scholar 9hayes rj moulton lh cluster randomised trials crc press 2017google scholar 10murray dm design analysis grouprandomized trials review recent developmentsann epidemiol 1997 7 s69s77crossref pubmed scopus 25 google scholar thus would appropriate account clustering effect chcs repeated measures using gee lmm models included district covariate account stratification chcs two city districts random effect subjects nested within chcs account clustering effect chcs random effect repeated measures nested within individual participants sas software version 94 found evidence supporting efficacy interventions improving hba1c levels whereas concluded study interventions resulted statistically significant improvements hba1c among 12 betweengroup hba1c comparisons 11 originally reported nominally significant pvalue005 none statistically significant clustering nesting accounted statistical analyses table 1 present pvalues derived gee models accounting clustering nesting used study analyses using lmm accounting clustering nesting similarly none betweengroup pvalues originally reported nominally significant secondary outcomes subgroup analyses statistically significant analysestable 1time intervention effect pvalues generalized estimating equation gee models account clustering nesting linear mixed models lmm account clustering nestingall participantsparticipants suboptimal clinical measurementsdwang et al defined suboptimal clinical measurements hba1c70 sbp130 mmhg dbp 80 mmhg ldl26 mmoll hdl104 men 130 mmoll womencontrolhealth literacyexercisecomprehensivecontrolhealth literacyexercisecomprehensivecomparison vs baseline controlgeeageneralized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidlmmbour reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsgeeageneralized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidlmmbour reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsgeeageneralized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidlmmbour reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsgeeageneralized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidlmmbour reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsgeeageneralized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidlmmbour reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsgeeageneralized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidlmmbour reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsgeeageneralized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidlmmbour reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsgeeageneralized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidlmmbour reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsa1c3 monthsbaseline0276075800010314001103380001030608470897000102700004023000010265control cpvalues intervention group compared control arm changes baseline followup timerefref000103530007037100010347refref0008038400310345001503786 monthsbaseline0337077000010224000100510001020207570988000101640001002900010170control cpvalues intervention group compared control arm changes baseline followup timerefref000102860001010800010266refref00050316000101090008032312 monthsbaseline0016042500010110000101410011046102870510000100840001009900090427control cpvalues intervention group compared control arm changes baseline followup timerefref000100950001011400010282refref00010095000101060014030824 monthsbaseline0610086600010086000100570838099105430882000100600001003106310922control cpvalues intervention group compared control arm changes baseline followup timerefref000101760001013506350912refref000102090001013909770970sbp3 monthsbaseline0349062309030939022805430605070600010038000100160001004901700259control cpvalues intervention group compared control arm changes baseline followup timerefref056407680868093103330540refref0678081406520860028604506 monthsbaseline0156045200890320000101290463066200010002000100050001001000010019control cpvalues intervention group compared control arm changes baseline followup timerefref077208590237056006660823refref04900713013705150185045312 monthsbaseline0897095105640725005403100872090900010008000100370001004700010036control cpvalues intervention group compared control arm changes baseline followup timerefref062307700158044109830970refref02690547017504790387060524 monthsbaseline0995099500660217080408550699082900010004000100050001004800010016control cpvalues intervention group compared control arm changes baseline followup timerefref018803730858089307920875refref079309100097037604390662dbp3 monthsbaseline0002003802310418006001450721074000010001000100130001000100010012control cpvalues intervention group compared control arm changes baseline followup timerefref000200440478062901100195refref0165033307220774027903736 monthsbaseline0540063501850338018402760484057900010001000100020001000100010001control cpvalues intervention group compared control arm changes baseline followup timerefref017503140585064809420951refref01650336092409480291046612 monthsbaseline0389047402000290006001220527060100010001000100020001000100010001control cpvalues intervention group compared control arm changes baseline followup timerefref012802140429052308830894refref02840404077108380730079524 monthsbaseline0622070408490872099309820082019900010001000100020001000100010001control cpvalues intervention group compared control arm changes baseline followup timerefref061807030720080404400506refref023904460559070503640495ldl12 monthsbaseline0970097300010042005105210399082000810592000901080160071400510624control cpvalues intervention group compared control arm changes baseline followup timerefref000101200164063105650853refref00020131085709070977097824 monthsbaseline0001019200110236055908640118059000010130041909060001019900010221control cpvalues intervention group compared control arm changes baseline followup timerefref000100920021040601170561refref000602400753086005640817hdl12 monthsbaseline0088024800140122007502490433072700010003000901180001000500010016control cpvalues intervention group compared control arm changes baseline followup timerefref067407310014011603190558refref02000246078907410329033624 monthsbaseline0042033400990040013603020011016200010012000300300001000300010001control cpvalues intervention group compared control arm changes baseline followup timerefref036803380018017004820707refref094509240141020401940353a generalized estimating equations repeated measure nested subject id random effect gaussian family statistical model used wang et al ignores clustering effect ie invalidb reanalyzed results using linear mixed models repeated measure nested subject id subject id nested chc random effects district covariate accounting clustering effect chcsc pvalues intervention group compared control arm changes baseline followup timed wang et al defined suboptimal clinical measurements hba1c70 sbp130 mmhg dbp 80 mmhg ldl26 mmoll hdl104 men 130 mmoll women open table new tab one may argue study interventions implemented individual participants eg oneonone sessions group level eg participants receiving intervention simultaneously clustering independent treatment implementation thus accounting clustering nesting necessary consider correct even clustering truly independent treatment implementation eliminate effect clustering variance components also need emphasize posthoc calculation samples icc appropriate gatekeeper test determine whether one account clustering11jamshidinaeini brown aw mehta et ala practical decision tree support editorial adjudication submitted parallel cluster randomized controlled trialsobesity 2022 30 565570crossref pubmed scopus 1 google scholar accounting clustering regardless sample iccs magnitude potentially leads miscalculation variance components type error rates nominal significance level4brown aw li p bohan brown mm et albest oftforgotten practices designing analyzing reporting cluster randomized controlled trialsam j clin nutr 2015 102 241248crossref pubmed scopus 41 google scholar12golzarriarroyo l dickinson sl jamshidinaeini et alevaluation type error rate using parametric bootstrap analysis cluster randomized controlled trial binary outcomes small number clusterscomput methods programs biomed 2022 106654crossref pubmed scopus 2 google scholar table 1 statistical analyses crcts ignore clustering nesting invalid drawn conclusions unsubstantiated ensure integrity scientific record errors methods results conclusions study publicly acknowledged corrected results used instead conceptualization authors data analysis lga yjn investigation lga yjn writing original draft yjn review editing authors funding acquisition dba awb authors approved final draft statistical code available upon request corresponding author dataset used analyses shared us corresponding author httpsdoiorg101016jeclinm201911004 special access privileges others would last thirtysix months prior original submission manuscript dba received personal payments promises american society nutrition alkermes inc american statistical association big sky health inc biofortis california walnut commission clark hill plc columbia university fish richardson pc frontiers publishing gelesis henry stewart talks ikea indiana university arnold ventures formerly laura john arnold foundation johns hopkins university kaleido biosciences law offices ronald marron md anderson cancer center medical college wisconsin national institutes health nih medpace national academies science sage publishing obesity society sports research corp elements agency llc tomasik kotin kasserman llc university alabama birmingham university miami nestle ww formerly weight watchers international llc donations foundation made behalf northarvest bean growers association dr allison previously unpaid member international life sciences institute north america board trustees last thirtysix months prior original submission manuscript awb received travel expenses university louisville speaking fees purdue university university arkansas consulting fees la norc pennington biomedical research center award honorarium american society nutrition foundation wife employed reckitt benckiser institution dba awb indiana university received funds contracts donations support research educational activities nih usda soleno therapeutics national cattlemens beef association eli lilly co reckitt benckiser group plc alliance potato research education american federation aging research dairy management inc american egg board california walnut commission almond board peanut institute mondelez hass avocado board whistle labs inc dynamic aqs arnold ventures gordon betty moore foundation alfred p sloan foundation indiana ctsi center open science numerous forprofit nonprofit organizations support work school public health university broadly authors declare competing interests authors supported part r25dk099080 r25hl124208 r25gm141507 gordon betty moore foundation opinions expressed authors necessarily represent nih organization