Katherine S. Button
Statistical Rigor and the Perils of Chance
University of Bath
concerns reliability reproducibility biomedical research voiced across several arenas commentary discuss poor appreciation role chance statistical inference contributes problem particular poor scientific design low statistical power questionable research practices post hoc hypothesizing undisclosed flexibility analyses yield high proportion falsepositive results discuss current publication funding system perpetuates poor practice rewarding positive yet often unreliable results rigorous methods conclude discussing scientists prevent fooled chance findings adopting well established often ignored methodological bestpracticethere increasing awareness problem unreliable findings across biomedical sciences ioannidis 2005 many landmark findings could replicated scott et al 2008 begley ellis 2012 steward et al 2012 many promising preclinical findings failed translate clinical application perel et al 2007 prinz et al 2011 leading many question whether science broken economist2013 central problem poor appreciation role chance scientific process neuroscience developed past 50 years many large easily observable effects found field likely pursuing smaller subtle effects corresponding growth computational capabilities moore 1998 means researchers run numerous tests single dataset matter minutes human brain processes randomness poorly huge potential undisclosed analytical flexibility modern datamanagement packages leaves researchers increasingly vulnerable fooled chanceresearchers cannot measure entire population interest take samples use statistical inference determine probability results