J O'donnell
<i>P</i>?value. What value?
Christchurch Hospital|Canterbury District Health Board|Canterbury Health Laboratories
although problems identified statement known several decades previous expressions concern calls action fostered broad improvements practice2 p value 005 carries 5 risk false positive result ie true difference treatments trial meant provide proof genuine treatment difference beyond reasonable doubt much smaller p value say p 0 001 required5 disagree statement erroneous according null hypothesis p 005 occur 5 time6 editorial corrigendum appeared pvalue area curve probability distribution defined mathematical model model usually presented graphically describes expected distribution sample statistic around central measure parameter theoretical true value example population mean central limit theorem would standard normal distribution sample means generated repeat sampling population variable interest mean sample means would equal true population mean medicine rare us ever know true value variable interest however usefully assign value special case difference statistic example difference mean outcome variables placebocontrolled drug trial case sampling distribution would represent difference statistic case value assign zero mathematical model becomes null hypothesis used nhst way contrast noninferiority drug trials require nonzero value assigned cumulative auc sampling distribution continuous variable represented mathematical function called cumulative density function medical science study variables continuous categorical transformed using logit model pvalue mathematical integral cumulative auc cannot take precise value auc defined single point curve example pvalue 005 p 005 may seem pedantic semantics statistical inference influential thinking decisionmaking yet misinterpretation misuse terminology commonplace null hypothesis one sample mean happens fall within extreme region standard normal distribution may expected occur low frequency say p 005 meaning sample mean one extreme would expected occur frequency 5 less valid assumptions independence random selection sample mean selected normal distribution sample means must assumed another way stating conditional probability note means given important understand pvalue measure conditional assumption mathematical model describes distribution sample means measure probability truth mathematical model make claim would invert conditional probability statement commit error reasoning called transposing conditional7 aka prosecutors fallacy reasoning nhst commonly used definition pvalue measure evidence null hypothesis potentially misleading seems legitimise transposing conditional mathematically valid function rather matter intuition intuitive interpretation fisher used posteriori model nhst8 9 aim use pvalue aid deciding experiments repeat several repetitions consistent extreme pvalue sample statistic obtained would accumulate evidence true experimental effect effect present regression mean parameter would expected p 005 reallife scenarios many factors inhibit repetition replication experiments however modelling give us insight precision reproducibility extreme pvalues10 11 hence intuitive weight place pvalue measure evidence null hypothesis table 2 reproduction10 describes results simulating repeat experimentation probability producing pvalue 005 prescribed conditions simulated experiment may surprising many poorly reproducible pvalue bright line test bright line test clearly defined rule standard purpose produce consistent predictable results example first experiment p 005 produced would 50 probability reproducing p 005 repeat experiment p 001was produced first experiment probability producing p 005 repeat experiment would 73 p 0001 produced first experiment probability p 005 repeat experiment would 91 magnitudes number first experiment pvalues commonly used pharmaceutical trials medical analyses pvalue also sensitive sample size irrespective effect size increasing sample size n pvalue made small wish12 standard error proportional inverse n statistical significance substituted clinical significance even small irrelevant differences may regarded worthy investment large sample sizes often feature pharmaceutical trials secondary primary prevention interventions preventive therapies atherosclerotic diseases osteoporosis quoted extract article clinical trials mistakenly promotes pvalue measure error states error rate legitimately adjusted depending magnitude pvalue thus providing proof genuine treatment difference beyond reasonable doubt erroneous interpretation arisen illusion coherence resulting conflation dominant models hypothesis testing8 9 setting theoretical type 1 type 2 error rates neyman pearson model envisions frequency error long run experience experimental repetition given randomness independence sample means two juxtaposed probability distributions priori two identical populations imagined except differ mean parameters null 0 alternative model valuable providing rationality sample size selection however conflation resulted confusion fishers pvalue neymans giving pvalue apparent legitimacy posteriori sliding type 1 error rate even logical decreasing would increase resulting decrease power 1 also dichotomous approach pitting null hypothesis alternative hypothesis carries risk blinding researcher consumer explanatory hypotheses think use confidence intervals ci overcomes problems described think although greater intuitive value especially respect estimating effect size ci relies premises pvalue example ci juxtaposed probability distributions made large small paid increasing sample size small difference ci made overlap statistical analyses valuable tools extracting information data however reliability knowledge generated dependent many important factors inter alia evidential justification experimental hypothesis study design study conduct data collection cleansing competence choice statistical model valid reasoning reviewer bias publication bias replication much criticism medical science centres overemphasis importance pvalue nhst statistically defined effect sizes better understanding sound statistical inferences made influence decision making key elements improving aspects healthcare critically important acknowledgement individuals complex adaptive systems characteristics emergence adaptability nonlinearity unpredictability13 rather static population averages surveys suggest statistical literacy amongst doctors low14 15 teaching assessing knowledge application statistical inference critical appraisal decisionmaking skills primary focus medical schools specialist colleges difficult concepts underpinning statistical inference may effectively efficiently taught using computer simulation whereby learner manipulate effect sizes sample sizes statistics order see parameter estimates pvalues ci change reproduction replication16 foster indepth understanding limits statistical inference making clinicians better able choose wisely amongst myriad investigations treatment options offer subsequent article submission review author attended referenced asa conference2 special issue asa journal reporting conference proceedings planned 2018 opening addresses 400 participants encouraged devote energies developing proposals goals address long standing yet stubbornly persistent errors statistical inference described article concrete proposals yet endorsed asa many speakers emphasised need place greater emphasis teaching conceptual framework different philosophical approaches science mastering concepts priority rather mechanics statistical inference need better understanding statistical semantics part nonstatistician scientists also highlighted best way achieve understanding would develop contextspecific learning modules aspect conference resonated author respect prediction medical science idea science defines degrees uncertainty certainty apropos caution must applied use prediction models medical practice lest overextended