David B. Elliott
Lies, damned lies and……….
University of Bradford
wellknown quotation lies damned lies statistics suggests statistics doesnt greatest reputations certainly statistics used mislead logically lies damned lies statistics reflects high levels statistical misunderstanding rather indicating statistics inherently bad bad science ben goldacre highlights incorrect presentation statistical facts figures provided newspapers television attributes authorship humanities students lack science background1 surprising given science students clinicians also find statistics difficult understand emergence evidencebased medicine evidencebased optometry2 means clinicians need least basic understanding statistics able reasonably determine validity published research apply patient care unfortunately studies reported low level knowledge basic principles research methods statistical analysis among doctors3 4 even researchers reviewers editors show lack understanding statistical issues growing number published papers highlighting errors statistical design andor analysis research studies make review process published biomedical5 6 ophthalmic7 8 research literature course may represent differences opinion statisticians9 others may include misuse statistical analyses best help publication example10 given academic pressure publish perish11 seem relatively minor factors developments evidencebased practice highlight need develop statistical literacy amongst clinicians3 4 undergraduate curricula need change appropriate modules already present2 addition seems important postgraduate programmes include modules principles research design statistics finally given statistical understanding continues advance researchers need keep abreast current knowledge perhaps research journals help 1986 assessment clinical tests terms agreement tests repeatability radically altered martin bland doug altman classic paper statistical methods assessing agreement two methods clinical measurement12 highlighted inappropriate use correlation coefficients instances presented alternative approach based graphical techniques simple calculations called blandaltman analysis see figure 1 example one inadequacies correlation coefficients analyses highly dependent range values used large range data say visual acuity patients large range va including low vision would tend provide high correlation coefficient small range va patients normal healthy eyes would provide low coefficient effect beautifully illustrated gunillahaegerstromportnoy colleagues13 shown figure 2 shows correlation coefficient 091 correlation coefficients literature ophthalmic agreement studies compared range visual acuity used analyses studies another classic statistical paper ophthalmic clinical research clinical grading effects scaling ian bailey mark bullimore tom raasch hugh taylor14 highlighted tests procedures show high levels concordance score test retest typically good tests merely tests large step size think visual acuity chart 10 050 00 logmar lines ie snellen lines 660 618 66 20200 2060 2020 limitations chart detecting change differences eyes obvious yet testretest scores would highly concordant paper led changes sorts tests grading scales byline scoring byletter scoring 4point grading scales much finer grading scales providing much greater repeatability discriminative ability ability detect change simple statistical tweak despite classic papers unfortunately always read andor adhered continue receive papers assess agreement tests repeatability tests using correlation coefficients papers reporting qualities tests excellent concordance values tend classic four clinical grades also regularly receive papers statistical errors show reason journal published series invited papers commonly encountered statistical errors collated virtual issue journal webpage httponlinelibrarywileycomjournal101111issn14751313homepagestatisticalguidelineshtm made free access briefly summarised ophthalmic journals also providing statistical notes15 detailed summary analyses appropriate assessment agreement repeatability ophthalmic tests briefly discussed provided colm mcalinden jyoti khadka konrad pesudovs16 also provide suggestions regarding sample sizes studies common error use data right left eyes subjects evaluating test procedure analysing data though independent advantage approach sample size easily doubled seems likely commonly misused however richard armstrong highlighted problems approach suggested ways around easiest use data one eye subject although whether using right eye cases seems regularly case appropriate open debate discussed paper17 many statistical analyses used typically large number correlation coefficients variety clinical test results increasing chances gaining statistically significant pvalue due chance example 100 correlation coefficients made could assumed five would significant p 005 due chance common adjustment correct potential type 1 error bonferroni adjustment typically would involve dividing 005 number analyses made conservative adjustment held high regard authors reviewers much others18 discussed detail richard armstrong issue opo19 mistakes also often made presentation statistics following suggestions mainly derived doug altman colleagues 1983 paper statistical guidelines contributors medical journals20 part author guidelines opo truths great truths statistics may never catch attempt make progress albeit slow