Leslie Citrome|Terence A. Ketter
Teaching the philosophy and tools of evidence?based medicine: misunderstandings and solutions*
Nathan Kline Institute for Psychiatric Research|New York University|Stanford University
requirements residency programs include competency research literacy concepts process evidence based clinical practice however remains misunderstanding meant evidencebased medicine ebm emphasis placed content rather process solution incorporate course philosophy process ebm early residency curriculum reinforces need critically appraise evidence clinical relevancy using simple tools number needed treat nnt potential benefit number needed harm nnh potential harm clinicians field also need brought uptodate taught intuitive readily adopted tools ebm authors review lecture course content outline contrasting needs expectations resident physicians training programs versus clinicians field brief mini course curriculum approach continuing medical education presentation described recent years increasing emphasis placed practice ebm originally proposed sackett colleagues ebm exhorts clinicians incorporate best available research evidence regarding efficacy safety individualized patient assessment patient preferences medical decision making 1 figure 1 outlines key steps process medical schools devote substantial amount time topic 2 however amount emphasis process formulating questions searching answers appraising evidence applying results measuring outcome 3 consistent across institutions 4 incoming residents residency programs thus different exposures material likely little instruction apply clinical practice clinicians trained ebm became part medical school curricula may completely unaware ebms existence fivestep evidencebased medicine process particularly problematic wide gap exists best evidence customary practice medicine clinical trials provide large amount data useful daytoday practice clinicians often struggle find interventions make difference wellbeing patients often challenging determine whether study result actually transform routine practice however clinically intuitive ebm tools available help gauge clinical significance clinical relevance instead focusing exclusively whether study result statistically significant p 05 level clinical significance quantified calculating effect size difference two interventions effect size measured compared variety ways one approach particular easy calculate understand nnt potential benefit nnh potential harm 5 agreeing statistically significant clinical trial result important clinician patient nnt nnh calculated determine whether difference probably affect success daytoday clinical practice magnitudes nnt nnh help clinicians decide whether findings statistically significant eg p 005 level also clinically relevant although accreditation council graduate medical education requirements residency program includes competency research literacy concepts process evidence based clinical practiceeg see psychiatry requirements 6 sometimes remains misunderstanding meant ebm emphasis placed content rather process whereas residency programs allocated substantial amount resources teaching process philosophy ebm 7 8 others topic part american society clinical psychopharmacologys model psychopharmacology curriculum one optional lecture 9 authors engaging psychiatric residents postresidency clinicians incorporate philosophy ebm daytoday clinical decision making mainly providing practical introduction basic tools appraising clinical trial evidence two commonly encountered beliefs statistical significance alone mean study result impressive sophisticated appraisal tools effect size statistical concepts esoteric complicated practical use clinician frontlines moreover whenever term evidencebased practice discussed assumption often content treatmentguidelines rather process whereby evidence obtained appraised applied individual patient using clinicians clinical judgment using shareddecision model incorporates patients preferences values minicurriculum ebm introduced fall 2008 tuesday teaching afternoons organized senior residents adult psychiatry residency program new york university langone medical center 10 selected readings distributed electronically advance book practice teach ebm distributed first session 11 four slide sets created using ascp model curriculums ebm lecture base 9 presented four 1h inperson live didactic sessions opportunities questions answers among approximately 12 participants content overlaps allowing redundancy objectives lecture include distinguishing ebm 1 outlining steps involved practicing ebm 3 reviewing concept benefit risk quantify 5 major teaching points ebm provides clinicians strategy coping overwhelming amount data flood conscientious clinicians provides systematic way formulating clinical questions structuring search information integrating best available data patients needs values arrive optimal treatment decisions data bases evaluation tools algorithms available internet facilitate adoption ebm methods save valuable time improving patient care hook harnessing attention participants showing graph similar figure 2 makes explicit difference statistical clinical significance residents usually readily notice degree difference figure exaggerated starting vertical axis 35 rather 0 large ns yield high degree statistical significance new quantification often difference actually encountered clinical practice case nnt 50 ebm defined process philosophy set tools definitely cookbook medicine learning process rather focusing content important evidence ever changing lecture goes detail properly formulating questions searching answers productive exercise hierarchy evidence quality reviewed well find including availability cochrane database systematic reviews review benefit risk provides framework discussion absolute vs relative risk effect size lecture concludes revisiting initial slide quantifying little clinical relevance particular statistically significant finding despite appearance graph robust pvalue following points reinforced ebm goes beyond anecdotal evidence allows integration clinical research clinical practice tools ebm include calculation effect size tell us clinical significance statistically significant result ebm requires us use clinical judgment weigh benefits risk individual patient moreover ebm important paradigm applicable mental health help us explain justify treatment decisions increase clinical effectiveness appraise value treatment interventions differences statistical clinical significance quantify relevant irrelevant clinical trial outcome expands detail upon introductory lecture objectives include able quantify clinical significance using nnt expressing statistical significance nnt describing confidence interval ci rules thumb interpreting different magnitudes nnt provided using numerous examples psychiatric general medical literature mechanics calculating ci reviewed limitations using nnt discussed usually entailing lively discussion uniqueness patients currently treated residents relative measures relative risk ratio odds ratio hazard ratio defined guidance provided interpret 12 following points reinforced absolute differences place data clinically meaningful context relative differences deceiving concept nnt allows clinician estimate medications potential clinically relevant effect examining magnitudes nnt nnh clinician start make risk benefit decisions tailored individual patients needs preferences lecture steps involved answering clinical question reviewed ebm tool nnt used compare interventions limitations approach discussed major teaching points nnt quantify clinical significance statistically significant result nnts must interpreted within clinical context individual patient magnitudes nnt nnh may may comparable terms importance two examples provided agitated patient schizophrenia 13 ambulatory patient chronic schizophrenia 14 15 clinical question formulated using pico approach patientproblem intervention comparisoncontrol outcomes 16 searching answers demonstrated seeking appropriate references controlled studies appraising quality evidence reviewed relevant nnt nnh measures calculated applied hypothetical individual patient combination nnt nnh single measure called likelihood helped harmed lhh introduced 17 following points reinforced nnt nnh easily calculated nnt nnh must interpreted clinical context every patient individual patterns treatment different set preferences lhh help discussing tradeoffs benefit risk provided meaningful clinician patient comparable terms time course objectives last lecture review steps involved searching literature review publication process know set electronic resource library last topic typically covered elsewhere residency curriculum essential todays weboriented world major teaching points include online search engines locate papers interest provided know look controversy surrounding journal impact factor discussed 18 followed discussion online resources including medical literature databases httppubmedgov online government university libraries commercial web portals httpwwwmedscapecom process journal publication outlined including differences indexed nonindexed journals journal tiers importance preliminary inquiry editor contemplating review mechanics online submission vagaries review process including reviewer referee ostensibly blind referee remains anonymous newer processes open reviews posted commentaries last topic creation organization personal electronic library including discussion scanningdownloading storagebackup retrieval viewing 19 following points reinforced impact factors often talked terms important journal however find articles journals low impact factor easily articles journals high impact factor pubmed great equalizer terms finding articles interest found article interest keeping electronic library makes portable easily retrievable time available practicum added small groups 24 residents tackle clinical problem use ebm process address presenting information clinicians residency training long since completed poses special challenges luxury several hours usually available although grand rounds serve vehicle occasional focused presentations ebm useful daytoday clinical practice process ebm use ebm tools nnt nnh incorporated content driven continuing medical education focuses treatment interventions example recent continuing medical education presentations one authors regarding treatment bipolar disorder start overview treatments approved united states food drug administration fda examples treatments evidence regarding efficacy safety thus important enough warrant assessment nnt nnh 20 method calculating nnt presented along sample calculation well broad interpretation nnt thus emphasized fda approved treatments acute mania acute bipolar depression maintenance treatment singledigit nnts response relapse prevention compared placebo indicating treatments least 10 advantage placebo next method calculating nnh presented along sample calculation well broad interpretation nnh example instances nnhs doubledigit magnitudes substantially greater corresponding nnts considered desirable depending severity adverse effects considered followed discussion need personalize assessments benefit risk clinicians initially experience effort needed think quantitatively issues exposures approach process appreciated requires little additional effort applied new treatment options straightforward fashion another alternative postresidency teaching includes pharmaceutical companysponsored events recent dinnertime promotional lecture series sponsored pharmaceutical company assessing benefit risk antipsychotic treatment using evidencebased approach added 2030 min component 60min talk 21 lecture intended focused disease state without mention claims sponsors product topics covered ebm portion included definitions discussion differences p values effect sizes calculate apply nnt nnh example calculation nnt based data presented first half talk attendees eligible receive free ebm handbook 11 whether initiatives successful changing ones practice remains established informal pre posttest questions confirm least participants able understand different levels evidence pvalue effect size ebm slavish adherence preprinted practice guideline rather incorporates clinical experience patient preferences research evidence decisionmaking process increase knowledge worthwhile accomplishment persons never even exposed acronym ebm greater appreciation need teaching ebm 22 ideally hours allocated permit thorough coverage material opportunity work relevant exercises reinforce concepts learned didactically retrospect minicourse presented would useful earlier residency program periodic refreshers subsequent training years important message ebm merely esoteric theoretical exercise rather dynamic process intelligent decision making made explicit ultimately however clinically integrated teaching ebm throughout training experience likely bring changes skills attitudes behavior didactic instruction 23 although lectures prepared minicurriculum included pre postassessment questions determining whether attendees actually applied learned requires formal testing followup underemphasized potential benefit ebm provides opportunities enhance therapeutic alliance facilitate shared decision making 24 example using best available evidence presenting quantifiable terms favorable outcomes response remission maintenance response unfavorable outcomes adverse events help clinician patient decide among interventions second generation antipsychotics vs lithium anticonvulsants vs combination treatment bipolar disorder choice highly dependent individuals values preferences including tolerability risk guidance curriculum development teaching obtained training initiatives including ebm pediatric psychiatry 25 26 table 1 outlines additional resources limitations ebm applies mental disorders also acknowledged 27 namely reliance randomized clinical trials evidence efficacy proposed intervention limited restrictive inclusion criteria trial patients comorbid psychiatric medical conditions often excluded participating trials also even though combination therapies extremely common adequately powered randomized controlled trials interventions moreover randomized controlled trials shortterm thus fail answer questions regarding longterm treatment another limitation outcomes measured studies may capture patients actually experience disorder therapy thus turn value intervention evidencebased medicine taught trainees well clinicians already midst careers emphasis teaching process rather focusing solely content proverb give man fish feed day teach man fish feed lifetime particularly apt clinical evidence moving target treating patients able find answers clinical questions appraise applicability importance answer basic tools nnt nnh provide means quantifying benefit risk clinician patient fostering collaboration shared decision making leslie citrome md mph consultant received honoraria conducted clinical research supported following abbott laboratories astrazeneca pharmaceuticals barr laboratories bristolmyers squibb eli lilly company forest research institute glaxosmithkline janssen pharmaceuticals jazz pharmaceuticals pfizer inc terence ketter md consultant received honoraria conducted clinical research supported following abbott laboratories astrazeneca pharmaceuticals bristolmyers squibb company cephalon eli lilly company glaxosmithkline janssen pharmaceutica jazz pharmaceuticals novartis pharmaceuticals organon international inc pfizer inc repligen corporation solvay pharmaceuticals valeant pharmaceuticals vanda pharmaceuticals wyeth pharmaceuticals xenoport inc