Leslie Curry|Ingrid M. Nembhard|Elizabeth H. Bradley
Qualitative and Mixed Methods Provide Unique Contributions to Outcomes Research
Robert Wood Johnson Foundation|Yale University|Robert Wood Johnson Foundation|Yale University|Robert Wood Johnson Foundation|Yale University
homecirculationvol 119 10qualitative mixed methods provide unique contributions outcomes research free accessresearch articlepdfepubaboutview pdfview epubsections toolsadd favoritesdownload citationstrack citationspermissions shareshare onfacebooktwitterlinked inmendeleyreddit jump tofree accessresearch articlepdfepubqualitative mixed methods provide unique contributions outcomes research leslie curry phd mph ingrid nembhard phd ms elizabeth h bradley phd leslie curryleslie curry robert wood johnson clinical scholars program department internal medicine lac ehb section health policy administration lac imn ehb department epidemiology public health yale university school medicine new conn search papers author ingrid nembhardingrid nembhard robert wood johnson clinical scholars program department internal medicine lac ehb section health policy administration lac imn ehb department epidemiology public health yale university school medicine new conn search papers author elizabeth h bradleyelizabeth h bradley robert wood johnson clinical scholars program department internal medicine lac ehb section health policy administration lac imn ehb department epidemiology public health yale university school medicine new conn search papers author originally published17 mar 2009httpsdoiorg101161circulationaha107742775circulation 200911914421452outcomes research examines effects medical care interventions policies health outcomes individuals society1 investigators conducting outcomes research seek inform development clinical practice guidelines evaluate quality medical care foster effective interventions improve quality care2 outcomes research traditionally used quantitative sciences examine utilization cost clinical effectiveness medical care randomized nonrandomized experimental designs quantitative methods well suited measure complex aspects healthcare delivery system organizational change clinical leadership implementing evidencebased guidelines patient perceptions quality care also critical issues outcomes research37 nuanced aspects healthcare delivery may appropriately examined qualitative research methods810qualitative approaches becoming common clinical medicine health services research51115 federal encouragement qualitative research regularly reflected funding program announcements issued national institutes health16 decade federal agencies foundations national science foundation demonstrated commitment supporting qualitative research funding scientific conferences workshops monographs field inquiry1720 despite steady growth qualitative research outcomes investigators cardiology relatively little guidance best implement methods investigationsthe purpose present report introduce qualitative methods providing unique critical contributions outcomes research report describe situations qualitative approaches helpful summarize primary principles practices study design sampling data collection data analysis qualitative studies present representative examples cardiovascular outcomes research uses qualitative methods synthesize current standards ensuring rigor enhancing credibility qualitative researchdefining qualitative researchqualitative research form scientific inquiry spans different disciplines fields subject matter comprises many varied approaches21 qualitative methods used understand complex social processes capture essential aspects phenomenon perspective study participants5 uncover beliefs values motivations underlie individual health behaviors2224 research also illuminate aspects organizational context healthcare delivery influence organizational performance quality care1025 qualitative studies often exploratory nature seek generate novel insights9232627 using inductive starting observations developing hypotheses rather deductive starting extant hypotheses testing observations approachesqualitative research distinguished quantitative research several ways first whereas quantitative research counts occurrences eg estimates prevalence frequency magnitude incidence qualitative research describes complexity breadth range occurrences phenomena second whereas quantitative research seeks statistically test hypotheses qualitative research seeks generate hypotheses phenomenon precursors consequences third quantitative research performed randomized nonrandomized experimental natural settings generates numeric data standardized processes instruments predetermined response categories qualitative research occurs natural rather experimental settings produces textbased data openended discussions observationsmixed methods quantitative qualitative methods combined increasingly recognized valuable capitalize respective strengths approach28 pairing quantitative qualitative components larger study achieve various aims including corroborating findings generating complete data using results 1 method enhance insights attained complementary method2930 approaches mixedmethods studies differ basis sequence components occur emphasis given each2930 qualitative quantitative components may performed concurrently sequentially emphasis may placed either component equal weight given instance preliminary qualitative component may serve generate hypotheses develop content questionnaire used followup quantitative study conversely preliminary quantitative component may generate surprising inconsistent findings may examined greater depth followup qualitative component strategies enhance validity mixedmethods studies include recognizing role complementary strategy adhering methodological assumptions method31 principles practices specific mixedmethods research described extensively2829 current applications reported recently established journal mixed methods32 although present report focuses primarily qualitative research illustrative examples also provided mixedmethods studies highlight full range potential applicationswhen consider using qualitative methodsqualitative methods considered research aim 1 following investigate complex phenomena difficult measure quantitatively generate data necessary comprehensive understanding problem gain insights potential causal mechanisms develop sound quantitative measurement processes instruments study special populations table 1 table 1 consider using qualitative methodsresearch aimexamples contributions qualitative methodsinvestigate complex phenomena difficult measure quantitativelycharacterize organizational processes dynamics change time describe social interactions elicit individual attitudes preferencesgenerate data necessary comprehensive understanding problemprovide detailed descriptions individual perceptions experiences enhance quantitative measures phenomenagain insights potential causal mechanismsgenerate hypotheses given intervention specific impact impact occurs organizational context occursdevelop sound quantitative measurement processes instrumentsidentify patientcentered measures healthrelated constructs assess crosscultural equivalency existing toolsstudy special populations traditionally underrepresented research low literacyimprove methods recruitment retention measurementinvestigating complex phenomena difficult measure quantitativelycomplex phenomena organizational processes change processes time social interactions underlying specific outcomes may difficult measure quantitatively9 qualitative methods helpful identifying characterizing multifaceted organizational dynamics influence outcomes including organizational culture clinical leadership teambased interactions10 instance 1 large qualitative study examined process implementing innovative technology cardiac surgery hospitals toptier cardiac surgery departments researchers conducted interviews across 16 hospitals average 10 key staff per site characterize changes organizational group routines triggered technology adoption findings generated process theory collective learning within teams influences success technology adoption hospital settings33generating data necessary comprehensive understanding problemin cases quantitative data alone fully address research question qualitative approaches useful qualitative component provide detailed perspectives individuals descriptions processes thereby ensuring comprehensive understanding phenomenon interest instance quantitative evidence regarding selfcare practices patients congestive heart failure indicated interventions designed enhance selfcare modest success suggests need improve understanding potentially modifiable patientlevel factors horowitz colleagues34 illuminated dynamics selfregulation processes patients congestive heart failure using qualitative methods researchers interviewed 19 congestive heart failure patients characterize processes found patients limited knowledge congestive heart failure failed recognize act symptom exacerbation encountered variety barriers care findings used modify existing model selfcare fit unique experiences patients congestive heart failuregaining insights potential causal mechanismsqualitative research also illuminate potential causal mechanisms associated given outcome generate hypotheses mechanisms qualitative approach useful researchers interested looking beyond identified variables statistically linked desired effect understand given intervention specific impact impact occurs organizational context exploration causal mechanisms underlying statistical association may performed either quantitative association found instance bradley colleagues used qualitative methods characterize potential causal factors influence increased blocker use acute myocardial infarction35 well reduced time treatment patients stsegment elevation myocardial infarction3637 studies set foundation subsequent quantitative research determine statistical associations qualitatively identified factors outcomes broader samplesdeveloping sound quantitative measurement processes instrumentsqualitative methods useful development patientcentered quantitative instruments outcomes research example qualitative methods used early stages developing patientcentered quantitative measures healthrelated constructs38 kressin colleagues39 used approach developing psychometrically sound patientcentered survey instrument measuring healthrelated beliefs attitudes experiences cardiac patients veterans administration system first phase qualitative data gathered 13 patients undergone cardiac stress testing positive results elicit patients perceptions interaction providers including communication patient concerns preferences40 findings subsequently used construction validation standardized questionnaire assess patient beliefs preferences experiences surrounding cardiac decision makingqualitative approaches also important concerns crosscultural equivalency existing tools psychological principles measures crossvalidated different populations may based researchers preconceived perhaps erroneous assumptions41 qualitative studies help researchers evaluate saliency acceptability defined constructs identify missing constructs employ appropriate language tool create suitable response formatsstudying special populationsqualitative research offer important benefits studies involve special populations including traditionally underrepresented research low literacy42 qualitative data collection methods openended interviews may effective less intimidating surveys historically marginalized research qualitative approaches also used improve methods recruitment retention underrepresented groups health research instance koops colleagues43 sought address ethical challenges conducting research thrombolysis acute ischemic stroke patients large clinical trials required determine potential benefits treatment constrained inherent ethical barriers including known risk early fatal intracranial hemorrhage patients potentially compromised ability provide timely informed consent researchers explored views informed consent participation clinical trials stroke gathering feedback trial recruitment materials 54 potential study participants findings guided development recruitment informed consent procedures resulted ethically appropriate trial designconducting rigorous qualitative researchthere substantial body literature defining principles practices scientifically sound qualitative mixed methods health services research92327294448 qualitative methods involve systematic collection organization interpretation data accordance rigorous widely accepted techniques research strategy sampling data collection analysisresearch strategyprimary qualitative research strategies include grounded theory ethnography case study phenomenology949 approach uniquely suited specific types investigations choice design determined overarching aim study235051grounded theory uses systematic procedures generate theory insights describing phenomenon grounded views expressed study participants92749 based sociology inductive nature approach involves iterative process data collection analysis grounded theory relies theoretical sampling sample developed refined throughout course study explore insights emerge data27 analysis grounded theory studies typically guided constant comparative method verbatim quotations observations catalogued essential concepts use codes labels developed iteratively reflect dataethnography form field research seeks learn culture particular setting environment52 often relies participant observation prolonged field work may include qualitative quantitative methods53 researcher becomes embedded ongoing relationships research participants purpose observing recording talk behavior cases researcher opposed instance surveys questionnaires primary instrument data collection analysis5254 researcher seeks place specific events broader meaningful context focus culture social interaction observed people groups55 ethnography particularly valuable understanding influence social cultural norms effectiveness health interventions5253case studies involve investigation specific unique system patterned behavior dynamic properties defined features5658 designs may involve single indepth case study set several case studies depending scope goals research project case may defined clinical case critical incident activity process family organization system case study design useful individual case appears produce new insights might transferred larger group cases activities aim case study may exploratory define questions hypotheses descriptive depict phenomenon within context explanatory identify causeandeffect relationships nature59 quality case study enhanced use diverse data sources including documentation archival records interviews direct observation participantobservation notes physical artifacts56phenomenology seeks describe individuals experience specific phenomenon rooted philosophy approach characterizes individuals lived experiences phenomenon grief gathering extensive narrative data small number participants goal generate deeper understanding essence meaning particular phenomenon individuals perspective9an additional research approach may use qualitative methods independently combination quantitative methods communitybased participatory action research60 topic addressed depth forthcoming article outcomes research seriessampling strategy sizesystematic scientifically sound methods developing samples qualitative mixedmethods studies well established94461 contrast quantitative sampling techniques rely statistical probability theory qualitative sampling based purposeful theoretical sampling principles aim identify informationrich participants certain characteristics detailed knowledge direct experience relevant phenomenon interest26 common approach known purposeful sampling seeks include full spectrum cases reflect diversity within given population including extreme negative cases9 sample size varies depending breadth complexity inquiry although samples generally smaller used quantitative studies studied intensively adequacy sample size determined principle theoretical saturation theoretical saturation refers point new concepts emerge review successive data sample diverse pertinent characteristics experiences274562 although possible define number participants advance range 20 30 interviews9 4 6 focus groups63 may achieve saturationdata collection methodsprimary methods qualitative data collection include indepth interviews focus groups observation document review table 2 table 2 qualitative data collection methodsapproachapplicationpurposeindepth interview discussion researchers participant driven participantexplore individual experiences perceptions rich detailfocus group guided discussions among group people share common characteristic interestgenerate unique insights shared experiences social normsobservation systematic detailed observation people events learn behaviors interactions natural settingslearn behaviors interactions natural settings examine situations processes typically hidden public study cultural aspects setting phenomenondocument review objective systematic analysis written communication categorize classify essential conceptsidentify patterns communication analyze traits individuals describe characteristics organizations processes make inferences antecedents effects communicationindepth interviewsan indepth interview typically 1to1 interaction researcher study participant interviews allow exploration individual experiences perceptions great detail95464 interviews particularly useful rapport researchers respondents required ensure candor instances privacy may alleviate fear reprisal negative statements program evaluation patient satisfaction studies although interviews typically conducted either person via telephone electronic online formats emerging method particularly useful isolated difficulttoreach study participants65indepth interviews guided conversations allow respondent direct course discussion much possible66 format allows respondent identify describe concerns concepts may anticipated considered researchers interviews highly interactive interviewer aims responsive language concepts used interviewee data collection instruments often referred discussion guides may semistructured using openended questions within predetermined set topics indepth 1 2 broad questions explore issue experience great detail design addition established questions interviewer uses prompts probes clarify concepts elicit detail extend narrative67these interview methods require highly experienced researchers specific skills including ability establish rapport respondents use discussion guides flexibly use probes followup questions draw responses interviewers must skilled passive listening use neutral nonjudgmental language encouraging respondents speak detail time interviewers must maintain control data gathering vigilant attention purpose interview asking right questions obtain relevant information giving appropriate verbal nonverbal feedback9 staff diverse backgrounds conduct highquality interviews provided adequate training methodsfocus groupsfocus groups guided discussions among small group people share common characteristic central topic interest466368 group interaction serve catalyst generate unique insights understanding shared experiences social norms focus groups appropriate goal understand differences perspectives groups categories people uncover factors influence opinions behavior68 may especially effective socially marginalized populations useful facilitating comfort among members discussing potentially sensitive intimate topics69 focus group discussions conducted person via telephone electronic online formats also emerging effective tool particularly adolescents70 decision whether use focus groups individual interviews dependent format likely encourage participants speak greatest candor privacy individual interviews versus reassurance group shared experience nature data desired eg individual experiences versus social norms feasibility concerns logistics resources research teamfocus groups rely interactions among group members widen range responses activate forgotten details individual experiences release inhibitions otherwise discourage participants disclosing information46 depending depth complexity topic saturation may occur 4 6 focus groups63 discussion guides generally consist 5 10 openended questions designed stimulate discussion central topics interest68 focus group moderators must adept guiding dialogue permitting free exchange ensuring participants feel comfortable expressing discordant views encouraging members describe compare experiences opinions group members uncover degree consensus diversity topic46 number important considerations construction groups whether group comprise novices experts strangers acquaintances factors may negatively influence group dynamic race gender income power differentials68 guiding criterion ensure group composition inhibit members speaking discussion generates comprehensive information reflects full spectrum opinions experiencesobservationobservational data collection involves systematic detailed observation people events learn behaviors interactions natural settings26 study designs useful study goal understand cultural aspects setting phenomenon71 situation interest hidden public setting appear notably different views outsiders observational data collection accomplished extended period interaction researcher study participants participants milieu72 researcher may participant nonparticipant observer participant observers known study subjects researchers although role nonparticipant observers made clear setting notable challenge latter method hawthorn effect degree participants may alter talk behavior watched although covert observation participants unaware researchers role may minimize effect raises ethical questions therefore rare health research data consist extensive detailed observational field notes researcher collects systematically unobtrusively notes may include verbatim paraphrased commentary observations environment researcher reflectionsdocument reviewa wide variety written materials may serve valuable source data documents include limited institutional documents clinical programmatic organizational records personal documents diaries letters artistic expressions public historical documents legislative testimony legal documents9 one method systematic document review content analysis strategy generates inferences objective systematic identification core elements written communication73 content analysis involves categorization classification data make inferences antecedents communication describe make inferences characteristics communication make inferences effects communication73data analysisunlike quantitative studies data collection generally completed data analysis begins qualitative data collection analysis occur iterative fashion research team moves back forth data collection data analysis processes allow new avenues inquiry develop additional data collected2374a commonly used analytic approach constant comparative method274445 form analysis data reviewed line line detail concept becomes apparent code assigned segment document entire document codes tags44 labels help catalogue key concepts preserving context concepts occur ascertain whether code assigned appropriately analyst compares text segments segments assigned code previously decides whether reflect concept using constantcomparison method researchers refine dimensions existing codes identify new codes process code structure evolves inductively accordance analytic principles grounded theory2745qualitative data collection generates substantial amount data instance 20 onehour indepth interviews may generate 400 singlespaced pages transcripts researchers often use computer software facilitate qualitative data analysis c