Table 4: Summary of methods to assess provider knowledge and delivery of care. Method Strengths Weaknesses Reliability Feasibility Relative Costs Provider knowledge Provider Measures practical knowledge; Hawthorne effect – Altered Good for provider perceptions Can be done in large Low interviews – measures provider perceptions behavior or performance of the and knowledge (but many poor sample sizes, wide knowledge, health worker resulting from questions are used) variety of cases, attitudes awareness of being part of an including rare events experimental study; Does not measure actual practice Provider Measures practical knowledge, Hawthorne effect – Altered Variability based on Depends on high Higher interviewer vignettes decision-making behavior or performance of the interviewer; can have low quality interviewer, qualifications and health worker resulting from reliability (though may be good uses lower sample training required awareness of being part of an in certain circumstances) size and limited experimental study; Does not variation in cases, measure actual practice some rare events Health service delivery Observation Measures actual quality delivered Hawthorne effect – Altered Good -- Limited inter-rater Requires high case Modest training of care behavior or performance of the problems, but not well load &/or common (e.g. 1 week) of health worker resulting from documented conditions; more literate or student awareness of being part of an intrusive than other health workers; experimental study; Measures methods supervision costs optimum care delivered Exit Interview Lower Hawthorne effect than Does not measure actual sequence Poor patient/caretaker recall: Can do large Modest training – History observation if provider can be of physical exam low correlations with actual numbers, but better (e.g. 1 week) of taking and blinded to study history documented for common literate or student physical exam conditions health workers; supervision costs
Table 4: Summary of methods to assess provider knowledge and delivery of care. Method Strengths Weaknesses Reliability Feasibility Relative Costs Provider knowledge Provider interviews – knowledge, attitudes Measures practical knowledge; measures provider perceptions Hawthorne effect – Altered behavior or performance of the health worker resulting from awareness of being part of an experimental study; Does not measure actual practice Good for provider perceptions and knowledge (but many poor questions are used) Can be done in large sample sizes, wide variety of cases, including rare events Low Provider vignettes Measures practical knowledge, decision-making Hawthorne effect – Altered behavior or performance of the health worker resulting from awareness of being part of an experimental study; Does not measure actual practice Variability based on interviewer; can have low reliability (though may be good in certain circumstances) Depends on high quality interviewer, uses lower sample size and limited variation in cases, some rare events Higher interviewer qualifications and training required Health service delivery Observation of care Measures actual quality delivered Hawthorne effect – Altered behavior or performance of the health worker resulting from awareness of being part of an experimental study; Measures optimum care delivered Good -- Limited inter-rater problems, but not well documented Requires high case load &/or common conditions; more intrusive than other methods Modest training (e.g. 1 week) of literate or student health workers; supervision costs Exit Interview – History taking and physical exam Lower Hawthorne effect than observation if provider can be blinded to study Does not measure actual sequence of physical exam Poor patient/caretaker recall: low correlations with actual history documented Can do large numbers, but better for common conditions Modest training (e.g. 1 week) of literate or student health workers; supervision costs
3.01a_RBF_Indicators.2012.06.13.docx