***TASK*** the task is to summarize an input biomedical literature in six sentences 
    ***INPUT*** the input is a biomedical literature 
    ***OUTPUT*** the output is the summary of an input biomedical literature in six sentences 
    ***DOCUMENTATION*** 
    ***EXAMPLES*** 
    Input: OBJECTIVE : Measure the effect of specialty society-developed continuing medical education ( CME ) on clinical decision making . DESIGN : R and omized controlled trial . SETTING : National sample of neurologists . PARTICIPANTS : Of 492 neurologists r and omly selected from an ongoing American Academy of Neurology CME program , 248 were r and omized to receive a mailed CME course , and 244 did not receive it . INTERVENTION : A mailed educational course on movement disorders , developed by the specialty society , containing information on diseases and practice recommendations with illustrative case presentations . MEASUREMENTS AND MAIN RESULTS : We assessed adherence to 16 practice recommendations on disease detection , diagnostic test use , and treatments by mailed survey sent to all subjects 4.5 months after the intervention group received the course ( 73 % response rate ) . The survey contained detailed clinical scenarios to measure self-reported clinical decision making and short open-ended questions to measure factual knowledge . More intervention participants ( up to 2.6 times more ) than control subjects reported clinical decision making adherent to 9 of the 16 recommendations ( p<.05 ) . For 4 of the other 7 recommendations , adherence exceeded 85 % in both groups . Within the intervention group , neurologists who read the educational course were 2 to 6 times more likely to be adherent than neurologists who did not . The intervention group had better factual knowledge than control subjects in six of seven areas ( p<.01 ) . CONCLUSIONS : This educational course improved neurologists ’ reported decision making . Specialty society-developed CME that utilizes a similar format may enhance the effectiveness of mailed CME information to improve physicians ’ approach to clinical decisions CONTEXT All assessment data , like other scientific experimental data , must be reproducible in order to be meaningfully interpreted . PURPOSE The purpose of this paper is to discuss applications of reliability to the most common assessment methods in medical education . Typical methods of estimating reliability are discussed intuitively and non-mathematically . SUMMARY Reliability refers to the consistency of assessment outcomes . The exact type of consistency of greatest interest depends on the type of assessment , its purpose and the consequential use of the data . Written tests of cognitive achievement look to internal test consistency , using estimation methods derived from the test-retest design . Rater-based assessment data , such as ratings of clinical performance on the wards , require interrater consistency or agreement . Objective structured clinical examinations , simulated patient examinations and other performance-type assessment s generally require generalisability theory analysis to account for various sources of measurement error in complex design s and to estimate the consistency of the generalisations to a universe or domain of skills . CONCLUSIONS Reliability is a major source of validity evidence for assessment s. Low reliability indicates that large variations in scores can be expected upon retesting . Inconsistent assessment scores are difficult or impossible to interpret meaningfully and thus reduce validity evidence . Reliability coefficients allow the quantification and estimation of the r and om errors of measurement in assessment s , such that overall assessment can be improved Abstract OBJECTIVE : We evaluated the effect of implementing quality improvement ( QI ) programs for depression , relative to usual care , on primary care clinicians ’ knowledge about treatment . DESIGN AND METHODS : Matched primary care clinics ( 46 ) from seven managed care organizations were r and omized to usual care  ( mailed written guidelines only ) versus  one of two QI interventions . Self-report surveys assessed clinicians ’ knowledge of depression treatments prior to full implementation ( June 1996 to March 1997 ) and 18 months later . We used an intent-to-treat analysis to examine intervention effects on change in knowledge , controlling for clinician and practice characteristics , and the nested design . PARTICIPANTS : One hundred eighty-one primary care clinicians . INTERVENTIONS : The interventions included institutional commitment to QI , training local experts , clinician education , and training nurses for patient assessment and education . One intervention had re sources for nurse follow-up on medication use ( QI-meds ) and the other had reduced copayment for therapy from trained , local therapists ( QI-therapy ) . RESULTS : Clinicians in the intervention group had greater increases compared with clinicians in the usual care group over 18 months in knowledge of psychotherapy ( by 20 % for QI-meds , P=.04 and by 33 % for QI-therapy , P=.004 ) , but there were no significant increases in medication knowledge . Significant increases in knowledge scores ( P=.01 ) were demonstrated by QI-therapy clinicians but not clinicians in the QI-meds group . Clinicians were exposed to multiple intervention components . CONCLUSIONS : Dissemination of QI programs for depression in managed , primary care practice s improved clinicians ’ treatment knowledge over 18 months , but breadth of learning was somewhat greater for a program that also included active collaboration with local therapists The purpose of the study was to compare the impact of continuing medical education for primary care physicians in Halifax , Nova Scotia , Canada , delivered through a problem-based learning ( PBL ) format with that of a lecture-based format , in the clinical area of headache diagnosis and management . From January to June 1995 , 38 physicians participated in three problem-based learning sessions , and 49 in a lecture-based session in the clinical area of headache . Pre- and posttest assessment s of knowledge were made of each group before and immediately after the educational sessions . A second evaluation using Key Features Problems ( KFP ) to measure clinical reasoning was administered to both groups 3 months later . Analysis of covariance between groups on the post-test , using the pre-test result as covariate , showed significantly greater knowledge in the PBL group . Mean post-test KFP scores also were significantly higher for the PBL group . The PBL group 's satisfaction with several programme dimensions was significantly higher than that of the lecture group . These results must be viewed in the context of differing amounts of exposure for the two groups . Problem-based learning in continuing medical education in the area of headache management was associated with greater knowledge acquisition and with greater improvement in clinical reasoning skills than in a lecture-based approach . Problem-based learning also was preferred by family physicians . However , the cost-benefit of this approach was question ed , since the PBL group had more exposure . Several factors limit the generalizability of this study , e.g. the impossibility of r and omly composing the two experimental groups . Further research is required to determine if the results are generalizable and whether a PBL approach results in change in practice behaviour CONTEXT Despite evidence that a variety of continuing medical education ( CME ) techniques can foster physician behavioral change , there have been no r and omized trials comparing performance outcomes for physicians participating in Internet-based CME with physicians participating in a live CME intervention  using approaches documented to be effective . OBJECTIVE To determine if Internet-based CME can produce changes comparable to those produced via live , small-group , interactive CME with respect to physician knowledge and behaviors that have an impact on patient care . DESIGN , SETTING , AND PARTICIPANTS R and omized controlled trial conducted from August 2001 to July 2002 . Participants were 97 primary care physicians drawn from 21 practice sites in Houston , Tex , including 7 community health centers and 14 private group practice s. A control group of 18 physicians from these same sites received no intervention .  INTERVENTIONS Physicians were r and omly assigned to an Internet-based CME intervention that could be completed in multiple sessions over 2 weeks , or to a single live , small-group , interactive CME workshop . Both incorporated similar multifaceted instructional approaches demonstrated to be effective in live setting s. Content was based on the National Institutes of Health National Cholesterol Education Program -- Adult Treatment Panel III guidelines . MAIN OUTCOME MEASURES Knowledge was assessed immediately before the intervention , immediately after the intervention , and 12 weeks later . The percentage of high-risk patients who had appropriate lipid panel screening and pharmacotherapeutic treatment according to guidelines was documented with chart audits conducted over a 5-month period before intervention and a 5-month period after intervention . RESULTS Both interventions produced similar and significant immediate and 12-week knowledge gains , representing large increases in percentage of items correct ( pretest to posttest : 31.0 % [ 95 % confidence interval { CI } , 27.0%-35.0 % ] ; pretest to 12 weeks : 36.4 % [ 95 % CI , 32.2%-40.6 % ] ; P<.001 for all comparisons ) . Chart audits revealed high baseline screening rates in all study groups ( > or = 93 % ) with no significant postintervention change . However , the Internet-based intervention was associated with a significant increase in the percentage of high-risk patients treated with pharmacotherapeutics according to guidelines ( preintervention , 85.3 % ; postintervention , 90.3 % ; P = .04 ) . CONCLUSIONS Appropriately design ed , evidence -based online CME can produce objective ly measured changes in behavior as well as sustained gains in knowledge that are comparable or superior to those realized from effective live activities BACKGROUND Few primary care physicians routinely counsel for exercise , despite the benefits of physical activity and the high prevalence of inactivity . The objective of this study is to assess the effectiveness of Physician-Based Assessment and Counseling for Exercise ( PACE ) , a brief , behavior-based tool for primary care providers counseling healthy adults . METHODS This study is a r and omized controlled trial of 812 patients age 30 years or older registered for well visits at 32 primary care physician offices at a staff model health maintenance organization . Intervention physicians were trained to deliver PACE exercise counseling protocol s at the index visit , and one reminder telephone call occurred at 1 month . An enhanced intervention group received additional activity reminders . RESULTS At the 6-month follow-up , the control group did not differ significantly from the intervention group for energy expended ( 2,048 kcal/week versus 2,108 kcal/ week , P = 0.77 ) , time spent in walking or other moderate to vigorous activities ( 202 min/week versus 187 min/ week , P = 0.99 ) , mental health , physical function , or behaviors previously shown to predict activity change . Among the intervention patients , the stages-of-change score for Contemplators increased significantly compared with controls ( P = 0.03 ) , but without a significant change in energy expended . Baseline levels of physical activity counseling were high ( 50 % ) , as were baseline patient physical activity levels ( 61 % exercised at least three times a week ) . CONCLUSIONS These results suggest that a one-time PACE counseling session with minimal reinforcement , in a setting with high baseline levels of activity , does not further increase activity . The finding that Contemplators advanced in stage of behavior change suggests that further studies are needed to examine long-term , repeated counseling interventions This research examined the malleability of physicians ' knowledge about psychiatric disorders ( and their recommended treatments ) based on a brief educational intervention .  Primary care physicians were r and omly assigned to a single-session intervention dealing with selected DSM-IIIR affective and anxiety disorders or to a wait-list control group . Posttest ( vs. pretest ) measures indicated that experimental physicians showed greater increases than controls in diagnostic accuracy on relevant disorders , specifically , major depression , dysthymic disorder , and agoraphobia with panic attacks . Treatment recommendations were also influenced , specifically , for major depression and panic disorder . Interestingly , treatment recommendations were also influenced for certain somatic disorders not addressed in the intervention , suggesting " halo " effects for these disorders . Overall , the intervention was effective in increasing diagnostic knowledge and altering treatment recommendations . Implication s for research on psychiatric interventions among physicians are discussed A multifaceted , individualized , physician education program design ed to increase the breast cancer screening practice s of community-based primary care physicians is described and the results are evaluated . Community-based surveys identified primary care providers with breast cancer screening educational needs who were assigned , using a factorial design , to an intervention or control condition . The sample included 154 control and 128 intervention physicians . The intervention consisted of a 1 - 2h in-office training program and /or self- study workbook .  Self-reported overall breast cancer screening need scores improved for a greater proportion of intervention than control physicians , particularly those receiving the in-office intervention ( P=0.03 ) . Clinical breast examination ( CBE ) need declined ( P=0.01 ) ; use of provider reminder systems increased ( P=0.02 ) ; preparedness to counsel about CBE ( P=0.04 ) and recognition that age is an important risk factor for breast cancer ( P=0.02 ) improved in more intervention compared to control physicians BACKGROUND Educational interventions that support the implementation of complex clinical practice guidelines ( CPGs ) require substantial time commitments from participants . We conducted a comparative study to evaluate if a 90-minute workshop would increase compliance with the recommendations of the Canadian Task Force on Preventive Health Care as well as decrease the ordering of tests not the subject of specific recommendations . METHODS Eighty-seven family physicians from Quebec participated in the study . Group assignment was initially r and omized , but , owing to logistic problems , r and omization was not maintained . After unannounced visits , 2 st and ardized patients coded the physicians ' performance of 23 items recommended for inclusion in the periodic health examination ( 10 for men and 13 for women ) and 8 items recommended for exclusion ( 4 for both men and women ) . The " exposed " physicians were visited within 4 to 6 months after the workshop . The " nonexposed " physicians were visited within 4 to 6 months after consent was obtained but before they attended the workshop . We used linear regression analysis to determine if exposure to the workshop result ed in improved performance . RESULTS Exposure to the workshop was not associated with a difference in the adjusted mean score for items recommended for inclusion ( 12.07 for exposed 
    Output:
    
When reported , statistical tests yielded modest evidence of validity and reliability .
Translated to the contemporary classification approach , our data indicate that reporting about internal structure validity exceeded reporting about other categories of validity evidence .
The evidence for CME effectiveness is limited by weaknesses in the reported validity and reliability of evaluation methods .