in 1999 , the institute of medicine ( iom ) reported that nearly 100,000 lives and approximately $ 25 billion are lost each year due to medical errors .
the iom concluded that most such errors could be reduced through changes in the healthcare system , including the implementation of team training programs . as a result
, there is a great need for valid and reliable training programs targeted towards teamwork in medical tasks . despite the great potential that teamwork training has for improving team functioning
, few studies have provided empirical assessments of teamwork training programs for medical teams.[37 ] much of the existing work has focused on traditional , lecture - based training with moderate results.[811 ] a number of training programs have been developed for applications in healthcare organizations ( eg , teamstepps , acrm ) . although case reports of these training programs are generally positive , empirical evaluations on the effects of such programs have been scarce .
thus , there is much to learn about the optimal training design , delivery , and evaluation approaches to teamwork training in medicine .
the current study provides an evaluation of a teamwork training program for residents who are beginning to work in a trauma room .
the training used multiple methods , including low - fidelity role play , lecture , and high - fidelity simulation in order to maximize the impact of training and increase the chances of transfer of training to the work setting .
it was hypothesized that the residents would demonstrate positive reactions to the training , exhibit more targeted behaviors after training than before as measured by the situational judgment test ( sjt ) , and exhibit more targeted behaviors after training than would comparable control residents .
this study was conducted at southeastern american level i trauma center that is a university affiliated teaching hospital .
a total of 41 first - year interns ( 10 emergency medicine and 31 surgery ) were recruited to participate in the study .
twenty interns took classes on day one and 21 on day two so as to keep the class size manageable .
participating interns had completed and passed the advanced trauma life support ( atls ) course .
the sample was relatively representative demographically , with 18 women sounds better and 9 asian , african - american , or hispanic residents .
participants completed a survey on a demographics form and a pre - test situational judgment test ( described subsequently ) .
the first exercise was a role play designed to serve as an icebreaker and demonstrate ineffective teamwork .
the exercise featured a science fiction scenario in which all trainees received a loosely scripted personalities and instructions on their contribution to the team while adopting the scripted personality .
the team was given a problem to solve , and the scripted personalities were designed to cause breakdowns in teamwork , thus illustrating the difficulty of problem - solving without effective teamwork . after the role play , participants were debriefed on the personalities being played and the real purpose of the exercise .
thereafter , the trainers guided participant discussion on the dynamics that led to poor teamwork and poor task performance .
the next exercise was a wilderness survival exercise designed to illustrate how groups may outperform individuals .
participants must choose the best answer to multiple choice questions about surviving in the wilderness , first individually , and then within groups .
typically , the group score was better than the individual score , which demonstrated to the participants that working in groups may benefit them .
after these first two role plays , residents were directed to consider links between the role plays and teamwork in the trauma room . following these exercises ,
teamwork concepts were explicitly provided to participants in the form of a brief lecture given by a psychologist and an attending physician .
the lecture was meant to complement the exercises by giving clear information about the recently demonstrated concepts in the context of medical teamwork including error framing and management techniques , learning from mistakes , and managing a problematic team member .
the lecture recognized the unique role of the first - year residents and the challenges before such residents in handling resuscitations in the trauma room .
after the lecture , 5 - 6 participants were assigned as teams to complete emergency medical tasks within a high - fidelity medical simulation .
the scenarios were developed by the researchers and an expert physician in trauma resuscitation who was affiliated with the teaching hospital .
along with the participants , a confederate ( recruited from nursing staff ) was instructed to cause a specific issue in teamwork .
a total of four scenarios were run in order to allow participation of all trainees .
the confederates used were a respiratory therapist , an emergency department ( ed ) nurse , and an orthopedic resident was played by a senior ed resident and a senior surgical resident as a trauma surgeon who were not otherwise involved in the study .
the confederate thus provided opportunities for the residents to deal with common teamwork issues of leadership , deviations from standard procedures , prioritizing tasks , and dealing with interpersonal distractions .
the residents were assigned medical roles but knew neither the details of the patient nor the problem(s ) to be presented by the confederate prior to running the scenarios .
these scenarios were also recorded and broadcast live to a viewing room , so that trainees not currently participating in the scenario could watch and learn from the role play . to facilitate feedback and guided discussion ,
trainers provided comments on specific issues and critical incidents and also asked questions to the participants in order to encourage an interactive discussion .
ineffective behaviors were highlighted , and trainers were then asked to suggest more effective behaviors .
the course concluded with a course summary and administration of a survey measuring reactions to the training and a second administration of the situational judgment test . prior to training
this questionnaire also asked whether the participant had been involved in any prior teamwork training programs ( none had ) . because trainee interest in training often relates to training success ,
given the practical difficulty of measuring trainee behaviors during later medical emergencies , the training was validated using a sjt for behavioral responses to teamwork issues in medical emergency tasks .
sjts on teamwork skills provide valuable insight on trainee knowledge for appropriate behavior , and they have produced strong relations to performance . although sjts are not commonly used in medical educational research , there are no validated measures for assessment of individual behavioral responses to teamwork issues in the trauma room ; moreover , sjt can assess behavioral choices , and not just attitudes or values .
therefore , subject matter experts ( smes ) constructed items that reflected previous or potential issues in trauma room teamwork .
the items asked trainees to choose behavioral responses to critical teamwork incidents in the trauma room .
the items reflected situations that were analogous , but not identical , to medical scenarios used in the training scenarios , such as how to deal with an ineffective leader , inappropriate comments by a team member , or procedural errors that need correction .
trainees and a set of four smes chose , for each item , their most likely and least likely response out of total of four options .
the smes included attending physicians , two specialized in general trauma resuscitation and one in neonatal resuscitation , along with the last one as an attending emergency physician .
the sjt was scored by giving a point for every response that reflected the modal expert response .
five items were only partially scored while computing scale totals , as experts provided more than one mode for either the most or least likely response .
thus , the potential range of scores on the situational judgment test was 0 - 25 .
because categorical variables were used , the kappa coefficient was used to calculate inter - rater agreement ( 0.68 ) .
this coefficient corrects for chance agreement , resulting in a lower ( but more accurate ) index of true rater agreement than would be produced by simply measuring the proportion of item agreement among sme ratings .
trainee utility reactions were measured using a scale that asked participants about the degree to which they liked the training and the degree to which they believed the training would be useful . because these questions are training specific , and reaction measures were not assessed until after the team had completed the training .
residents completed study measures before and after the training module for two separate training days .
therefore , situational judgment test scores could be compared within groups pre - test to post - test , and each group 's pretest scores could serve as the control group for the other group 's post - test scores .
reactions to training could not be compared to a control group , but if training is poorly received , then it is likely to be rejected , so it is important to assess and report trainee reactions .
paired and independent t - tests were computed to assess mean differences in situational judgment test total scores .
independent t - tests were planned contrasts between pre - training measures for the first day and post - training measures for the second day , and vice versa .
paired t - tests were used to test whether situational judgment test scores improved from pre - test to post - test within groups .
thus , four different t - tests were computed , two between groups and two within groups . to better understand any training effects ,
the responses to individual items were also analyzed in detail using frequency tables and chi - square goodness - of - fit tests .
the goodness - of - fit test treated the observed frequencies from pre - training sjt item responses as the expected frequencies for post - training sjt responses .
a chi - square then estimated the degree of fit between the observed post - training responses to the expected frequencies for each response option ( most and least likely for each item ) .
a significant chi - square suggested a significant effect of training on item responses , based on shifts in the frequency distribution .
participants completed a survey on a demographics form and a pre - test situational judgment test ( described subsequently ) .
the first exercise was a role play designed to serve as an icebreaker and demonstrate ineffective teamwork .
the exercise featured a science fiction scenario in which all trainees received a loosely scripted personalities and instructions on their contribution to the team while adopting the scripted personality .
the team was given a problem to solve , and the scripted personalities were designed to cause breakdowns in teamwork , thus illustrating the difficulty of problem - solving without effective teamwork . after the role play , participants were debriefed on the personalities being played and the real purpose of the exercise . thereafter , the trainers guided participant discussion on the dynamics that led to poor teamwork and poor task performance .
the next exercise was a wilderness survival exercise designed to illustrate how groups may outperform individuals .
participants must choose the best answer to multiple choice questions about surviving in the wilderness , first individually , and then within groups .
typically , the group score was better than the individual score , which demonstrated to the participants that working in groups may benefit them .
after these first two role plays , residents were directed to consider links between the role plays and teamwork in the trauma room . following these exercises ,
teamwork concepts were explicitly provided to participants in the form of a brief lecture given by a psychologist and an attending physician . the lecture was meant to complement the exercises by giving clear information about the recently demonstrated concepts in the context of medical teamwork including error framing and management techniques , learning from mistakes , and managing a problematic team member .
the lecture recognized the unique role of the first - year residents and the challenges before such residents in handling resuscitations in the trauma room .
after the lecture , 5 - 6 participants were assigned as teams to complete emergency medical tasks within a high - fidelity medical simulation .
the scenarios were developed by the researchers and an expert physician in trauma resuscitation who was affiliated with the teaching hospital .
along with the participants , a confederate ( recruited from nursing staff ) was instructed to cause a specific issue in teamwork .
a total of four scenarios were run in order to allow participation of all trainees .
the confederates used were a respiratory therapist , an emergency department ( ed ) nurse , and an orthopedic resident was played by a senior ed resident and a senior surgical resident as a trauma surgeon who were not otherwise involved in the study .
the confederate thus provided opportunities for the residents to deal with common teamwork issues of leadership , deviations from standard procedures , prioritizing tasks , and dealing with interpersonal distractions .
the residents were assigned medical roles but knew neither the details of the patient nor the problem(s ) to be presented by the confederate prior to running the scenarios .
these scenarios were also recorded and broadcast live to a viewing room , so that trainees not currently participating in the scenario could watch and learn from the role play . to facilitate feedback and guided discussion ,
trainers provided comments on specific issues and critical incidents and also asked questions to the participants in order to encourage an interactive discussion .
ineffective behaviors were highlighted , and trainers were then asked to suggest more effective behaviors .
the course concluded with a course summary and administration of a survey measuring reactions to the training and a second administration of the situational judgment test .
this questionnaire also asked whether the participant had been involved in any prior teamwork training programs ( none had ) . because trainee interest in training often relates to training success ,
given the practical difficulty of measuring trainee behaviors during later medical emergencies , the training was validated using a sjt for behavioral responses to teamwork issues in medical emergency tasks .
sjts on teamwork skills provide valuable insight on trainee knowledge for appropriate behavior , and they have produced strong relations to performance . although sjts are not commonly used in medical educational research , there are no validated measures for assessment of individual behavioral responses to teamwork issues in the trauma room ; moreover , sjt can assess behavioral choices , and not just attitudes or values .
therefore , subject matter experts ( smes ) constructed items that reflected previous or potential issues in trauma room teamwork .
the items asked trainees to choose behavioral responses to critical teamwork incidents in the trauma room .
the items reflected situations that were analogous , but not identical , to medical scenarios used in the training scenarios , such as how to deal with an ineffective leader , inappropriate comments by a team member , or procedural errors that need correction .
trainees and a set of four smes chose , for each item , their most likely and least likely response out of total of four options .
the smes included attending physicians , two specialized in general trauma resuscitation and one in neonatal resuscitation , along with the last one as an attending emergency physician .
the sjt was scored by giving a point for every response that reflected the modal expert response .
five items were only partially scored while computing scale totals , as experts provided more than one mode for either the most or least likely response .
thus , the potential range of scores on the situational judgment test was 0 - 25 .
because categorical variables were used , the kappa coefficient was used to calculate inter - rater agreement ( 0.68 ) .
this coefficient corrects for chance agreement , resulting in a lower ( but more accurate ) index of true rater agreement than would be produced by simply measuring the proportion of item agreement among sme ratings .
trainee utility reactions were measured using a scale that asked participants about the degree to which they liked the training and the degree to which they believed the training would be useful . because these questions are training specific , and reaction measures were not assessed until after the team had completed the training .
this questionnaire also asked whether the participant had been involved in any prior teamwork training programs ( none had ) . because trainee interest in training often relates to training success ,
given the practical difficulty of measuring trainee behaviors during later medical emergencies , the training was validated using a sjt for behavioral responses to teamwork issues in medical emergency tasks .
sjts on teamwork skills provide valuable insight on trainee knowledge for appropriate behavior , and they have produced strong relations to performance . although sjts are not commonly used in medical educational research , there are no validated measures for assessment of individual behavioral responses to teamwork issues in the trauma room ; moreover , sjt can assess behavioral choices , and not just attitudes or values . therefore , subject matter experts ( smes ) constructed items that reflected previous or potential issues in trauma room teamwork .
the items asked trainees to choose behavioral responses to critical teamwork incidents in the trauma room .
the items reflected situations that were analogous , but not identical , to medical scenarios used in the training scenarios , such as how to deal with an ineffective leader , inappropriate comments by a team member , or procedural errors that need correction .
trainees and a set of four smes chose , for each item , their most likely and least likely response out of total of four options .
the smes included attending physicians , two specialized in general trauma resuscitation and one in neonatal resuscitation , along with the last one as an attending emergency physician .
the sjt was scored by giving a point for every response that reflected the modal expert response .
five items were only partially scored while computing scale totals , as experts provided more than one mode for either the most or least likely response .
thus , the potential range of scores on the situational judgment test was 0 - 25 .
because categorical variables were used , the kappa coefficient was used to calculate inter - rater agreement ( 0.68 ) .
this coefficient corrects for chance agreement , resulting in a lower ( but more accurate ) index of true rater agreement than would be produced by simply measuring the proportion of item agreement among sme ratings .
trainee utility reactions were measured using a scale that asked participants about the degree to which they liked the training and the degree to which they believed the training would be useful . because these questions are training specific , and reaction measures were not assessed until after the team had completed the training .
residents completed study measures before and after the training module for two separate training days .
trainees were assigned randomly to training days . therefore , situational judgment test scores could be compared within groups pre - test to post - test , and each group 's pretest scores could serve as the control group for the other group 's post - test scores .
reactions to training could not be compared to a control group , but if training is poorly received , then it is likely to be rejected , so it is important to assess and report trainee reactions .
paired and independent t - tests were computed to assess mean differences in situational judgment test total scores .
independent t - tests were planned contrasts between pre - training measures for the first day and post - training measures for the second day , and vice versa .
paired t - tests were used to test whether situational judgment test scores improved from pre - test to post - test within groups .
thus , four different t - tests were computed , two between groups and two within groups . to better understand any training effects ,
the responses to individual items were also analyzed in detail using frequency tables and chi - square goodness - of - fit tests .
the goodness - of - fit test treated the observed frequencies from pre - training sjt item responses as the expected frequencies for post - training sjt responses .
a chi - square then estimated the degree of fit between the observed post - training responses to the expected frequencies for each response option ( most and least likely for each item ) .
a significant chi - square suggested a significant effect of training on item responses , based on shifts in the frequency distribution .
table 1 presents the means , standard deviations , and inter - correlations of continuous variables in the study ( five residents provided missing data ) .
the level of interest in training did not relate to any of the study variables except the reaction measures ; thus , using this variable as a covariate in additional analyses was deemed unnecessary .
that is , performance on the situational judgment test was not related to resident reactions to the training .
the distributions of all continuous variables appeared approximately normal , suggesting that the data met the assumptions for mean comparisons .
descriptive statistics and correlations for study measures situational judgment test scores were compared between groups .
post - training scores of group 1 ( m=17.29 , sd=1.99 ) for the teamwork sjt were significantly higher than pre - training score ( m=13.77 , sd=3.01 , t=4.39 , p<0.01 ) of group 2 , showing a strong effect size ( d=1.38 ) . when post training scores of group 2 ( m=16.55 , sd=1.99 ) were compared to pre - test scores of group 1 ( m=15.63 , sd=4.06 ) , the result was in the expected direction , but not significantly different ( t=0.81 , n.s . ) and the effect size was moderate ( d=0.26 ) .
situational judgment test scores were also compared within groups , pre - test to post - test , once for each group . for group 1
, there was a marginal difference for sjt scores ( t=1.95 , p<0.10 ) and a moderate effect size ( d=0.34 ) .
for group 2 , there was a large mean difference between pre- and post - training ( t=2.78 , p<0.01 ) .
the data suggest a significant effect of training on at least one of the two responses ( most or least likely ) for sjt items , save for two items .
not all shifts following training resulted in higher scores on the situational judgment test ( several items showed shifts from one distractor to another and few items showed shifts away from the scored response ) .
illustrative results for three items ( items 3 , 4 , and 7 ) are presented in which the shift improved scores on sjt in order to better illustrate how the training resulted in improved scores . for item 3 ,
the shift occurred for responses to the least likely response , while for the other two items , the shift occurred for the most likely response .
item 3 presented a situation where the team leader hesitated to make a time - critical decision . after the training , more residents chose  call a team meeting as the action to avoid in that situation than before the training .
although calling for a discussion is clearly a team - oriented response , it is not the best choice during a crisis with limited time to respond . for item 4 ,
the given situation involved people making distracting jokes during the resuscitation , and the appropriate response was to direct attention back to the task at hand . in item 7 , the team leader was fixated upon something and had forgotten the abc protocol . according to experts , the most appropriate response in this situation is to remind the leader of the protocol , and as seen in table 2 , participants were more likely to choose this action following training .
summary results from chi - square goodnessof - fit analyses on sjt items a summary of results from chi - square goodness - of - fit analyses for all sjt items are summarized in table 2 .
the data suggest a significant effect of training on at least one of the two responses ( most or least likely ) for sjt items , save for two items .
not all shifts following training resulted in higher scores on the situational judgment test ( several items showed shifts from one distractor to another and few items showed shifts away from the scored response ) .
illustrative results for three items ( items 3 , 4 , and 7 ) are presented in which the shift improved scores on sjt in order to better illustrate how the training resulted in improved scores . for item 3 ,
the shift occurred for responses to the least likely response , while for the other two items , the shift occurred for the most likely response .
item 3 presented a situation where the team leader hesitated to make a time - critical decision .
after the training , more residents chose  call a team meeting as the action to avoid in that situation than before the training .
although calling for a discussion is clearly a team - oriented response , it is not the best choice during a crisis with limited time to respond . for item 4 ,
the given situation involved people making distracting jokes during the resuscitation , and the appropriate response was to direct attention back to the task at hand . in item 7 ,
the team leader was fixated upon something and had forgotten the abc protocol . according to experts , the most appropriate response in this situation is to remind the leader of the protocol , and as seen in table 2 , participants were more likely to choose this action following training .
summary results from chi - square goodnessof - fit analyses on sjt items a summary of results from chi - square goodness - of - fit analyses for all sjt items are summarized in table 2 .
the purpose of the present study was to validate a teamwork training program for emergency medicine and surgery interns in the trauma room .
teamwork behavioral intentions were assessed using an sjt that presented critical incidents in the trauma teamwork .
we thought this to be important for the educational process to evaluate the individuals teamwork capabilities , as other studies have mainly looked at the outcomes of the training process.[1619 ] although outcomes are important , it is necessary to evaluate how those outcomes were achieved by the team members . in particular
, the immediate purpose of a team training program is to improve the teamwork behaviors in trainees .
thus , we believe that an important question to ask is :  did the training make a team member a better team player ?  behavioral choices to sjt items are one index of whether the training resulted in better team players .
the sjt scale score demonstrated an overall scale effect in three of the four group comparisons planned by the study , and more than half of the items showed a significant or marginal effect in responding from before and after training . in several instances ,
the post- training resident responses matched the expert responses more closely than pre - training responses . even in cases where response shifts did not necessarily match expert ratings , there was an increase in resident assertiveness on item responses .
although situational judgment tests provide information about choices in difficult situations , the scores do not reflect perfect agreement about appropriate behavior by experts .
unfortunately , perfect agreement among experts is likely to be achieved only in simple situations calling for a response that is likely to also be endorsed by novices .
thus , interpretation of the results should be based more on total scores , which represent summaries of tendencies across multiple situations , and less on the scores for any individual item .
examination of individual items may still prove useful ; however , determining which scenarios had the greatest effect on the pre- and post - training mean differences in the sjt total scores .
there are many elements of the training that may explain its positive effect on sjt responses .
an active training approach , such as a role play , should serve as the primary focus for teamwork training .
while some of the previous teamwork studies have attempted to train teamwork using only lecture - based methods , this study saw a positive shift from primary use of role plays , with a complementary lecture .
furthermore , the fidelity of the medical role plays to critical teamwork incidents in the trauma room served as another valuable asset in the training.[1718 ] by having task - related scenarios to practice and observe ineffective and effective behaviors , trainees effectively learned the training content .
the feedback provides a mechanism for trainees to reflect upon previous behaviors and adapt accordingly in future situations .
although the sample of residents was appropriate for the study , there are not many first- year residents in a given year .
therefore , a design containing large numbers in each group and a placebo - training control group was not feasible .
because of the small sample available , the effects of specific training features were not examined in this study ( eg , a no - lecture control group ) .
future training studies may address this issue by comparing different training approaches to a sample of teams .
unlike the majority of literature in teamwork training , this study was not an outcome - based study .
we chose to measure trainee behaviors by their responses to an sjt that presented critical incidents in trauma teamwork .
although this training was designed to measure the behavioral choices of trainees in specific situations , it does not necessarily indicate their actual behaviors on the job .
furthermore , the base rate for critical incidents and teamwork errors in medicine is relatively low , so it may be difficult to obtain an adequate sample of errors to determine whether a reduction in errors was due to a training program .
although the sample of residents was appropriate for the study , there are not many first- year residents in a given year .
therefore , a design containing large numbers in each group and a placebo - training control group was not feasible .
because of the small sample available , the effects of specific training features were not examined in this study ( eg , a no - lecture control group ) .
future training studies may address this issue by comparing different training approaches to a sample of teams .
unlike the majority of literature in teamwork training , this study was not an outcome - based study .
we chose to measure trainee behaviors by their responses to an sjt that presented critical incidents in trauma teamwork .
although this training was designed to measure the behavioral choices of trainees in specific situations , it does not necessarily indicate their actual behaviors on the job .
furthermore , the base rate for critical incidents and teamwork errors in medicine is relatively low , so it may be difficult to obtain an adequate sample of errors to determine whether a reduction in errors was due to a training program .
although teamwork training in the field of medicine is a valuable endeavor , previous studies on teamwork training have shown moderate results by using traditional educational methods ( eg , a lecture or presentation ) and neglecting criteria that are focused on teamwork skills .
the use of more active training methods in addition to a lecture tends to yield a more effective training program .
the current study sought to improve on previous training studies by providing a comprehensive program that featured lecture , role plays , task simulations , and feedback .
results from this study suggested that such a comprehensive training approach can positively affect trainee reactions and behavioral choices for teamwork in the trauma room .
such a result is consistent with the concept that teamwork training will result in improved patient care .
 introduction : poor teamwork leads to preventable medical errors , and thus negatively impacts medical care . 
 one way to improve teamwork is training . 
 a multimodality team training program was designed to impact the attitudes and behavior of first - year residents who will encounter medical situations in the trauma room . 
 the training program included low - fidelity role plays , lectures , and high - fidelity simulation with feedback.materials and methods : the training program was a one - day workshop that was conducted twice , once for each of the two groups over two days at the beginning of the academic year in july . 
 a total of 41 first - year interns ( 10 emergency medicine and 31 surgery ) were recruited for participation . 
 participants completed a situational judgment test ( sjt ) on trauma teamwork before training . 
 the training began with a low - fidelity simulation that served as an icebreaker to team concepts . 
 subsequently , a lecture with discussion provided key points regarding teamwork in the trauma room . 
 a high - fidelity simulation then allowed participation in one of four trauma room scenarios with medical expert debriefing . 
 the course concluded with a course summary and an assessment of participant attitudes regarding training along with a second administration of sjt.results:participant reactions to the training were positive overall . 
 results of sjt showed a positive effect for team training in three of the four possible comparisons.conclusion:the program was well received by the residents . 
 results suggest that a comprehensive training approach using role play , lecture , and simulation can positively affect behavioral choices for teamwork in the trauma room . 
INTRODUCTION
MATERIALS AND METHODS
Training content
Measures
Pre-training questionnaire
Behavioral intentions
Reactions
Statistical analysis
RESULTS
Item analysis for SJT
DISCUSSION
Limitations
CONCLUSION