<clinical_study>
  <!-- This xml conforms to an XML Schema at:
    https://clinicaltrials.gov/ct2/html/images/info/public.xsd -->
  <required_header>
    <download_date>ClinicalTrials.gov processed this data on September 20, 2023</download_date>
    <link_text>Link to the current ClinicalTrials.gov record.</link_text>
    <url>https://clinicaltrials.gov/ct2/show/NCT03229226</url>
  </required_header>
  <id_info>
    <org_study_id>15100805-IRB01-AM01</org_study_id>
    <nct_id>NCT03229226</nct_id>
  </id_info>
  <brief_title>Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations</brief_title>
  <acronym>OPPE16</acronym>
  <official_title>Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations</official_title>
  <sponsors>
    <lead_sponsor>
      <agency>Rush University Medical Center</agency>
      <agency_class>Other</agency_class>
    </lead_sponsor>
  </sponsors>
  <source>Rush University Medical Center</source>
  <oversight_info>
    <has_dmc>Yes</has_dmc>
    <is_fda_regulated_drug>No</is_fda_regulated_drug>
    <is_fda_regulated_device>No</is_fda_regulated_device>
  </oversight_info>
  <brief_summary>
    <textblock>
      Since 2007, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has&#xD;
      required evaluation of practicing attending physicians by management to ensure that&#xD;
      procedural skills and medical care are meeting the accepted standards of care as assessed by&#xD;
      each individual institution. This assessment is known as the ongoing professional practice&#xD;
      evaluation (OPPE). There are several methods by which this can be accomplished, including&#xD;
      through chart review of clinician practice and by verbally assessing knowledge of the steps&#xD;
      required to do a particular procedure. However, for infrequently performed or complicated&#xD;
      procedures, these methods may not allow objective evaluation on a regular basis. Simulation&#xD;
      using task trainers or manikin models offers an alternative method of objective evaluation in&#xD;
      a standardized setting.&#xD;
&#xD;
      The goal of this study is to develop two simulated scenarios to assess physician skill in&#xD;
      relatively uncommon procedures and compare the data obtained against the verbal assessment&#xD;
      and chart review model previously used at our institution. Two raters will assess each&#xD;
      practitioner's verbal and simulated procedures. They will also assess globally their&#xD;
      confidence that the physician is capable of performing the procedure safely and correctly&#xD;
      based on the verbal or simulated trial.&#xD;
&#xD;
      The investigators anticipate that using a simulated experience for assessment will increase&#xD;
      the ability of raters to assess proficiency for the purposes of an OPPE, specifically by&#xD;
      increasing the number of critical procedural components that can be objectively evaluated.&#xD;
      The investigators hypothesize that there will not be a strong relationship between the scores&#xD;
      obtained on the verbal assessment and the scores obtained on the task trainer assessment. The&#xD;
      investigators anticipate that there will be a portion of study participants that do not meet&#xD;
      a minimum passing standard and may require additional deliberate practice and further&#xD;
      testing.&#xD;
    </textblock>
  </brief_summary>
  <detailed_description>
    <textblock>
      Physician training has developed over recent years to include simulation-based learning as a&#xD;
      method of both formative and summative assessment. Simulation based medical education (SBME)&#xD;
      with deliberate practice has been shown to be superior to traditional clinical education with&#xD;
      respect to procedural skills in physicians in training (McGaghie,W.C. 2011), and a similar&#xD;
      concept with Mastery Learning has shown to increase resident skill in lumbar puncture&#xD;
      placement. In addition, it has been shown that Simulation Based Mastery Learning (SBML) in&#xD;
      central line placement reduces hospital wide rates of central line associated bacterial&#xD;
      infections.&#xD;
&#xD;
      When used in continuing medical education (CME), simulation has been shown to be feasible,&#xD;
      and to increase clinician confidence in patient management. A recent study of nephrology&#xD;
      fellows and attendings in practice showed that both groups scored similarly on a pre-test&#xD;
      examination, and that all fellows who completed mastery learning subsequently passed the&#xD;
      checklist assessment at the competency level and rated the training session highly.&#xD;
&#xD;
      In other work, simulation has been used for ongoing professional practice assessments (OPPE),&#xD;
      for example in psychiatry using online simulated patients and assessments as a way of&#xD;
      identifying a smaller number of attendings requiring focused assessment in person). Using&#xD;
      electronically collected clinical data, anesthesia cases were reviewed for deviation from the&#xD;
      standard of care on several key pieces of documentation common to many anesthesia procedures.&#xD;
      This method of direct chart assessment for OPPE may be possible in anesthesia, a field where&#xD;
      many cases have similarities in procedure, but would not work for a field such as Emergency&#xD;
      Medicine, where the type and acuity of patient encounters varies greatly from day to day.&#xD;
&#xD;
      The ability of Simulation to present a standardized, specific procedural need offers an&#xD;
      opportunity to assess Attending Emergency Physicians on a variety of critical procedures in a&#xD;
      concise way. The investigators' research aims to present one possible approach to this&#xD;
      assessment.&#xD;
    </textblock>
  </detailed_description>
  <overall_status>Completed</overall_status>
  <start_date type="Actual">October 1, 2015</start_date>
  <completion_date type="Actual">May 1, 2016</completion_date>
  <primary_completion_date type="Actual">May 1, 2016</primary_completion_date>
  <study_type>Observational</study_type>
  <has_expanded_access>No</has_expanded_access>
  <study_design_info>
    <observational_model>Case-Only</observational_model>
    <time_perspective>Other</time_perspective>
  </study_design_info>
  <primary_outcome>
    <measure>Performance on Procedural Checklist</measure>
    <time_frame>Immediate</time_frame>
    <description>A 29-item procedural checklist for Central Line and a 21-item procedural checklist for Lumbar puncture will be used to assess performance by clinicians on Central Venous Catheter insertion and Lumbar Puncture performance.</description>
  </primary_outcome>
  <secondary_outcome>
    <measure>Acceptance of simulated training and assessment format</measure>
    <time_frame>Immediate</time_frame>
    <description>A survey tool will be used to query participants on their preferred way of both practicing and being assessed on procedural skill. It will also assess whether participants feel inclined to complete a future simulated procedural training session.</description>
  </secondary_outcome>
  <number_of_groups>1</number_of_groups>
  <enrollment type="Actual">17</enrollment>
  <condition>Competence</condition>
  <condition>Emergencies</condition>
  <arm_group>
    <arm_group_label>2016 OPPE</arm_group_label>
    <description>All faculty participating in yearly ongoing professional Practice evaluation were eligible for enrollment</description>
  </arm_group>
  <intervention>
    <intervention_type>Other</intervention_type>
    <intervention_name>Verbal versus procedural assessment</intervention_name>
    <description>Faculty were assessed on both verbal and simulated performance of procedures.</description>
    <arm_group_label>2016 OPPE</arm_group_label>
  </intervention>
  <eligibility>
    <study_pop>
      <textblock>
        All faculty practicing in an urban emergency department were eligible for enrollment&#xD;
      </textblock>
    </study_pop>
    <sampling_method>Non-Probability Sample</sampling_method>
    <criteria>
      <textblock>
        Inclusion Criteria:&#xD;
&#xD;
          -  Emergency Medicine physician&#xD;
&#xD;
        Exclusion Criteria:&#xD;
&#xD;
          -  Not Emergency Medicine physician&#xD;
      </textblock>
    </criteria>
    <gender>All</gender>
    <minimum_age>N/A</minimum_age>
    <maximum_age>N/A</maximum_age>
    <healthy_volunteers>Accepts Healthy Volunteers</healthy_volunteers>
  </eligibility>
  <overall_official>
    <last_name>Sara M Hock, MD</last_name>
    <role>Principal Investigator</role>
    <affiliation>Rush University Medical Center</affiliation>
  </overall_official>
  <reference>
    <citation>OPPE and FPPE: Tools to help make privileging decisions 2013. Wise RA. (Accessed Jul 19, 2017, at http://www.jointcommission.org/jc_physician_blog/oppe_fppe_tools_privileging_decisions/).</citation>
  </reference>
  <reference>
    <citation>McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011 Jun;86(6):706-11. doi: 10.1097/ACM.0b013e318217e119.</citation>
    <PMID>21512370</PMID>
  </reference>
  <reference>
    <citation>Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents' lumbar puncture skills. Neurology. 2012 Jul 10;79(2):132-7. doi: 10.1212/WNL.0b013e31825dd39d. Epub 2012 Jun 6.</citation>
    <PMID>22675080</PMID>
  </reference>
  <reference>
    <citation>Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009 Oct;37(10):2697-701.</citation>
    <PMID>19885989</PMID>
  </reference>
  <reference>
    <citation>Barsuk JH, Cohen ER, Potts S, Demo H, Gupta S, Feinglass J, McGaghie WC, Wayne DB. Dissemination of a simulation-based mastery learning intervention reduces central line-associated bloodstream infections. BMJ Qual Saf. 2014 Sep;23(9):749-56. doi: 10.1136/bmjqs-2013-002665. Epub 2014 Mar 14.</citation>
    <PMID>24632995</PMID>
  </reference>
  <reference>
    <citation>Kerr B, Hawkins TL, Herman R, Barnes S, Kaufmann S, Fraser K, Ma IW. Feasibility of scenario-based simulation training versus traditional workshops in continuing medical education: a randomized controlled trial. Med Educ Online. 2013 Jul 18;18:21312. doi: 10.3402/meo.v18i0.21312.</citation>
    <PMID>23870304</PMID>
  </reference>
  <reference>
    <citation>McQuillan RF, Clark E, Zahirieh A, Cohen ER, Paparello JJ, Wayne DB, Barsuk JH. Performance of Temporary Hemodialysis Catheter Insertion by Nephrology Fellows and Attending Nephrologists. Clin J Am Soc Nephrol. 2015 Oct 7;10(10):1767-72. doi: 10.2215/CJN.01720215. Epub 2015 Sep 25.</citation>
    <PMID>26408550</PMID>
  </reference>
  <reference>
    <citation>Barsuk JH, Cohen ER, Nguyen D, Mitra D, O'Hara K, Okuda Y, Feinglass J, Cameron KA, McGaghie WC, Wayne DB. Attending Physician Adherence to a 29-Component Central Venous Catheter Bundle Checklist During Simulated Procedures. Crit Care Med. 2016 Oct;44(10):1871-81. doi: 10.1097/CCM.0000000000001831.</citation>
    <PMID>27336437</PMID>
  </reference>
  <verification_date>July 2017</verification_date>
  <study_first_submitted>July 17, 2017</study_first_submitted>
  <study_first_submitted_qc>July 24, 2017</study_first_submitted_qc>
  <study_first_posted type="Actual">July 25, 2017</study_first_posted>
  <last_update_submitted>July 26, 2017</last_update_submitted>
  <last_update_submitted_qc>July 26, 2017</last_update_submitted_qc>
  <last_update_posted type="Actual">July 28, 2017</last_update_posted>
  <responsible_party>
    <responsible_party_type>Sponsor</responsible_party_type>
  </responsible_party>
  <keyword>Simulation</keyword>
  <keyword>Task Trainer</keyword>
  <condition_browse>
    <!-- CAUTION:  The following MeSH terms are assigned with an imperfect algorithm            -->
    <mesh_term>Emergencies</mesh_term>
  </condition_browse>
  <patient_data>
    <sharing_ipd>Undecided</sharing_ipd>
    <ipd_description>If requested, data may be shared. Limited generalizability of data may limit its usefulness to other investigators</ipd_description>
  </patient_data>
  <!-- Results have not yet been posted for this study                                          -->
</clinical_study>


download date: ClinicalTrials.gov processed this data on September 20, 2023
link text: Link to the current ClinicalTrials.gov record.
url: https://clinicaltrials.gov/ct2/show/NCT03229226
org study id: 15100805-IRB01-AM01
nct id: NCT03229226
lead sponsor: 
has dmc: Yes
is fda regulated drug: No
is fda regulated device: No
Since 2007, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has
      required evaluation of practicing attending physicians by management to ensure that
      procedural skills and medical care are meeting the accepted standards of care as assessed by
      each individual institution. This assessment is known as the ongoing professional practice
      evaluation (OPPE). There are several methods by which this can be accomplished, including
      through chart review of clinician practice and by verbally assessing knowledge of the steps
      required to do a particular procedure. However, for infrequently performed or complicated
      procedures, these methods may not allow objective evaluation on a regular basis. Simulation
      using task trainers or manikin models offers an alternative method of objective evaluation in
      a standardized setting.

      The goal of this study is to develop two simulated scenarios to assess physician skill in
      relatively uncommon procedures and compare the data obtained against the verbal assessment
      and chart review model previously used at our institution. Two raters will assess each
      practitioner's verbal and simulated procedures. They will also assess globally their
      confidence that the physician is capable of performing the procedure safely and correctly
      based on the verbal or simulated trial.

      The investigators anticipate that using a simulated experience for assessment will increase
      the ability of raters to assess proficiency for the purposes of an OPPE, specifically by
      increasing the number of critical procedural components that can be objectively evaluated.
      The investigators hypothesize that there will not be a strong relationship between the scores
      obtained on the verbal assessment and the scores obtained on the task trainer assessment. The
      investigators anticipate that there will be a portion of study participants that do not meet
      a minimum passing standard and may require additional deliberate practice and further
      testing.
Physician training has developed over recent years to include simulation-based learning as a
      method of both formative and summative assessment. Simulation based medical education (SBME)
      with deliberate practice has been shown to be superior to traditional clinical education with
      respect to procedural skills in physicians in training (McGaghie,W.C. 2011), and a similar
      concept with Mastery Learning has shown to increase resident skill in lumbar puncture
      placement. In addition, it has been shown that Simulation Based Mastery Learning (SBML) in
      central line placement reduces hospital wide rates of central line associated bacterial
      infections.

      When used in continuing medical education (CME), simulation has been shown to be feasible,
      and to increase clinician confidence in patient management. A recent study of nephrology
      fellows and attendings in practice showed that both groups scored similarly on a pre-test
      examination, and that all fellows who completed mastery learning subsequently passed the
      checklist assessment at the competency level and rated the training session highly.

      In other work, simulation has been used for ongoing professional practice assessments (OPPE),
      for example in psychiatry using online simulated patients and assessments as a way of
      identifying a smaller number of attendings requiring focused assessment in person). Using
      electronically collected clinical data, anesthesia cases were reviewed for deviation from the
      standard of care on several key pieces of documentation common to many anesthesia procedures.
      This method of direct chart assessment for OPPE may be possible in anesthesia, a field where
      many cases have similarities in procedure, but would not work for a field such as Emergency
      Medicine, where the type and acuity of patient encounters varies greatly from day to day.

      The ability of Simulation to present a standardized, specific procedural need offers an
      opportunity to assess Attending Emergency Physicians on a variety of critical procedures in a
      concise way. The investigators' research aims to present one possible approach to this
      assessment.
observational model: Case-Only
time perspective: Other
measure: Performance on Procedural Checklist
time frame: Immediate
description: A 29-item procedural checklist for Central Line and a 21-item procedural checklist for Lumbar puncture will be used to assess performance by clinicians on Central Venous Catheter insertion and Lumbar Puncture performance.
measure: Acceptance of simulated training and assessment format
time frame: Immediate
description: A survey tool will be used to query participants on their preferred way of both practicing and being assessed on procedural skill. It will also assess whether participants feel inclined to complete a future simulated procedural training session.
arm group label: 2016 OPPE
description: All faculty participating in yearly ongoing professional Practice evaluation were eligible for enrollment
intervention type: Other
intervention name: Verbal versus procedural assessment
description: Faculty were assessed on both verbal and simulated performance of procedures.
arm group label: 2016 OPPE
study pop: 
sampling method: Non-Probability Sample
criteria: 
gender: All
minimum age: N/A
maximum age: N/A
healthy volunteers: Accepts Healthy Volunteers
last name: Sara M Hock, MD
role: Principal Investigator
affiliation: Rush University Medical Center
citation: OPPE and FPPE: Tools to help make privileging decisions 2013. Wise RA. (Accessed Jul 19, 2017, at http://www.jointcommission.org/jc_physician_blog/oppe_fppe_tools_privileging_decisions/).
citation: McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011 Jun;86(6):706-11. doi: 10.1097/ACM.0b013e318217e119.
PMID: 21512370
citation: Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents' lumbar puncture skills. Neurology. 2012 Jul 10;79(2):132-7. doi: 10.1212/WNL.0b013e31825dd39d. Epub 2012 Jun 6.
PMID: 22675080
citation: Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009 Oct;37(10):2697-701.
PMID: 19885989
citation: Barsuk JH, Cohen ER, Potts S, Demo H, Gupta S, Feinglass J, McGaghie WC, Wayne DB. Dissemination of a simulation-based mastery learning intervention reduces central line-associated bloodstream infections. BMJ Qual Saf. 2014 Sep;23(9):749-56. doi: 10.1136/bmjqs-2013-002665. Epub 2014 Mar 14.
PMID: 24632995
citation: Kerr B, Hawkins TL, Herman R, Barnes S, Kaufmann S, Fraser K, Ma IW. Feasibility of scenario-based simulation training versus traditional workshops in continuing medical education: a randomized controlled trial. Med Educ Online. 2013 Jul 18;18:21312. doi: 10.3402/meo.v18i0.21312.
PMID: 23870304
citation: McQuillan RF, Clark E, Zahirieh A, Cohen ER, Paparello JJ, Wayne DB, Barsuk JH. Performance of Temporary Hemodialysis Catheter Insertion by Nephrology Fellows and Attending Nephrologists. Clin J Am Soc Nephrol. 2015 Oct 7;10(10):1767-72. doi: 10.2215/CJN.01720215. Epub 2015 Sep 25.
PMID: 26408550
citation: Barsuk JH, Cohen ER, Nguyen D, Mitra D, O'Hara K, Okuda Y, Feinglass J, Cameron KA, McGaghie WC, Wayne DB. Attending Physician Adherence to a 29-Component Central Venous Catheter Bundle Checklist During Simulated Procedures. Crit Care Med. 2016 Oct;44(10):1871-81. doi: 10.1097/CCM.0000000000001831.
PMID: 27336437
responsible party type: Sponsor
mesh term: Emergencies
sharing ipd: Undecided
ipd description: If requested, data may be shared. Limited generalizability of data may limit its usefulness to other investigators