Trainee perspectives on diversity, equity, and inclusion: a group concept mapping study
With growing recognition of the impact of racism on medical trainees, national organizations such as the Accreditation Council for Graduate Medical Education (ACGME) have called for greater focus on diversity, equity, and inclusion (DEI) in graduate medical education . Institutional responses to this issue have included increased recruitment of groups underrepresented in medicine (URiM) , curricular efforts focused on implicit bias, microaggressions, and social determinants of health , and leadership structures that center DEI as a program priority ; however, creating a just culture in academic medicine requires a commitment to building community and centering trainee perspectives in such reform efforts . Thus, planning and evaluation frameworks for graduate medical education programs that meaningfully incorporate trainee expertise are critical. Group concept mapping (GCM) is a participatory research methodology that allows for structured representation of community perspectives to aid in program evaluation, strategic planning, and intervention development . GCM represents an equitable research approach where power is shared between participants and investigators, making it particularly salient for conversations related to DEI. It has previously been used in graduate medical education to understand factors that promote effective clinical learning environments , to describe core professional competencies for practicing physicians , and to delineate influences of well-being during medical education . Limited work has considered use of GCM to solicit trainee perspectives on DEI. This gap is notable, as leveraging GCM to collaboratively organize ideas, opinions, and priorities related to DEI can directly inform planning of DEI-focused initiatives and evaluation targeted at these areas. In the current study, we conducted a GCM activity among pediatric residents at a large academic medical center. Through this participatory research exercise, our goals were to: 1) characterize trainee perspectives related to DEI at our institution; 2) prioritize this information based on importance and perceived feasibility to impact change; and 3) propose actions to address ideas emerging from this process. We present details regarding GCM implementation, analysis, and interpretation, with the goal of articulating how this can inform policies to meaningfully support DEI in our institution, which may also be generalizable to other graduate medical education settings. 

We used GCM to conduct a mixed methods evaluation of trainee perspectives on DEI at a large academic medical center. GCM employs six key steps: preparation, generation, structuring, representation, interpretation, and utilization. In the preparation phase, researchers identify study participants and create prompts for generation and structuring activities. During generation , participants brainstorm ideas or concepts related to this focal prompt. Structuring involves sorting and rating activities, in which participants group items from the generation phase based on similarity to one another and rate each item using a predetermined scale (e.g., importance or feasibility to change). Researchers subsequently implement multidimensional scaling and hierarchical clustering analyses to represent this data via several types of visual plots (e.g., point maps, cluster maps, and ‘go-zone’ analyses). Participants review and interpret these visual representations, and utilization of this dialogue by participants, research team members, and other stakeholders enables subsequent program evaluation and planning. The present study leverages data from meeting notes, surveys, and group discussions to describe the GCM activity at our institution. Additional information regarding each step in GCM is presented below. Preparation We recruited pediatric residents from a large academic medical center located in a diverse urban area (63.7% White, 22.5% Black, 6.2% multiracial, 5.8% Asian, 4.2% Hispanic or Latino origin)  for participation in the GCM activity. Residents were informed of this study by email and were eligible to participate in any or all research activities. The research team selected the focal prompt to align with the specific aims of our study, and structuring activities utilized importance and feasibility to change scales to prioritize items generated from brainstorming. Generation We held an hour-long hybrid in-person/virtual brainstorming session during existing resident educational time. Medical students and rotating residents could also attend. Following a brief introduction to GCM methodology, we asked participants to respond to the following focal prompt: ‘ List all factors, good or bad, that impact diversity, equity, and inclusion (DEI) in the [residency program name] .’ Individuals who attended the session in-person wrote responses on adhesive note paper and those attending virtually completed an anonymous online form accessible via QR code. After individually recording responses to the prompt, we asked participants to share items aloud to promote discussion and reflection. The principal investigator (NS) compiled individual and group responses into a single list for review. Two members of our research team (NS & OT) revised item phrasing for clarity, combined items that were redundant, and divided items which represented more than one construct. Phrases indicating valence (e.g., ‘lack of’) were removed to ensure neutrality. Using iterative consensus finding, we established a final list of items for use in structuring and rating activities. Structuring We emailed all residents to invite them to participate in sorting/rating exercises, regardless of their attendance at the brainstorming session. Participants completed these activities electronically using a secure web-based interface (Concept Systems, Incorporated; Ithaca, NY). The sorting program instructed residents to sort items into groupings by similarity to one another. Residents also rated each item on two scales: 1) importance (e.g., ‘What factors are most important?’) and 2) likelihood to change (e.g., ‘What factors are most likely to be changed?’) (5-point Likert scale). Residents completing sorting/rating activities for at least 75% of items were included in analysis. Basic demographic information, including age, gender identity, training year, and race/ethnicity, was also collected. Representation We conducted multidimensional scaling and hierarchical clustering analyses using Concept Systems software. Results of this analysis were used to generate point maps, cluster maps, and ‘go-zone’ illustrations. Two members of the research team (NS & OT) reviewed cluster solutions ranging from 4 to 12, and a 5 cluster solution was selected (median number of participant piles: 6). We presented these visual representations to residents during another synchronous session. We also calculated the stress value, a goodness-of-fit measure for multidimensional scaling . Interpretation We presented point maps, cluster maps, and go-zone analyses to trainees during a synchronous hybrid in-person/virtual interpretation session. Participants reviewed point maps, item ratings, and the cluster map solution and developed names for each cluster. Using go-zone plots, attendees of this session prioritized several areas for further discussion as outlined below. The principal investigator (NS) shared data from this session via a residency-wide email list. All residents in the training program were invited to offer additional feedback, including any other areas that should be prioritized. Utilization After conducting these activities with trainees, two members of the research team (NS & JW) presented to residency program directors and the department vice chair of DEI. This dialogue generated several measurable action steps to inform program planning and evaluation specific to DEI. Analysis We summarized demographic characteristics of survey respondents using descriptive statistics (frequencies and proportions). We calculated a mean score for each item on importance and likelihood to change scales. We present results of concept mapping according to each cluster, integrating quantitative information on items in that cluster (e.g., number of items, rating of items) and qualitative data from group discussions. Our study was determined to be exempt by the University of Pittsburgh Institutional Review Board. Brainstorming and interpretation sessions were audio-recorded, but no other identifying information was collected. Although these occurred during existing resident educational time, attendance was optional, and residents choosing not to participate could pursue independent learning. Residents who completed online sorting and rating activities received $15 renumeration for their time. 

We recruited pediatric residents from a large academic medical center located in a diverse urban area (63.7% White, 22.5% Black, 6.2% multiracial, 5.8% Asian, 4.2% Hispanic or Latino origin)  for participation in the GCM activity. Residents were informed of this study by email and were eligible to participate in any or all research activities. The research team selected the focal prompt to align with the specific aims of our study, and structuring activities utilized importance and feasibility to change scales to prioritize items generated from brainstorming. 

We held an hour-long hybrid in-person/virtual brainstorming session during existing resident educational time. Medical students and rotating residents could also attend. Following a brief introduction to GCM methodology, we asked participants to respond to the following focal prompt: ‘ List all factors, good or bad, that impact diversity, equity, and inclusion (DEI) in the [residency program name] .’ Individuals who attended the session in-person wrote responses on adhesive note paper and those attending virtually completed an anonymous online form accessible via QR code. After individually recording responses to the prompt, we asked participants to share items aloud to promote discussion and reflection. The principal investigator (NS) compiled individual and group responses into a single list for review. Two members of our research team (NS & OT) revised item phrasing for clarity, combined items that were redundant, and divided items which represented more than one construct. Phrases indicating valence (e.g., ‘lack of’) were removed to ensure neutrality. Using iterative consensus finding, we established a final list of items for use in structuring and rating activities. 

We emailed all residents to invite them to participate in sorting/rating exercises, regardless of their attendance at the brainstorming session. Participants completed these activities electronically using a secure web-based interface (Concept Systems, Incorporated; Ithaca, NY). The sorting program instructed residents to sort items into groupings by similarity to one another. Residents also rated each item on two scales: 1) importance (e.g., ‘What factors are most important?’) and 2) likelihood to change (e.g., ‘What factors are most likely to be changed?’) (5-point Likert scale). Residents completing sorting/rating activities for at least 75% of items were included in analysis. Basic demographic information, including age, gender identity, training year, and race/ethnicity, was also collected. 

We conducted multidimensional scaling and hierarchical clustering analyses using Concept Systems software. Results of this analysis were used to generate point maps, cluster maps, and ‘go-zone’ illustrations. Two members of the research team (NS & OT) reviewed cluster solutions ranging from 4 to 12, and a 5 cluster solution was selected (median number of participant piles: 6). We presented these visual representations to residents during another synchronous session. We also calculated the stress value, a goodness-of-fit measure for multidimensional scaling . 

We presented point maps, cluster maps, and go-zone analyses to trainees during a synchronous hybrid in-person/virtual interpretation session. Participants reviewed point maps, item ratings, and the cluster map solution and developed names for each cluster. Using go-zone plots, attendees of this session prioritized several areas for further discussion as outlined below. The principal investigator (NS) shared data from this session via a residency-wide email list. All residents in the training program were invited to offer additional feedback, including any other areas that should be prioritized. 

After conducting these activities with trainees, two members of the research team (NS & JW) presented to residency program directors and the department vice chair of DEI. This dialogue generated several measurable action steps to inform program planning and evaluation specific to DEI. 

We summarized demographic characteristics of survey respondents using descriptive statistics (frequencies and proportions). We calculated a mean score for each item on importance and likelihood to change scales. We present results of concept mapping according to each cluster, integrating quantitative information on items in that cluster (e.g., number of items, rating of items) and qualitative data from group discussions. Our study was determined to be exempt by the University of Pittsburgh Institutional Review Board. Brainstorming and interpretation sessions were audio-recorded, but no other identifying information was collected. Although these occurred during existing resident educational time, attendance was optional, and residents choosing not to participate could pursue independent learning. Residents who completed online sorting and rating activities received $15 renumeration for their time. 

Participants A total of 127 trainees were in the residency program during the 2021–2022 academic year (PGY-1: 41; PGY-2: 41; PGY-3: 36; PGY-4 or greater: 9). Attendance varied across sessions: 21 participants attended the brainstorming session, 48 participants completed sorting and rating activities, and 20 participants attended the interpretation session. Of those participating in sorting and rating, most ( n = 32, 67%) were female, White ( n = 34, 71%) and earlier in training (PGY-1: n = 16, 33%; PGY-2: n = 22, 46%; PGY-3: n = 8, 17%; PGY-4 or greater: n = 2, 4%). Additional demographic information for these participants is shown in . Group concept mapping During the brainstorming session, individual responses were collected and compiled. An additional 15 items were generated from group discussion, resulting in 147 total items that were reviewed by the research team. After removing duplicates, combing redundant items, and separating items representing multiple concepts, a final list of 64 items was prepared for use in sorting and rating activities (Supplemental Table S1). A five cluster solution was selected (stress value: 0.282), reviewed, and named during the interpretation session as follows: 1) city factors, 2) institutional factors, 3) program representation, 4) program components, 5) non-DEI program perks . The city factors cluster included items pertaining to the population of the surrounding city (e.g., ‘diversity of Pittsburgh’), historical aspects of the environment (e.g., ‘Pittsburgh’s geographic segregation’), and other community infrastructure (e.g., ‘public transportation’ and ‘affordable housing’). Participants generally felt items in this cluster were congruent with one another, but they noted that rural/urban divide should also be considered. Items in the institutional factors cluster related to a broad range of topics. Some items, such as ‘paternalistic approaches to patient care’ and ‘disparities in mandated reporting,’ referenced cultural aspects of clinical care. Other items (e.g., ‘inclusion of race on clinical forms and documents’) involved specific administrative or workflow elements. Two items in this category, ‘gender tax’ and ‘minority tax,’ were identified as factors that may not be unique to this cluster and features that should be considered more broadly. For the program representation cluster, most items corresponded to diversity of various employee subgroups (e.g., hospital leadership, faculty, program leadership, staff, residents). Efforts involving recruitment of URiM candidates were also included in this category. Residents did identify additional domains of employee diversity (e.g., diversity of chief residents) and other metrics for representation (e.g., persons represented in hospital artwork and social media) that were not on the initial list of sorted/rated items. A few items (e.g., ‘safe and open conversations about DEI’ and ‘time commitment for DEI activities’) were thought better suited for the program components cluster. Program components involved items such as ‘orientation activities’ and ‘residency social media,’ indicating a theme of residency-specific programming. Other cultural aspects of the program (e.g., ‘residency willingness to change’) were also included in this cluster. Items referencing residency recruitment initiatives were felt to align with this category, though these were sorted with the program representation cluster in hierarchical analyses. The cluster representing non-DEI program perks included aspects of the larger graduate medical education climate (e.g., ‘similarities/differences across [name of academic center] training programs’). Some items related to program benefits, such as salary and promotion structure. Other logistics of the program (e.g., ‘residency workload’ and ‘visiting students and residents’) were also sorted in this cluster. Topics that were not listed in this cluster but were brought up by participants during the interpretation session were the program’s jeopardy system (e.g., emergency coverage for unanticipated absences, illnesses, etc.) and holiday scheduling (e.g., which holidays are prioritized in making coverage schedules). The program representation and program components clusters were rated as the most important (mean: 4.21 ± 0.30 and 3.72 ± 0.79, respectively) and the most likely to be changed (mean: 3.56 ± 0.47 and 3.63 ± 0.39, respectively). The non-DEI program perks cluster was rated as the least important (mean: 3.05 ± 0.29). Go-zone analyses revealed 25 individual items that participants rated as the most important and the most likely to be changed . Interpretation During the interpretation session, trainees identified several priority areas from the GCM analyses. In order to address language equity, residents recommended increasing the availability of language service resources (e.g., in-person translators, translation of discharge paperwork). Residents felt simple steps could be taken to reduce microaggressions related to name pronunciation, such as including phonetic spelling of names on identification badges or recording a name pronunciation clip for inclusion in email signatures as part of onboarding. Trainees who parented children during training felt additional supports could be in place to improve their training experience, such as restructuring their elective time to allow greater flexibility in selecting their schedules. To promote greater recruitment and retention of URiM individuals, residents suggested the creation of dedicated program tracks for URiM applicants. In addition, they emphasized the need to reduce the minority tax and engage non-URiM residents and faculty in DEI efforts. Finally, trainees recognized the importance of advocacy opportunities during residency training but felt many were resident-initiated and called for increased partnership with faculty to support such initiatives. Utilization We disseminated a summary of GCM discussions to all pediatric residents in the program. Results were also presented to the departmental vice chair of DEI and residency program directors to inform new and ongoing initiatives related to DEI at our institution. 

A total of 127 trainees were in the residency program during the 2021–2022 academic year (PGY-1: 41; PGY-2: 41; PGY-3: 36; PGY-4 or greater: 9). Attendance varied across sessions: 21 participants attended the brainstorming session, 48 participants completed sorting and rating activities, and 20 participants attended the interpretation session. Of those participating in sorting and rating, most ( n = 32, 67%) were female, White ( n = 34, 71%) and earlier in training (PGY-1: n = 16, 33%; PGY-2: n = 22, 46%; PGY-3: n = 8, 17%; PGY-4 or greater: n = 2, 4%). Additional demographic information for these participants is shown in . 

During the brainstorming session, individual responses were collected and compiled. An additional 15 items were generated from group discussion, resulting in 147 total items that were reviewed by the research team. After removing duplicates, combing redundant items, and separating items representing multiple concepts, a final list of 64 items was prepared for use in sorting and rating activities (Supplemental Table S1). A five cluster solution was selected (stress value: 0.282), reviewed, and named during the interpretation session as follows: 1) city factors, 2) institutional factors, 3) program representation, 4) program components, 5) non-DEI program perks . The city factors cluster included items pertaining to the population of the surrounding city (e.g., ‘diversity of Pittsburgh’), historical aspects of the environment (e.g., ‘Pittsburgh’s geographic segregation’), and other community infrastructure (e.g., ‘public transportation’ and ‘affordable housing’). Participants generally felt items in this cluster were congruent with one another, but they noted that rural/urban divide should also be considered. Items in the institutional factors cluster related to a broad range of topics. Some items, such as ‘paternalistic approaches to patient care’ and ‘disparities in mandated reporting,’ referenced cultural aspects of clinical care. Other items (e.g., ‘inclusion of race on clinical forms and documents’) involved specific administrative or workflow elements. Two items in this category, ‘gender tax’ and ‘minority tax,’ were identified as factors that may not be unique to this cluster and features that should be considered more broadly. For the program representation cluster, most items corresponded to diversity of various employee subgroups (e.g., hospital leadership, faculty, program leadership, staff, residents). Efforts involving recruitment of URiM candidates were also included in this category. Residents did identify additional domains of employee diversity (e.g., diversity of chief residents) and other metrics for representation (e.g., persons represented in hospital artwork and social media) that were not on the initial list of sorted/rated items. A few items (e.g., ‘safe and open conversations about DEI’ and ‘time commitment for DEI activities’) were thought better suited for the program components cluster. Program components involved items such as ‘orientation activities’ and ‘residency social media,’ indicating a theme of residency-specific programming. Other cultural aspects of the program (e.g., ‘residency willingness to change’) were also included in this cluster. Items referencing residency recruitment initiatives were felt to align with this category, though these were sorted with the program representation cluster in hierarchical analyses. The cluster representing non-DEI program perks included aspects of the larger graduate medical education climate (e.g., ‘similarities/differences across [name of academic center] training programs’). Some items related to program benefits, such as salary and promotion structure. Other logistics of the program (e.g., ‘residency workload’ and ‘visiting students and residents’) were also sorted in this cluster. Topics that were not listed in this cluster but were brought up by participants during the interpretation session were the program’s jeopardy system (e.g., emergency coverage for unanticipated absences, illnesses, etc.) and holiday scheduling (e.g., which holidays are prioritized in making coverage schedules). The program representation and program components clusters were rated as the most important (mean: 4.21 ± 0.30 and 3.72 ± 0.79, respectively) and the most likely to be changed (mean: 3.56 ± 0.47 and 3.63 ± 0.39, respectively). The non-DEI program perks cluster was rated as the least important (mean: 3.05 ± 0.29). Go-zone analyses revealed 25 individual items that participants rated as the most important and the most likely to be changed . 

During the interpretation session, trainees identified several priority areas from the GCM analyses. In order to address language equity, residents recommended increasing the availability of language service resources (e.g., in-person translators, translation of discharge paperwork). Residents felt simple steps could be taken to reduce microaggressions related to name pronunciation, such as including phonetic spelling of names on identification badges or recording a name pronunciation clip for inclusion in email signatures as part of onboarding. Trainees who parented children during training felt additional supports could be in place to improve their training experience, such as restructuring their elective time to allow greater flexibility in selecting their schedules. To promote greater recruitment and retention of URiM individuals, residents suggested the creation of dedicated program tracks for URiM applicants. In addition, they emphasized the need to reduce the minority tax and engage non-URiM residents and faculty in DEI efforts. Finally, trainees recognized the importance of advocacy opportunities during residency training but felt many were resident-initiated and called for increased partnership with faculty to support such initiatives. 

We disseminated a summary of GCM discussions to all pediatric residents in the program. Results were also presented to the departmental vice chair of DEI and residency program directors to inform new and ongoing initiatives related to DEI at our institution. 

We demonstrate the utility of GCM, a structured and replicable form of participatory research, for characterizing trainee perspectives related to DEI in a large pediatric residency program. Our findings highlight adaptability of this mixed-methods research approach in the graduate medical education setting, particularly using an asynchronous online survey platform. We reaffirm the importance of elevating trainee voice in DEI efforts, as these may differ substantially from those of faculty . While others have used GCM in medical education , our study is the first to our knowledge that applies this methodology specifically to pediatric residents or to factors impacting DEI, which remains a priority area for residency training programs . In addition, cultivating dialogue on DEI using the GCM approach can generate meaningful qualitative data to support funding and interventions in this space, particularly in an era when DEI may be deprioritized. Issues that were felt to be most impactful among participants included items primarily in the program representation and program components clusters. More specifically, residents identified the importance of having diverse faculty, residents, and staff in the hospital at large. In addition, they recognized the need for engagement and buy-in from hospital and program leadership, including bolstering efforts related to DEI recruitment, programming, and educational initiatives. Notably, while increasing resident, faculty, and staff diversity was felt to be important, these items were not rated as very likely to be changed, indicating perceived obstacles that may exist to achieving this goal. Thus, beyond intentional recruitment strategies, these findings underscore the need for focused, sustainable supports that foster an environment of inclusion. Multiple items involving the relationship between the institution and the community (e.g., ‘Awareness of community resources,’ ‘Availability of community resources,’ ‘Community involvement/voice in hospital’) were recognized as critical aspects of DEI in the program. Prior to this study, our program included several training components in alignment with this principle. For example, as of 2018, all first year residents participate in a two-week outpatient rotation in which they meet with several community partners to develop greater understanding of neighborhood assets and patient-facing resources. Our program’s antiracist curriculum, developed in 2020–2021, features content developed in collaboration with community members . Additional efforts to center community voice in our institution include an ongoing large-scale, community-partnered research study . Such initiatives are aimed at redistributing existing forms of institutional power and promoting child health equity. We acknowledge limitations to the present study. First, while GCM serves as a useful tool for structured representation of complex topics, the present work does not capture the full range of opinions or ideas related to DEI. In addition, while many residents completed online sorting and rating activities, fewer were engaged in synchronous brainstorming and interpretation sessions. Although this may be due in part to scheduling conflicts (e.g., night float, off-site electives, etc.), it also highlights continued challenges related to in-person educational programming in the setting of the COVID-19 pandemic . We also recognize the lack of racial and ethnic diversity in our cohort, which may bias the current findings. Further, due to the small number of residents identifying as a race or ethnicity other than non-Hispanic White, we were unable to perform comparative analyses. In spite of these challenges, this study offered our program a conceptual framework from which to approach ongoing initiatives related to DEI. These results also directly informed several action steps generated as a part of the interpretation session. For example, during our meeting with residency program leadership, we discussed ways to incorporate name pronunciation guides into email signatures and name badges as a part of resident orientation and onboarding, which we look forward to collecting data on in the future. We also have worked with the departmental vice chair to identify opportunities for increased engagement of non-URiM residents in DEI initiatives, such as by establishing a ‘DEI Champion’ pathway for a subset of individuals in each residency class. Because this GCM activity focused on representation and discussion of trainee perspectives, it will be critical to leverage this data to operationalize and monitor key DEI-related outcomes, such as resident demographics, experiences of discrimination and bias, equitable attitudes and behaviors, and perceived inclusiveness of the program. We anticipate that this work can be translated effectively to other institutions to guide similar processes in program evaluation and strategic planning. Indeed, such an approach allows for novel and informative data collection methods regarding different aspects of medical education, all while centering the perspectives of its participants. 

