An Evaluation of an Experiential Learning Program in Global and Indigenous Health: The University of Manitoba’s Queen Elizabeth II Diamond Jubilee Scholarship Program

We conducted a mixed-methods outcome evaluation to examine student experiences and learning in the University of Manitoba’s Queen Elizabeth II Diamond Jubilee Scholarship Program in Global and Indigenous Health. Our scholarship program is a bi-directional, 3-month international experiential learning program, including both undergraduate and graduate students, with associated online course focused on community engagement. Students completed a semi-structured narrative report at the conclusion of their funding related to their experience and learning. The Likert questions were analyzed descriptively and student responses to the open-ended questions were utilized for thematic analysis. Also included in this paper is a summary of our lessons learned through program administration. A total of 38 students completed the program between 2016 and 2018, with 95% reporting that they either met or exceeded their goals in the program. Three overarching and inter-related themes emerged in our thematic-analysis of students’ narrative reports, including success through relationships and new perspectives, challenges of the unfamiliar, and personal growth through strong emotions. Many students reported personal growth as their greatest success and linked this with new perspectives and awareness of how different contexts shaped their understanding of health issues. Overcoming challenges in their placements contributed to students’ confidence in their ability to problem-solve. Overall, students reported value in their experiential learning, which further supports the growing trend to incorporate both experiential learning and formal education in community engagement in public health pedagogy. However, international experiential learning requires considerable financial and human resource commitments to ensure its success.


Introduction
Health equity and social justice are viewed as central themes in Canadian public health education, which was catalyzed by the Ottawa Charter for Health Promotion. 1 This focus informs our research approaches, particularly in global and Indigenous health, in order to address global and domestic health disparities. 2,3 (Indigenous in this paper refers to Indigenous to their own lands. More specifically in the context of our program, Indigenous health in settler colonial countries within the Commonwealth.) Increasingly factors influencing public health both globally and domestically are not contained by, nor independent of, national borders. This is currently evidenced by the 2020 global COVID-19 pandemic. 4 As such, public health professionals and researchers will be progressively required to understand and work in cross-cultural settings. At the same time, given that health issues are multi-causal, addressing complex and systemic global health issues requires ongoing advancements in public health education to facilitate new skills for trainees, particularly in community engagement, interdisciplinary and systems thinking, inter-professional collaboration, and leadership. As public health education evolves and responds to this changing context and workforce need, there is a growing interest in implementing experiential learning opportunities and course content in community engagement. We report here on the evaluation of the experiences and learning of students that took part in an experiential learning program in Indigenous and/or global health settings that included a course in community engagement.

Public Health Pedagogy
Public health education is interdisciplinary in nature, including a focus on the traditionally core disciplines of epidemiology and biostatistics, and more recently health promotion and critical social sciences. Though theoretical components of public health education will always remain important, the application of theory to public health practice has not received the same level of attention in public health "classrooms." Both Indigenous and global health research have colonial histories that have shaped past and present public health practice. 5 Given these power asymmetries, community engagement is widely understood to be of critical importance to both fields to contribute to relevant and prioritized research questions, enhance uptake of research findings, and ensure ethical conduct, among other reasons. Despite this acknowledged importance, a recent review of 76 graduate-level public health programs in Canada revealed only 16% required a course in community engagement. 6 With Canadian graduate programs offering 6 times more quantitative methods courses compared to qualitative, Yassi et al 6 further state that, "an appreciation of historical, socioeconomic, and cultural context cannot be captured in quantitative methods alone, nor could the political forces that determine the social processes driving the health inequities manifesting at the individual unit of analysis" (p. 6). The need for new paradigms combined with limited research on improving the training of public health workforce, 7,8 suggests further educational research in the area of communityengaged public health practice is urgently needed. This is especially pertinent given the documented challenges of ethical engagement and its complexities. 9

Experiential Learning
Learning by doing, or "experiential learning," is, in its foundation, interdisciplinary and constructivist. 10 Experiential learning is most effective when coupled with critical reflection of new experiences, which can spur new ways of thinking and new attitudes. Chapman et al 11 describe core, inter-related, attributes of experiential learning, as a balance between content and theory; safe space(s) to critically selfreflect and discuss learning; personally relevant and meaningful; big picture or systems thinking; presence of meaningful relationships; and getting outside one's comfort zone. 11 In the context of public health, the Harvard T. H. Chan School of Public Health is advocating for experiential learning to be a core component of public health moving forward, as well as complementary online learning. 12 We must acknowledge that real-world challenges cannot be duplicated in the classroom, nor can complex research questions be answered through datasets alone. Through experiential learning students develop practical skills in reflective learning that can be applied later in their professional lives, despite changing contexts and health issues.
Indigenous health accepted its first students in 2016. The national QES program, and its affiliated university-specific programs, are jointly funded through an initiative of the Rideau Hall Foundation, Community Foundations of Canada and Universities Canada. 13 The goal of the national QES program is to develop global citizens, activate a new generation of global leaders, enhance collaborative capacity and relationship building, and amplify community development across the Commonwealth. Our specific university QES program sought to: (1) provide 3-month, international community-based participatory research experiences to local and international students in Indigenous and global health; (2) sustain and strengthen relationships between the University of Manitoba and our partner organizations; and (3) increase local capacity with respect to research skills and mentor Indigenous leaders and leaders from lower income countries.
Program design. The conceptual framework for the program was based on a model, similar to that developed by Gough et al. 14 In this model a number of core components of the partnership-based approach were applied, including codevelopment and co-implementation of the program, ongoing critical and ethical engagement and reflexive learning, and ethical partner relationships. The network of program partners was facilitated through a number of units and individuals at the University of Manitoba engaged in research relationships in our partner countries: Australia, Belize, Kenya, India, and New Zealand. We sought to ensure reciprocal value to the host organizations, where possible, by providing students from our partner institutions abroad research opportunities at our institution within the program. In this way, further networks were nurtured between institutions, faculty, and students. Student eligibility criteria was purposively broad to capture multiple disciplines and programs, beyond public health, including at the undergraduate and graduate level ( Table 1). Eligibility criteria, established by Universities Canada, also greatly shaped our program structure, which, most notably, resulted in the exclusion of trainees >35 years old. Later concessions were made for some students on the age requirement by the funder. Initially, we noted that the age restriction was particularly challenging for recruiting Indigenous graduate students. Nevertheless, our broad wcriteria resulted in a diverse cadre of inter-disciplinary (eg, public health, medical microbiology, literature, science, nutrition, environment), and inter-professional (eg, medicine, occupational therapy, dietetics, pharmacy) trainees. The timelines for student planning, application, travel, course, and report submission is illustrated in Figure 1.
The pedagogical aspect of the program was informed by an experiential learning model. 10 All students were required to complete a 6-week online course in Community Engagement and Program Science, summarized in Table 2. This course was developed together with faculty conducting both global and Indigenous health research as well as with input from our Indigenous and global health partners. Keeping with the experiential learning pedagogical approach, we included weekly online discussion of course readings with application to the students ongoing research and fieldwork abroad. In this way, reflexivity was woven throughout the course.
Our purpose was to evaluate student learning and experiences in our program. Specifically, we sought to assess students' self-reported development of leadership, networking, and communication skills; establishment of networks; awareness of challenges faced by local or global communities; and achievement of their program-specific or project goals. Additional evaluation questions included: University (n = 9); community organization (n = 2); or healthcare facility (n = 1) Internationally (n = 1); a local community organization (n = 3); or another organization in Canada (n = 1) At the University of Manitoba (n = 11); government laboratory (n = 1); or local community organization (n = 1) a Must be a citizen of a commonwealth country. b Completing graduate degree at the University of Manitoba. c Travel could be split in 2 trips to meet the 90-day requirement. d Allowable monthly stipend by the funder increased mid-way through the program.

INQUIRY
i. How did students describe their experiences as they relate to the acquisition of the previously listed skills? ii. How did Canadian and international students differ in their descriptions? iii. How did undergraduate and graduate students differ in their descriptions?

Methods
Our evaluation of the program utilized a mixed-methods approach and the qualitative analysis was informed by theories of experiential learning, as previously described. [10][11][12] Each student was required by Universities Canada to complete a structured narrative report, which included a combination of open-and closed-ended questions. The narrative report was developed by Universities Canada and completed through an online portal directly by the students within 2 weeks of the students' placement end date. Individual institutional programs did not have input into the structure of the report, but it was approved for our use in this evaluation by Universities Canada. Notably, students did not provide their individual informed consent for our institutional analysis, as such we elected not to utilize direct quotes in our analysis. This study was approved by our institutional Joint Faculty Research Ethics Board. First, we conducted a descriptive analysis of the likertscale, closed-ended questions using Microsoft Excel. Students could select strongly disagree, disagree, neutral, agree, or strongly agree for the following statements: my QES experience has strengthened my knowledge and skills, • To provide feedback and support presenters in improving their research plans. • To practice positive and supportive criticism skills.
Formative: Student feedback to peers on presentation Summative: Student presentations and final paper/proposal a Midway through our program, we waived the requirement for students completing summative assessments, due to curricular limitations in completing the course for credit; though all students were required to receive credit in some way. increased my leadership skills, improved my communication skills, improved my networking skills, enhanced my personal/professional networks, and has allowed me to increase my awareness of the challenges faced by local/global communities. As a follow-up to each likert-scale questions, students were able to comment and explain their selection. Students were also asked to list their goals at pre-departure and then reflect as to whether they did not meet their goals, met their goals, or exceeded their goals. Again, there was additional space for students to comment further on meeting or not meeting their goals. Second, we completed a qualitative, thematic analysis of all the open-ended questions on the students' narrative reports. Beyond what was asked as follow-up to the likertscale questions, students were asked to reflect on the most important lessons learned, as well as share a success story and explain why it is a success story. Initially, the data were grouped according to the categories community engagement, success, challenges, and project description, which reflected to some extent the categories of questions. From these initial groupings, the first and second author, and a student research assistant examined the data and developed a code list with corresponding description of each code. Student narrative reports were then uploaded to NVivo version 12 and analyzed with the corresponding code list using a thematic content analysis. 15 The first author completed the remaining data analysis with frequent communication and verification regarding the collapsing of categories and emerging themes with the second author. Further analysis to compare answers between Canadian and international students, as well as undergraduate and graduate students was also completed.
Throughout the analysis, the first and second author were reflective of their roles as faculty lead and instructor, and coordinator of the program. The third author joined the program advisory group in 2018 and being more of an outsider to the program had the role of providing external evaluation supervisory support for this evaluation, given experience in educational program evaluation.

Results
A total of 38 students were funded by the program between January 2016 and September 2018. This included 7 Canadian graduate students, 13 Canadian undergraduate students, 13 visiting international graduate students, and 5 international graduate degree-seeking students enrolled at the University of Manitoba. Two Indigenous students were funded and 9 (24%) of the placements were Indigenous health-focused.
The majority of students reported that they met their goals (53%) or exceeded their goals (42%). The results from the likert scale questions are summarized in Figure 2. Results for all skills were particularly positive for undergraduate students in the program such that 62% reported that they exceeded their goals, as compared to 42% for the whole cohort. Similarly, scores for all skills were typically more positive for visiting international students as compared to either Canadian graduate or degree-seeking international graduate students.
Three overarching and inter-related themes emerged in our qualitative, thematic-analysis of students' narrative reports, including success through relationships and new perspectives, challenges of the unfamiliar, and personal growth through strong emotions. All themes were more prominent for undergraduate and visiting international students, partly mirroring the richer reflection the same groups put into their narrative report.

Success Through Relationships and New Perspectives
While some students were placed in academic institutions with related partnerships with community organizations, others were placed directly with community organizations. However, students discussed learning from community irrespective of their primary placement, either at a university or community organization. Learning from outside the academy contributed to students' confidence, particularly with respect to communication, networking, and leadership skills, summarized in Figure 2. The majority of students identified community engagement as a positive experience through which they had gained new perspectives.
In discussing their own successes, new "relationships," "friendships," "teams," and "networks" in their placements and with other students in the program were at the fore-front. Many students reported relationships as their greatest success and linked this with new perspectives and awareness of how different contexts shape their understanding of health issues. This led many students to embrace skills in patience, flexibility, and adaptation. The volume of learning was an overwhelming success for some, indicating a strong intrinsic motivation to learn, while a minority of students cited more tangible outcomes of success, such as a publication or news article about the research.
Given the known challenges of international work, the program coordinator and academic lead purposely designed several undergraduate placements and visiting international student placement in pairs, or in 1 case as a group of 3. This was strongly reflected in these students' narrative reports, and as a resounding success from their perspective. Having another student whom they were familiar with and in some cases, working on a related project, allowed students to engage in reciprocal reflection, peer mentorship, and psychosocial support. This likely contributed to the more positive learning outcomes reported by the undergraduate and visiting international students overall, as well as to buffer the challenges described in the next theme.

Challenges of the Unfamiliar
Most students experienced some challenges and for a minority the challenges overtook their experience. For example, some students were working in low resource settings and/or rural areas, which contributed to frustration, loneliness, and isolation. Many students initially experienced an "adjustment period," which for some was strongly linked with travel-related fatigue. Cultural differences in working environments were a barrier for some Canadian students. While these challenges were noted, students' ability to overcome them contributed to the feelings of success previously described. The ability to overcome challenges was seemingly tied to students' opportunity and/or ability to network and develop relationships in their placements, as well as having an easygoing, open-minded attitude and growth mindset with respect to some challenges. The lack of mentorship and/or relationships was mentioned as a challenge and seemed to be related to the most negative learning experiences. In this way, there was variation in terms of how challenges were perceived by students, as some challenges were considered positive (even if initially considered negative) and others wholly negative.

Personal Growth Through Strong Emotions
Tied to the strong emotions felt during challenges, including workplace challenges, was an overwhelming description of positive emotions and personal growth. Many students, particularly visiting international students and undergraduate students, were incredibly grateful for the experience. They reported increased confidence from the program selection, mentorship support, the development of networks and relationships, and the opportunity to travel somewhere they have never been. Many students, both Canadian and international, described the program as "life-changing." The program motivated several students to pursue graduate studies in the area of global health, which was partially the result of exposure to graduate students in the online course. The personal growth described also motivated students to pursue additional experiential learning opportunities abroad (including 2 international visiting students who returned for a second placement), and/or support QES students in subsequent iterations of the program; this linked the themes personal growth to success through new relationships.

Lessons Learned
We have documented the focus and evaluated student experiences and learning outcomes of an experiential learning program centered on community engagement in global and Indigenous health research settings. Many of our findings align with previous evaluations and support the global movement toward increasing experiential learning opportunities in public health training, 8,[16][17][18][19][20][21] formal education in community engagement, 6,16,20,22 interprofessional learning in public health, 19,23 and in international education experiences, generally. 17,24 Throughout the discussion reflections of directing, teaching in, and coordinating the program have been integrated in making meaning of the students' experiences, further documenting the lessons learned, and finally, providing recommendations for other global and Indigenous health experiential learning programs (Table 3).
A major strength of the program was its flexibility, including different eligible countries, disciplines, research methods, graduate and undergraduate students, Canadian and international visiting students as well as degree-seeking international students. This flexibility allowed students to work in settings/projects of interest to them, and learn from each other, in addition to their own respective placements and research, which was further facilitated through the online course. In turn, the diversity in the virtual classroom further supported inclusion and safe spaces for students to reflect on their learning, which is a key component of experiential learning models. 11 The role of graduate students in mentoring undergraduate students in the online course was noted as beneficial by the undergraduate students, though interestingly not by the graduate students. It is possible graduate students, many of whom were travelling for thesis-related research, were experiencing academic fatigue. Little attention has been paid to the value of peer mentorship in public health education despite its documented success in other disciplines. 25 Indeed, peer mentorship was integral to many students' success and the program's success through recruitment via word-of-mouth, and support for students in subsequent cohorts. We recommend that public health education programs, in general, facilitate greater opportunities for peer mentorship.
Another key finding in our evaluation was the importance of an open-minded and inclusive attitude to a successful student learning experience as much as technical knowledge and skills. Similarly, Frenk et al 8 list "common attitudes, values and behaviors" as priorities in their list of 10 major educational reforms required to train the next generation of health professionals. We acknowledge that students who self-selected into our program were more likely to possess Table 3. Lessons Learned and/or Recommendations for Experiential Learning in International Settings.
A. Administration

Institutional
• Faculty interest in student supervision was limited. Institutional commitment to incentivize mentorship for experiential learning is needed to maximize program success. • Institutional requirements for visiting students, particularly for English language requirements for visiting students, was a barrier to reciprocal exchange for partner institutions. We recommend waiving the English testing requirements for visiting students, as we did for our program, provided students had a working knowledge of English.

Program
• We experienced student recruitment challenges initially for Canadian students. We recommend program promotion through student alumni experiences by engaging with the institutional communications office. • Student supervision by community partners and host institutions, particularly in the Global South could be burdensome. It is important to negotiate the student supervision carefully, with consideration to the hosts' capacity to supervisor. • In order to maximize partner benefit, we offered reciprocal exchange for students of partner institutions with great success.

Funder
• The funding structure was inflexible such that funding was mostly limited to student scholarships, which limited funding for administration and any funding for partner host institutions. This contributed to asymmetrical power relations between institutions, community partners, and partner institutions (particularly in the Global South). More flexibility in how specific institutional programs can administer funding would allow for more equitable partnerships. • Scholarship amounts were fixed such that students received identical amounts regardless of the host country costs of living.
This was particularly challenging for students interested in Indigenous health placements in Australia or New Zealand and was a barrier to equitable program access. Some students were successful in receiving additional scholarship funds; program support for obtaining additional funding is recommended. • Funder requirements dictated the need for student to receive formal credit for their participation. This requirement, combined with limited curricular space, stringent course requirements in many program, and course timelines, created challenges for many students' participation. Again, greater flexibility on the part of the funder would allow for more students to "fit" an international, community-based experience in their program. B. Curriculum • Given the unpredictability of community-based work and the need to work with partner timelines, the program requires flexibility in scheduling of travel and course offering, within reason. Ideally, the online course offering would coincide with student travel to maximize reflexive practice; however, on a few occasions this alignment did not occur due to inflexibility of curriculum and also partner availability. C. Pedagogy • The broad disciplinary background of the students is a considerable pedagogical strength. Given the research foci of the instructors and departmental units, course readings disproportionately focused on infectious disease. Students requested more diverse readings to engage a greater number of students in content relevant to their individual interests. • Course discussion is optimal with approximately 5 to 9 students to allow for sufficient dialogue, space for critique, and feedback.
Fewer students resulted in less exposure to different research/program topics and perspectives. Too many students results in the development of more parallel discussion threads, thereby minimizing exposure to topics outside students' direct interest but allowing for exposure to more diverse perspectives.
these attitudes and values prior to their experience. However, we did find that personal transformation was also a key finding. We attribute this personal transformation to the experiential learning aspect of the program, as also reported by Sabo et al, 26 but also the psychosocial support students received through mentorship and relationships to overcome challenges. More educational research is needed to disentangle these relationships to examine the value of experiential learning among students with a more narrowminded attitude initially and explore how to effectively teach "attitudes." Every student dealt with challenges, though their responses to challenges varied, which was partially dependent on the attitudes previously discussed. While cultural differences were sometimes viewed as an opportunity to reflect on ones' own cultural assumptions, other students experienced frustration at differences, for example, in differential cultural understandings of time. Challenges coping with different cultural contexts in experiential learning settings have been previously documented. 27 There are ethical considerations to consider in this regard, as outlined by Gough et al 14 ; for example, student privilege and entitlement, shallow or superficial student reflections, as well as students perpetuating stereotypes on-site. The course instructor (first author) attempted to normalize challenges as part of the learning process as that can be an indication of values being challenged. This helped some students remain positive despite also feeling frustrated, thus building confidence in students' own ability to solve problems and cope. However, some challenges did require intervention from a programmatic standpoint and it is important to be able to differentiate what can be a fine line between student learning and creating a negative experience.
While student experiences and learning were generally a positive outcome of the program, we recognize that an institutions' ability to offer such programs may be limited as these programs are extremely resource-intensive. It was clear that success, as perceived by students, was directly dependent on mentorship support and supervision, which is labor intensive. During this time period, our program received nearly $300,000 in funding and we documented over $500,000 in university cash and in kind contributions, mostly attributed to coordinator salary, and faculty and community member workloads for student supervision. The recommendations listed in Table 3 also correspond to, mostly, faculty member workloads. Engaging faculty as mentors was a challenge for the program and in some cases, contributed to negative experiences for students. Limited faculty buy-in for experiential learning has been noted by others, 28 and may be particularly difficult at research intensive institutions, 12 such as in our case. As Frenk et al 12 noted in their description of updating the public health curriculum at Harvard, we must "forge the identity of research-teacher. . .into a single role." As is, the current academic climate in Canada does not support this. In our institution the majority of Indigenous and global health researchers are practicing physicians or salary award holders. We need to consider the way the current career reward structure frames research and teaching as mutually exclusive. Just as institutional culture and systems changes were, and still are, required to integrate community engagement into valued academic pursuits, 9,29 so too is institutional change required to fully value experiential learning opportunities in community-engaged research and practice.
As highlighted in the recent Lancet special issue on Women's Health, increasing the number of women in leadership roles in global health is of paramount importance. 30,31 While our high numbers of women trained is positive, we acknowledge that our numbers do not tell the whole story. A barrier for several women trainees, particularly Indigenous trainees and including those who were not able to participate, was the challenge of delegating care-taking responsibilities during a 3-month placement. This was compounded by the comparatively low level of funding for students who sought Indigenous-focused placements in Australia or New Zealand, given the higher cost of travel and living. As also noted in the Lancet special issue, sexual harassment and assault is rampant in the global health field, 32 with 55% of women in a recent survey reporting sexual harassment and/or assault during their training. 33 While more needs to be done to address this in academic institutions globally, including our own, it is crucial for any international experiential learning program to be prepared to receive disclosures of sexual harassment and assault.
There are a number of limitations to this evaluation. First, only informal feedback from host institutions of student placements was received. A formal evaluation was thought to be a burden on host institutions. Nonetheless, the first author was successful in receiving additional funding for the QES Program in 2018 based on the continued support of most international partners. Second, this evaluation was based on a structured narrative report. A formal open-ended, in person evaluation may have resulted in a greater depth of inquiry into student learning. Given the nature of open-ended survey questions, responses varied considerably in their length and depth. While the themes described in this paper reflect the data collected, data saturation was not reached, and in particular there was a divide in terms of how students perceived different challenges in relation to learning. However, it was felt that the existing application, orientation, and reporting structure already placed a heavy respondent burden on students given the funding amount. Furthermore, the faculty lead and coordinator were in regular contact with most students such that a great deal of informal feedback was received throughout the program. Third, the narrative reports were mandatory and not anonymous. Students may not have felt comfortable relaying negative feedback regarding the program to either the funder or program management. Moreover, the reports were completed in most cases prior to the conclusion of the students' respective placements, usually within the last week. It is possible that students' perceptions of their learning may have changed, either positively or negatively, as a result of a period of reflection following the conclusion of the placement.
In conclusion, experiential learning in global and Indigenous health research is a valuable pedagogical approach in community engagement, particularly for undergraduate students. However, international experiential learning requires considerable financial and human resource commitments to ensure its success, as well as mitigate equity issues. This evaluation can inform ongoing pedagogical advances in public health education to further the development of interdisciplinary, engaged public health leaders who can address emerging public health challenges in new ways. Furthermore, greater consideration for community-based and peer learning in public health pedagogy should be considered, regardless of the international context. Further evaluation is needed to examine how global experiential learning opportunities prepare trainees as they enter, or reenter, the public health workforce.