COVID-19: A MOTIVATOR FOR CHANGE IN ENGINEERING EDUCATION?

Despite recent research and initiatives, learner-centered instructional practices have not made their way into post-secondary Science, Technology,Engineering and Math (STEM) classrooms, even though there is clear evidence showing the benefits include increased grades, higher student engagement, and deeper learning. STEM educators rank the barriers associated with active learning higher than their colleagues in other disciplines, and identify the inability to cover all the content as a key factor in their decision to adhere to didactic practices. Insights and instructional strategies and methods garnered from teaching-related faculty development opportunities are often tried, but their use is not generally sustained unless a personal experiencedrives that change in practice.  Unquestionably, COVID-19 has had an immediate, global impact on higher education. Educators have been forced to alter their teaching practices to accommodate the switch to remote learning. Most Teaching and Learning Centers offered myriad workshops to facilitate this change. This quantitative study set out to determine if COVID-19 precautions created the personal experience necessary to initiate a change in STEM teaching practices. Using educator-related threshold concepts as a framework, it analyzed institutional registration records to determine the type of faculty development opportunitieschosen by engineering educators, and the extent to which they participated in those related to learner-centered instructional practices for remote delivery.Analysis shows that engineering educators participated proportionally less than their colleagues in other disciplines, and there is an indication that the pandemic may facilitate an ongoing change in the teaching practices of engineering educators.  Opportunities for enhancing faculty development practices for engineering educators are proposed.


INTRODUCTION
The global COVID-19 pandemic has had an immediate impact on higher education. Most educators were forced to change from in-person classes to some form of remote delivery. Initial expectations were that it would be a brief and temporary change, but as pandemic restrictions continued into a second and then third teaching semester, faculty realized that the emergency transition was becoming the norm.
Prior to the pandemic both student and faculty surveys of student engagement ranked engineering educators lowest in the categories of effective teaching practices and providing a supportive learning environment. This may be in part due to the reluctance of STEM instructors to adopt and sustain learner-centred instructional practices unless a personal experience drives a need for change. With the majority of STEM educators personally experiencing a transition to remote delivery, this research sets out to determine if this is enough to motivate a change in the level of participation in educational development opportunities and, by extension, the learning experience of their students.

BACKGROUND
Engineering educators seem reluctant to bring the findings of education-based research into their classrooms. In an effort to maintain their traditional teaching practices, educators limit their ability to master the threshold concepts associated with post-secondary teaching. This section describes the pre-pandemic teaching practices of undergraduate engineering educators, and identifies the key teaching-related threshold concepts.
Prior to the pandemic, benchmarks used in international student and faculty engagement surveys ranked the undergraduate engineering experience lowest among the disciplines [1] [2][3] [4]. Three key aspects were ranked lowest by both students and their instructors: learning strategies, effective teaching practices, and a supportive environment [5] [6]. While other disciplines seem to recognize the need to adjust their teaching University of PEI; June 20 -23, 2021 -2 of 6 -practices, engineering remains one of the most traditional and didactic disciplines in higher education [7].
Many instructors believe the demands of an engineering program are best accomplished using triedand-true methods [8], even though extensive educationbased research in Science, Technology, Engineering and Mathematics (STEM) shows that learning improves when students are more actively involved during class [9][10] [11]. As a result, typical undergraduate engineering classes remain lecture-based, and fewer than 20% include even one or two student-centered instructional strategies [7].
The reluctance to incorporate evidence-based teaching practices may correspond to the challenges associated with teaching-related threshold concepts. Meyer and Land define a threshold concept as a portal, a way of thinking about something in a "new and previously inaccessible way" [12]. It is a space between where a learner currently is, and where that learner needs to be in order to become an expert in their discipline. Mastering threshold concepts is individualized; a learner may master some threshold concepts quite easily, while their peers struggle. That same learner may struggle with other threshold concepts, placing them in a transitional state called liminality. This state can involve ambiguity and disorientation as the learner fluctuates between a state of knowing and not knowing.
A threshold concept is defined as having five key characteristics: it is (1) transformative, (2) troublesome, (3) irreversible, (4) integrative, and (5) bounded. Land describes these in a 2013 interview by saying [13]: "New conceptual understandings pull together (integrate) various stands of understanding into a new understanding that fundamentally changes (transforms) the way students think about the subject. Because the process involves the loss of a familiar way of thinking and the security that provides, the process of crossing the threshold commonly causes some mental and emotional discomfort (troublesome). There's some debate about whether these new understandings are reversible or not, but once a student "gets it," it seems hard for them to "un-get it." "Bounded" seems the least obvious descriptor perhaps because its meaning derives more from contextual concerns than specifically conceptual ones." Many post-secondary educators know that there are discipline-specific threshold concepts that their students must master, but do not recognize that there are also threshold concepts associated with teaching that must be mastered in order to become effective and proficient instructors. A systematic literature review identified three categories of threshold concepts associated with postsecondary teaching: (1) pedagogy, (2) learning, and (3) assessment [14]. These threshold concepts align with, and can help address, the three low-ranking student engagement factors previously discussed. Educators who master the pedagogy threshold concepts will improve the effectiveness of their teaching practice. Those who master the learning-related threshold concepts will make instructional decisions that strengthen student learning strategies, and those who master the assessment threshold concepts will provide a more supportive learning environment. See Figure 1 for a view of the progressions within each threshold concept.

Figure 1: Teaching-Related Threshold Concepts
The pedagogy category encapsulates two threshold concepts, the development of teaching knowledge, and the notion of subject-and discipline-specific threshold concepts. Shulman proposes that there are four types of teaching knowledge that an educator must acquire to become a proficient instructor [15]. It is generally assumed that educators begin their careers with subject matter or content knowledge (SMK), the first type of teaching knowledge. Instructors must have a clear understanding of the concepts and principles of their discipline and how they relate to professional practice. Pedagogical knowledge of teaching (PK), the second type, is a grasp of the general principles of teaching. This includes, but is not limited to, classroom management as well as class planning and delivery.
Pedagogical content knowledge (PCK), the third type in Shulman's model, is the ability to organize, represent, and convey discipline-specific knowledge and skills in a way that facilitates student learning. It requires educators to recognize bottlenecks in student learning, to appropriately scaffold the development of concepts, and to help students reorganize their knowledge base when they have misconceptions. Curricular knowledge (CK) is the final type of teaching knowledge. It requires educators to recognize that the way specific topics can, and should, be taught depends on where and when it appears in the program of study. It also involves a grasp of lateral and vertical curriculum knowledge, connecting what students learn in prior, concurrent, and future courses. University of PEI; June 20 -23, 2021 -3 of 6 -An instructor who has mastered the threshold concepts associated with these four types of teacher knowledge is one who will provide a pedagogically-sound learning experience for their students.
The second pedagogy-related threshold concept relates to threshold concepts themselves. Engineering tends to be a content-driven discipline where educators cover a lot of material at a very fast pace [16]. Instructors who have mastered this threshold concept are those who research and/or identify the topics that are troublesome and transformative for their students. They emphasize these difficult concepts and use PCK to ensure students are able to make sense of, and apply, these critical elements.
The two learning-related threshold concepts are providing a student-or learner-centered experience, and transitioning to an active learning environment. Student success is "informed by a deep understanding of the learners, along with their active involvement in selecting solutions that work for them" [17], and it is important that educators appreciate how students learn and what motivates them to do so. A student-centered environment considers what and how students approach learning, while a learner-centered approach offers a more personalized and holistic pathway to success. Educators who have mastered this threshold concept are those who make instructional decisions based on evidence about how students learn, and an awareness of the learners in their class.
Chi and Wylie's model of cognitive engagement (ICAP -Interactive, Constructive, Active, Passive) can help educators transition to a more active learning environment [18]. Passive learning, the least active of its four modes of engagement, is often found in lecturebased, didactic classrooms. Its transmission style is associated with the cognitive process of storing information. The active mode has students manipulating what they are learning. This is still considered a shallow engagement strategy but helps learners store, activate, and link new concepts to prior knowledge. In the constructive mode, students are challenged to engage more deeply. They generate new ideas or products that go beyond the materials provided during instruction. This helps learners add inference to the activation, linking and storage of concepts. Finally, interactive learning, the most active of the engagement strategies, requires learners to collaboratively support each other's learning, adding infer-from-others to the cognitive process. Educators who have mastered this threshold concept are those who challenge learners to actively and deeply engage with the concepts and skills they are expected to acquire.
The assessment category includes two threshold concepts: the transition from summative evaluation to an approach that provides ongoing feedback to students as they learn, and a move from formal examinations to more authentic and high-impact assessments. It is common for the majority of students' grades to be generated from summative assessments, but this provides learners with little feedback about what and how to adjust and improve what they're doing. Educators who master this threshold concept are those who plan for and provide regular assessment-based evidence to their students.
Summative assessments often take the form of tests or examinations which are rarely representative of the way in which students will apply knowledge within their discipline. The authenticity of high-impact activities and assessments deepens learning and helps improve student engagement, persistence, and satisfaction. Instructors who have mastered this threshold concept are those who regularly incorporate authentic, high-impact assessment practices into their curriculum.
The pandemic-induced shift to remote delivery challenged educators to adapt their teaching practices. Most Teaching and Learning departments offered myriad educational development opportunities to support this transition, but often the instructional strategies that educators learn and try are not sustained unless a personal experience or policy drives that change in practice [19] [20].
This research examines educational development opportunity choices made by engineering educators during the first nine months of the COVID pandemic. It compares their participation to that of educators in other disciplines, and analyzes whether those professional development choices align with teaching-related threshold concepts.

METHODOLOGY
This study reviewed the registration records for all institutional professional development opportunities offered at a large Canadian polytechnic school from March 16, 2020, the first day of COVID-19 lockdown, until November 15, 2020. Workshop topics fell into two distinct types, those designed to help transition from inperson to remote delivery, and those designed to support teaching and learning in any form (in-person, remote, hybrid, blended, or HyFlex). Multiple offerings of each workshop were advertised and made available to all educators across all disciplines. All were delivered remotely, and most were 1.5 hours long.

RESULTS
137 different workshops were offered during the studied period of time, 80 of which were added in direct response to COVID. Most workshops were offered multiple times, with 519 individual learning opportunities available, 348 of which were in response to COVID. Table 1 shows the distribution of these workshops over the three terms. Table 2 shows the number of registrations by discipline. Of the 12,545 registrations, 1192 were by those in the faculty of engineering (9.5% of the total University of PEI; June 20 -23, 2021 -4 of 6 -registrations) and 11,353 by instructors in other disciplines (90.5% of the total registrations). Engineeringrelated educators make up 20.2% of the overall faculty population.   Figure 2 shows the workshop registration numbers for each of the three terms, noting that the winter and fall graphs represent only one half of the total number of weeks represented in the spring term. There was an assumption during the winter term that the COVID lockdown would be temporary (1925 registrations, 117 from engineering), and instructors sought out what they needed to learn to quickly transition from in-person to remote delivery. Most seemed to recognize the need to move beyond emergency remote delivery when restrictions continued into the spring (9042 registrations, 999 from engineering), and by fall seem to have settled into a new "normal" (1578 registrations, 76 from engineering).

Figure 2: Registration by Term
A Kruskal-Wallis H test shows a statistically significant difference in engineering faculty registration numbers in workshops associated with teaching-related threshold concepts, χ 2 (5) = 14.485, p = 0.013, with a mean rank participation of 54.66 for workshops not related to threshold concepts, 81.13 for workshops related to knowledge types, 77.79 for workshops related to cognitive engagement, 62.40 for those related to the classroom environment, 110.00 for those related to the authentic assessment, and 62.43 for those related to learning through feedback. There were no workshops related to pedagogical threshold concepts.
Follow-up Mann-Whitney U tests indicate four statistically significant differences in registrations. Faculty registrations per non-threshold concept-related workshop (µ = 9.18) were lower than those in workshops related to knowledge type threshold concepts (µ = 14.13, p = 0.009), and those in workshops related to authentic assessment threshold concepts (µ = 26.00, p = 0.015). Faculty registrations per authentic assessment threshold concepts (µ = 26.00) were higher than those in workshops related to cognitive engagement threshold concepts (µ = 9.12, p = 0.029), and those in workshops related to learning through feedback threshold concepts (µ = 6.29, p = 0.030).
A total of 716 registrations were made by engineering faculty in 78 unique workshops that were not related to any of the threshold concepts associated with postsecondary teaching. The focus of these workshops ranged from academic integrity to using online tools like Zoom and Microsoft Teams.
There were nine unique workshops related to the pedagogical threshold concept associated with knowledge types: Collaborative Learning in the Cloud, Action Research, Best Practices in Teaching, Lesson Planning for Remote Learning, and those related to using educational technology and applications in the classroom. Registrations from the engineering school numbered 113.
A total of 155 engineering faculty registrations were made in the 17 workshops related to the learning-related threshold concept associated with cognitive engagement: Bring Discussions Online, Psychology & Learning, Balance Student Learning, and the rest related to developing interactive learning objects and applications to engage learners.
Three unique workshops were offered that related to the authentic assessment threshold concept: Alternatives to Traditional Assessment, Alternatives to In Person Assessment, and Choosing Assessments for Remote Delivery. There were 78 registrations for this group of workshops from the engineering faculty.

DISCUSSION
The results of this study offer unique insights into the effects of a forced change in instructional practices. The

Proceedings 2021 Canadian Engineering Education Association (CEEA-ACEG21) Conference
CEEA-ACEG21; Paper 159 University of PEI; June 20 -23, 2021 -5 of 6 -analysis, when viewed through the lens of teachingrelated threshold concepts, highlights two key findings: 1. engineering educators took less advantage of the opportunity to learn new concepts and skills than their colleagues in other disciplines 2. there is some evidence that the pandemic triggered a willingness on the part of engineering educators to explore aspects of their teaching practices associated with pedagogical, learning and assessment related threshold concepts There was a common perception prior to the pandemic that engineering educators participated in fewer faculty development opportunities than their peers in other disciplines, especially those that focus on improving teaching and learning (instructional development). This has proven to be true during the first eight months of the pandemic. Although engineering educators represent 20.2% of the overall population, they made up only 9.5% of those who registered.
Reasons for this low participation may include a lack of incentive and a perceived lack of relevance. Felder and his team [21] report that in general many educators believe that the quality of their teaching has nothing to do with low ratings or poor performance of their students, so there is little incentive to improve or change the way they teach. With instructional development workshops most often delivered by social scientists, engineering educators don't see the content as relevant to their courses, subject, students, or problems.
The Felder study goes on to say that engineering educators are more likely to be attracted to, and influenced by, instructional development programs designed specifically with them in mind and delivered by a teaching expert with an engineering background. This appears to be the case for the studied engineering faculty, with 55.1% attendance at multiple offerings of one workshop tailored specifically for the engineering school, and delivered by an engineer/teaching expert. The findings of this study support Felder and team's recommendation that development opportunities for engineering educators be led by facilitators with pertinent content knowledge and experience.
The second key finding of this study suggests that the pandemic had a significant effect on the instructional development of engineering educators. They learned what they needed to survive and transition to a temporary teaching venue, and showed interest in exploring ways to adapt the way they teach and evaluate students. This aligns with findings of a global survey on the impact of COVID-19 on higher education [22] that reports the shift to remote delivery built faculty capacity in the use of new tools and systems need to deliver classes online. It suggests that there may be a "shift in mindset" among post-secondary educators opening them to a "new horizon of opportunities for teaching and learning". This may lead to an increase in innovative teaching pedagogies.
We believe that the data used in this study represents the pandemic-induced faculty development response at most Canadian post-secondary institutions, but recognize they are not statistically representative. The findings of this study may have a bias associated with confounding factors since it is unknown what specifically prompted or limited faculty development choices during the study period. This may limit the generality of the results of this study. Consequently, the findings should be considered in the context of the limitations of this study.

CONCLUSIONS AND FUTURE WORK
This research supports and extends the findings that engineering educators participate in fewer instructional development opportunities than their colleagues in other disciplines. It also hints that there is a willingness to explore evidence-based teaching practices. Only time will tell whether the pandemic-induced participation in instructional development opportunities related to pedagogy, learning and assessment threshold concepts will result in sustained changes in the teaching practices of engineering educators.
This research lays the foundation for further study. Work could be done to compare the pre-and postpandemic teaching practices of engineering educators. Studies could also be done to determine the effect that discipline-specific instructional development has on participation rates and/or on teaching practices.
It is hoped that adding the findings of this research to the existing body of evidence on the instructional development of engineering educators will encourage those responsible for their educational development to recognize the need for discipline-specific opportunities that support mastery of teaching-related threshold concepts in the areas of pedagogy, learning, and assessment.