Assessment of the impact of COVID-19 on honors student learning, institutional connections, and intent to return to campus

The COVID-19 pandemic quickly converted classes to an online format in the middle of the academic semester at South Dakota State University (SDSU), USA. Our objectives were 1) to identify factors affecting student learning and connectivity following this transition and 2) to evaluate differences between honors and non-honors students. Students (n=230) were surveyed with Likert-type, descriptive, and open-ended questions about their experiences following the transition. Clear, frequent communication between students and SDSU was identified as the most appreciated aspect of SDSU’s response. Students who reported struggling academically following the transition were more likely to be facing difficulties with finances and access to or use of online learning technology. Honors students reported fewer technology barriers and financial stressors than non-honors students. Degree completion and social connections were driving the desire to return to face-to-face classes, but this enthusiasm was dampened by COVID-19-related health concerns. Communication, structure, and flexibility were identified as factors affecting student success.


Introduction
In the wake of COVID-19, higher education institutions have been faced with unprecedented challenges. In spring of 2020, students, faculty, staff, and their families transitioned to an online environment abruptly and faced negative impacts of transition. Past outbreaks of disease, such as MERS and SARS, brought with them concerns of impacts on financial, mental, and physical health, and quality of teaching and learning in distance environments (Al-Rabiaah et al., 2020). The 2020 COVID-19 pandemic is no exception. Assessment of student perceptions of this pandemic and their ability to pursue academic success while maintaining connections to their institution are valuable in understanding how universities can better prepare for future disruptions.
A growing concern in colleges, especially among honors students, is mental health (Pham, 2017). COVID-19 and the transition to distance education has introduced new and dire concerns. A study of Chinese university students estimated that 25 percent experienced new anxiety resulting from COVID-19 (Zhai & Du, 2020). Living alone, altered or limited access to counseling and support services, disruption of academic environment, degree completion delays, and family or friends falling ill are all concerns that increase students' risk of developing anxiety disorders (Cao et al., 2020;Zhai & Du, 2020). Also, many students have experienced cancellations or postponements of internships, jobs, and post-graduation plans. According to crowd-sourced data from Canada, up to 86 percent of students that experienced these cancellations reported worrying about their financial wellbeing (Wall, 2020;Zhai & Du, 2020).
Distance education is a commonly debated practice in honors education. Under normal circumstances, distance learning is rare in honors education, as it seems to go against common practices of hands-on, interactive, engaging curricula. Supporters of online education assert that quality honors education can be achieved through proper faculty training, planning, and support (Nightingale, 2014;Zubizarreta, 2020). However, the sudden transition to distance education due to COVID-19 was anything but planned. Emergency online teaching came abruptly and left many faculty members without sufficient support, for existing infrastructure was not designed to support full-time online instruction. While typical online courses take six to nine months to plan, the sudden shift was completed in only a few weeks (Hodges, Moore, Lockee, Trust, & Bond, 2020). Thus, neither faculty nor students had the benefits of structured planning for success in online teaching and learning.
The purpose of the current study was to understand the post-COVID-19 experiences of our students with the intent of using their feedback to refine and improve possible hybrid and online transitions for future courses which are traditionally offered face-to-face. Our objectives were to determine: 1. Factors affecting student learning during COVID-19 2. Whether students continue to feel connected to SDSU 3. Key factors impacting students' decisions to return to campus next year 4. Differences among honors and non-honors students While the current study is focused on the impact of COVID-19 in honors and in higher education, there are broader implications for understanding student preferences and learning ability in online or rapidly changes formats. Honors education is often touted as being nimble and innovative. The results of this study emerged from an opportune moment to collect student insight during a global pandemic and one that caused significant and sudden changes in educational formats. If honors education is to remain relevant, curricula and pedagogies must be nimble in the face of acute change and innovative in designing ongoing positive shifts in higher education.

Survey design and distribution
This survey has been adapted with permission from the © 2020 Higher Education Data Sharing Consortium (HEDS). HEDS is an independent, not-for-profit consortium of higher education institutions that aims to improve undergraduate education by serving as a platform for shared data and knowledge. They conduct periodic nation-wide surveys on student success, diversity, research practices, and campus climate. By adapting the survey from the HEDS, this project was able to incorporate certain questions that were already tested and utilized at the national level. The research team obtained approval under the SDSU Center for the Enhancement of Teaching and Learning, Scholarship of Teaching and Learning Umbrella IRB (Institutional Review Board). The QuestionPro survey included 22 substantive and five demographic questions (see Appendix A). Among the 22, seven were descriptive, six were open ended, and nine were Likert-type questions. The survey was delivered via email to all 2,023 students enrolled in the Honors College, the School of American and Global Studies, affiliated with the ADA (Americans with Disabilities Act) or multicultural centers, and to those enrolled in one of our TRiO programs. These are federally mandated programs that aim to provide a supportive and welcoming university experience to less advantaged students, including minorities, low-income, and first-generation college students. This distribution allowed us to not rely exclusively upon the honors students. The survey was delivered during the first week of May when final examinations were in session and remained active for three weeks. During this period, 573 students viewed the survey, and, of those, 230 (40 percent) fully completed it, yielding an overall response rate of at least 11.4 percent. The response rate could be higher, since some students belonged to more than one of the targeted groups the survey was sent to, and the list was not vetted for duplicates. SDSU is the largest and most comprehensive land grant university in South Dakota with over 11,500 students from all 50 states and 83 countries pursuing degrees in 82 undergraduate majors, 36 master's programs, and 15 doctoral programs.

Quantitative analysis
Two separate quantitative analyses were completed. In the first analysis, students were placed into one of four categories based on whether they self-reported struggling academically before and after the transition to online classes during the COVID-19 pandemic: Group 1: Those who struggled before and after COVID transition (n=13) Group 2: Those who struggled only before the COVID transition (n=6) Group 3: Those who struggled only after the COVID transition (n=99) Group 4: Those who did not struggle before or after the COVID transition (n=123). The above sample exceeds 230 students who completed the entire survey, because students who did not answer all questions but answered this particular question were included in this analysis. Because of the small sample size for the first two categories, they were not analyzed further in this first quantitative analysis only. A t-test statistic was used to compare responses to 41 questions between Group 3 and Group 4 with R software. The number of questions exceeds that reported above because most of the 9 Likert-type questions asked for feedback about more than one factor that could affect a student's experience. For example, one Likert question asked about time spent on different tasks (e.g., studying, working) and each task was treated as a different question in the quantitative analysis. A P value < 0.05 was our threshold for statistical significance; however, Bonferroni-corrected P values are also reported. In the second analysis, honors (n=126) and non-honors (n=116) students were compared with a t-test as described above. As before, the above sample exceeds 230 students because students who answered this question but did not answer all questions were included in this analysis. Students who reported intent to graduate with honors distinction were categorized as honors students and students who did not were categorized as non-honors students.

Coding and thematic analyses
In this project, the intention was to capture and reflect the student experience as accurately and transparently as possible. Therefore, a significant portion of the survey consisted of open-ended questions. When coding these, an inductive coding approach was implemented (Linneberg & Korsgaard, 2019). Instead of imposing a priori categories to the answers, phrases directly used by the students were selected. While the initial round of coding created nearly a dozen categories for each question, in the second round, similar categories were clustered together to generate higher level categories that would help with a more parsimonious analysis. In order to strengthen the reliability of our coding, two team members separately coded the data in the first round. Their results were systematically compared before moving on to the second round. Of the six open ended questions, scores of two team members were only two percent to five percent different from each other in the first round. A third team member also coded five percent of the answers for each of the open-ended questions separately, for a second test of reliability. The coding practices and results proved to be consistent among all team members.
Our thematic analysis was adapted from Braun and Clarke's (2006) method with team members openly discussing themes until reaching consensus. Each team member reviewed the open-ended responses independently prior to gathering to discuss results. Team members discussed potential themes, and two members of the team independently coded student responses based on the initial themes and revisited the themes alongside examples of initial coding. Finally, three team members commenced secondary independent coding, with team members identifying key participant quotes. Students who did not make comments in the open-ended questions were removed from the analysis. Coding from three investigators was compared, and, when two or more were in agreement, the average number of posts a student made within a category was calculated. The percentage of honors and non-honors students responding in each category was then calculated.

Results: Quantitative analysis
The majority of respondents (63 percent, n=126) reported intent to graduate with honors distinction. First-generation college students comprised 18.5 percent (n=37) of our sample. Survey respondents also self-identified as TRiO students (8.5 percent, n=17), living with a disability (7.5 percent, n=15), and student-athletes (8 percent, n=16). Our TRiO programs are designed to increase access, preparation, and success of first-generation college and economically disadvantaged students, whereas athletic teams are a component of many US colleges and universities. Percentages exceed 100 percent because students can self-identify with multiple groups.
The majority of respondents self-reported as white (90.75 percent, n=206), with minorities of American Indian or Alaska Native Asian (3.52 percent, n=8), Black or African American (4.41 percent, n=10), and Native Hawaiian or Pacific Islander (1.32 percent, n=3). Eight percent of respondents identified themselves as Hispanic or Latino/a/x (n=18). These demographics are consistent with demographics of South Dakota State University in the 2019-2020 academic year. Seventy-five percent of respondents also self-reported as female (n=167).
Students who reported not struggling academically before the transition to online classes were divided into two categories: 1) those who struggled academically after the transition (n=99) and 2) those who did not (n=123). Responses to 22 substantive survey questions were different between these groups (P < 0.05; Appendix B). Among these, responses to five questions were highly significantly associated with whether students reported struggling academically after transitioning to online classes: faculty care and concern for the student, satisfaction with support received from SDSU during the transition, concern with performance in online classes, worry about paying bills, and ability to participate in live or synchronous class activities because of other obligations (Bonferroni P < 0.0014). Students who reported struggling also reported decreased satisfaction with communication about the transition to online classes, increased difficulties with technology and internet access, and increased worry about basic living necessities (e.g., food, housing). Post-COVID-19 academic struggles were not associated with use of student services or perceptions of whether certain types of instruction were more or less challenging in an online environment, except for perceptions of group work (P = 0.023).
The abrupt transition to an online environment resulted in students feeling less connected to campus ( Figure 1A). However, the majority of respondents also believed this level of (dis)connection was appropriate given the circumstances (83.56 percent; Figure 1B).

Figure 1: (A) Student reflection on how connected they feel to campus, and (B) level of appropriateness of connection.
Students who reported struggling after the transition to online classes were more likely to believe the level of connection was not appropriate (P = 0.0046). Honors students were not more or less likely to believe the level of connection was appropriate than other respondents in the survey (P = 0.473). Responses to 15 questions (seven questions if using the more stringent Bonferroni correction) on the survey were significantly different (P < 0.05) between honors and nonhonors students (Table 1). Honors students were generally less concerned about economic issues after the transition to online learning, including paying bills, housing, healthcare, and food security. They were also less likely to report technology or accessibility problems. Honors students reported being able to participate in live or synchronous class meetings more often than non-honors students. When asked if students felt 1) more connected, 2) equally connected, or 3) less connected after COVID-19, honors students reported feeling less connected (2.82) than non-honors students (2.69) after COVID-19 (t-test = 1.99; P=0.0475).
Many of the questions where responses differed between honors and non-honors students were the same questions where responses differed between students who reported struggling after transitioning to an online environment as compared to those who did not report struggling after the transition.

Results: Qualitative analysis
The first and overarching theme of teaching and learning stood out in student comments, likely due to the nature of the open-ended questions. Access to and preferences for technology were common, as were those related to health and safety. The final theme was the social and economic impacts of COVID-19. Multiple themes were often addressed in a single response, indicating that the intersectionality of these themes during COVID-19 could either enhance or detract from student success. Thirty students who did not respond to open-ended questions were excluded from qualitative and thematic analyses.

Teaching and learning via technology in the COVID-19 era
Hallmarks of effective teaching and learning prior to COVID-19, such as strong and timely communication, clear organizational structure and expectations, and accommodation of multiple learning styles, continued to resonate for students during the transition. Active learning supplements to recorded lectures were appreciated, provided they did not make undue additions to the pre-COVID-19 workload. Students recognized when instructors were willing to adapt their approaches to meet student needs, including additional or flexible office hours. One student commented, " [To] those that have made necessary changes -given more time during exams, extra help sessions, etc. THANK YOU." Another said, "This has been hell as someone who avoids online courses like the plague, but my professors have really pulled out all the stops to make it work." Synchronous Zoom lectures were both the most and the least favored instructional method ( Figure 2). They provided real-time access to professors and peers, timely feedback to questions, as well as a sense of community and weekly structure. However, for a significant portion of the students, obligations such as work or sibling care made synchronous lectures less convenient. This second group appreciated asynchronous formats due to their flexibility and accommodating nature. Clear preference was expressed for shorter lectures with plenty of visuals over long lectures without much visual stimuli. Shorter recorded lectures "worked much better for learning the material because I could spread out studying which increased my retention of the material." Laboratories, clinicals and other hands on experiences did not work well after the COVID-19 transition; pre-existing problems were exacerbated, and new problems arose. Numerous comments addressed frustration surrounding learning laboratory techniques online, stating it was no longer "the real experience." This quote sums up the student sentiment: "I don't think reading about the labs I was supposed to perform was effective as interpreting the results did not allow for me to understand the test and how it is performed." Figure 2: Student reports on instructional methods (delivery, technology, assignments, and communication from faculty) that worked the best post-COVID transition (blue bars) and those that did not work well (yellow bars).
Students were adamant that the workload associated with classes should not be changed drastically, except to eliminate 'busy work' or assignments that they perceive as having a high time investment to learning value ratio. "Moving home, I have a plethora of other responsibilities to take care of… Lowering academic standards should not happen but lessening busy work should have been done in a time such as this." Pre-professional students were concerned about their level of preparation for the workforce. One respondent said, "My biggest worry is that the education I am currently getting is not the same quality of education that I would have gotten face-to-face. I am most concerned that I will become an inadequate healthcare professional." Others expressed frustration when faculty reduced content and thus the return on investment of their tuition. Students perceived the educational and financial value of online education inferior and felt those should be offered at a discount.
Students craved structure coupled with flexibility in their courses. One student said, "The transition to online was made a lot easier when professors made their expectations clear and maintained good communication, so I felt supported and knew what I was doing." A consistent organizational structure across courses was desired so that information regarding each course could be found in the same location. Even though most universities utilize platforms such as Desire2Learn or Blackboard, significant variation in their use existed among instructors. While some students preferred the steady flow of communication via learning management systems, others felt overwhelmed by the volume of email traffic or struggled to keep up due to lack of connectivity or access to devices. Independently motivated students thrived in transition: "I liked the independence of learning online, like the recorded lectures and [weekly] deadlines... I am pretty good with time management and was able to stay organized enough to hand in all my assignments on time." Synchronous methods were preferred by those craving social interaction since it "preserved the classroom dynamic and interaction" and by those who self-recognized a need for structure: "Online classes are my Achilles heel…but scheduled synchronous classes offered me some structure at least." Some students preferred the familiarity of the lecture setting and opted for live or recorded lectures. Seeing the professor and viewing notes and drawings made by the professor were commonly preferred to a voiced-over PowerPoint where students could hear the professor but had only slides to view. Most indicated that having recordings or notes posted enabled them to review and study better. Students craved a familiar, pre-COVID-19 approach; those from didactic courses felt disheveled if they needed to transition to more self-directed learning. Those who did not have prior experience with online learning were particularly distressed without explicit guidance: "as a new international student I am fully new to online education but most of the teachers went on as if I knew everything and that really gave me hard times understanding what was going on." Degree completion, campus connections, and hopes for face-to-face formats were the primary reasons students plan to return to SDSU in the Fall (Figure 3). While a small group highlighted COVID-19 as their top criterion for making a decision (n=13), an almost equal number of students expressed their strong desire to come back to campus. This response was largely due to their negative views of online learning at home. These students mentioned lack of motivation and used words like "isolation" to describe their situation at home.

Social and economic considerations in the COVID-19 era
The timing of COVID-19 affected university operations during spring break. Many campuses across the US made the decision to remove students from campus housing and closed access to their campuses. Several students noted their appreciation of the process by saying, "I appreciated how SDSU used multiple platforms to inform various groups about what was going on. I am also glad that students did not come back to Brookings after spring break, as that would have brought a larger chance of spreading COVID-19 to the area." University housing returned students strategically to help minimize the risk of exposure for all stakeholders: "When moving out of the dorms, they gave us time to prepare about a week between announcing the move out schedule and the move out days beginning." Meanwhile, students were pulled into many directions. Some had to care for siblings, sick parents, and contribute to household chores.  In times of uncertainty, SDSU students praised the effective communication efforts. Regular email messages and videos from the president helped them feel that the crisis was handled with transparency. Timely emails and updates from the university made students feel in the loop, well-informed, and respected. The second most frequently mentioned factor was the faculty help, support, and care. Students definitely appreciated the outreach of the faculty and the accommodations they provided (i.e. extra review sessions, more time for tests, etc.). Financial compensations and refunds were the third favorite. Students expressed appreciation for their timeliness (Figure 4).
In the coming up months, serious concerns continue to rise on the horizon. Economic issues seem to dominate the students' agenda. They worry about paying for school, having sufficient funds to cover their bills, and expenses. Finding employment during school and the job market upon graduation are frequently articulated concerns ( Figure 5). After the moveout process, another key issue was how the swift transition impacted them financially, emotionally, and mentally. Many highlighted the university's ability to maintain essential services, such as counseling and dining. Adjusting to the "new normal" required new and

Health and safety in the COVID-19 era
Health issues follow the economic concerns. Students worry about large-scale spreading of the pandemic and its consequences for their health, including their mental health. Not having face-to-face classes in the Fall ranks as the third cause of worry for the students, followed by their concern about grades based on their initial experience with online learning ( Figure 5). This group does not seem content about their academic performance after the sudden transitions due to COVID-19 and worry that it may negatively impact their timely graduation. General uncertainty about the progress of the COVID-19 and its negative impact on their social lives, friendships, and connections stand out among the causes of worry. Students who rely on more hands-on forms of learning such as internships, shadowing, and clinicals also express significant concern in the coming months.
Many students expressed concern for their mental health, as well as the health and safety of their family members. COVID-19 brought to light many concerns of underrepresented or marginalized communities: "My parents getting sick at work or me getting sick at work and then infecting them" illustrates the compounding nature of health problems. Students also express increased challenges to perform well academically, due to serious health problems. As noted, "I am immunocompromised, so even if others were able to return, I may not be able to, and I think it would be significantly worse to be online when everyone else is face-to-face than when everyone is online." They worry about a "second wave": "Whether or not it is actually safe to return to on-campus living; a residence hall is a breeding ground for such a virus and the thought of living in one now terrifies me." Students point to the abrupt changes in the academic setting compromising their mental health. For some students, the campus environment may be safer than their home environment. When asked about whether or not they would come back to SDSU next semester, one student responded: "I have nowhere else to go." At the onset of COVID-19, employment opportunities for students plummeted. Federal regulations limited off-campus employment opportunities for international students. Some students faced serious challenges to access basic needs, including housing instability and food insecurity. These fears can lead to a mistrust of the decision makers as noted by a respondent: "... SDSU raised all of the money for the student emergency fund and we still have international students going hungry which makes me want to know where that money is truly going." Another asked a series of questions: "will I be able to get a job to pay for food and tuition for this fall, can my GPA take another semester of online classes, are my friends ok mentally, is it even safe to get a job (doesn't matter I need one), etc." The uncertainty and insecurity seemed to be an overwhelming theme as students coupled their studies with progressively lessening resources, friends, and support. Even when the university had emergency measures in place, such as the food bank on campus that provided free food to students, including international students, there is no guarantee that the information about such a relief would reach to all students in need, given the extraordinary circumstances during the pandemic.

Honors vs. non-honors trends
Of the 30 students who did not respond to open-ended questions, 40 percent were honors and 60percent were non-honors. When asked about their intention to return back to campus, honors and non-honors students were equally willing to return to campus in the fall. While 75 percent of the honors students stated that they would "definitely" return to campus in the fall, and another eight percent stated they would "probably" come back, for the entire sample, these percentages were 71 percent and 11 percent, respectively. In short, over 80 percent of the students we surveyed, both honors, and non-honors students, intended to continue their education at SDSU in the fall semester. The percentage of honors and non-honors students who directly supported overall themes of technology use and access, health and safety concerns, social and economic considerations, and teaching and learning are reported in Figure 6, along with the average number of positive and negative comments made by an individual in each of the four categories. Despite reporting less financial strain than non-honors students in the quantitative portion of the survey, more honors students shared more positive comments and negative comments about social and economic considerations than non-honors students. The final question provided students with an opportunity to share any additional comments. Eighty-five percent of honors students expressed appreciation for the support of staff, 95 percent shared positive summative comments, and 85 percent shared negative summative comments. Similar populations of non-honors students shared positive summative (91 percent) and negative summative (92 percent) comments, with 78 percent expressing appreciation for staff.

Characteristics of students reporting struggling academically after COVID-19
Institutions of higher education must remain vigilant about the impact of the pandemic, particularly on the marginalized and underrepresented students. How we respond during this crisis will directly influence their future enrollment, persistence, and perception of higher education. Several broad trends were identified among students who struggled after the transition to an online environment. It is clear that these students desired improved communication. A comprehensive plan for communication from university administration, faculty, and staff could easily address this issue. These students often reported greater worry over finances, including the ability to pay bills, food, and housing. It is unclear to what extent financial problems played a role in academic success in this instance, but academic performance has long been associated with financial stress (Fosnacht & Calderone, 2017). Continued efforts towards more scholarships, financial aid packages, and teaching personal finance may improve academic performance by relieving financial stressors.
A significant challenge for this group was accessing and utilizing technology in the online classroom environment. It is possible that students may not be able to afford technology. They may lack access to high bandwidth internet, which is a key prerequisite of strong academic performance. Potential solutions could be to continue expanding internet access particularly in rural areas, design activities and assessments that require less advanced technology, and to improve training for students on the use of instructional technology. Our results also support the need to design activities and assessments that do not exclusively rely on live or synchronous class attendance.

Characteristics of honors students
Honors students reported less worry about financial stressors and technology problems. It is not possible to accurately predict why this result was observed from our data. Honors students at SDSU had higher academic performance in secondary schools than the average student at SDSU. While financial incentives are not provided for enrolling in the Honors College, high-achieving students may have received other performance-based scholarships. Honors students could be inherently motivated and/or conditioned through honors education to approach challenges nimbly, and thus, may be more likely to successfully access and utilize instructional technology. It should be noted that the sample of respondents was not random, which could contribute to the reported differences. The survey was distributed only to students who were members of particular groups on campus: students intending to graduate with honors distinction, TRiO students, students with disabilities, and students who were enrolled in the School of American & Global Studies. Indeed, some of the targeted groups would be more likely economically disadvantaged or first-generation college students. For example, most first-generation college students were not honors students among our respondents. Of 126 honors students, only nine were firstgeneration college students (n=37).

Conclusions
Students revealed myriad factors impacting their ability to learn as well as their preferred methods of learning online. Many craved organizational structure and clear instructions from instructors while maintaining flexibility to address the additional technical, social, health, and economic related factors in their lives that had been impacted by COVID-19.
Although statistically honors students faced less obstacles as a result of the transition to distance learning, they expressed similar concerns to those of non-honors students regarding their wellbeing. Despite the abrupt and drastic changes experienced this Spring, we are encouraged by this student's words: "I appreciate the fact that we didn't stop learning." Institutions of higher education must weigh both the pros and cons of creating the safest possible learning environment for all students. Based on our findings, we advocate that university administrators, faculty, and students collaborate to ensure: 1. Effective communication and social connections 2. Fiscal means for housing, food security, and tuition 3. Flexible and accessible infrastructure to support continued academic success 4. Safety of students, staff, and loved ones. Response: open-ended 11.Please answer the following questions by using the slider from 0 (minimum) to 10 (maximum) • In general, how much do you prefer face-to-face classes? • In general, how much do you prefer all online classes? • How much do you prefer hybrid (part face-to-face, part online) classes?
This section has questions about your general well-being.
12. Overall, how much stress are you feeling about the potential consequences of the spread of COVID-19? Little or none Some A great deal 13. How did the COVID-19 crisis impact the time you spend on the following tasks? Prefer not to answer Appendix B. Comparison of students who self-reported as 1) not struggling academically before and after the transition to online classes (n=123) and 2) not struggling academically before the transition to online classes but struggling academically after (n=99). Overall, faculty at SDSU have shown care and concern for me as they make changes in their courses in response to