Comparing the Outcomes of the Different Teaching Modes: All-in-Person, Hybrid, and Online, for Different Student Demographic Groups in a Business School

The concept of hybrid mode education is spreading. Little research compares hybrid teaching modes to online and all in person (AIP) teaching modes. Nearly all this research assumes that there is no difference in the students entering AIP, hybrid, or online sections of a course. This study used data from four years of all the courses in the Coles College of Business at Kennesaw State University. The data set, which included individual student and course section outcomes, included full student demographics and the student’s university GPA at the start of the course. The results showed that for all demographics, students in hybrid course sections earned higher final course grades than those in online sections, which in turn, earned better final grades than those in AIP sections.

While much previous research has explored the outcomes of fully online teaching compared to all in person (AIP) teaching, far less research has comparing hybrid teaching to online and AIP teaching (Amparo et al., 2018;Lovern, 2010;Slavkin, 2005;Stern, 2004). Moreover, previous research has suggested that there is no difference in the characteristics of the students entering AIP, hybrid, or online sections of a course. Studies such as that of McFarlin (2008) have only considered a single course or instructor. Some research, such as Blau and Drennan (2017) has considered student satisfaction with different modes, as well as academic outcome or grades.
Hybrid teaching may become a more common instructional delivery modality. It is important to examine hybrid class results to see how the hybrid teaching mode compares to AIP and fully online delivery. For example, if the hybrid teaching modes provide superior results for certain types of students, then institutions should encourage more instructors to use a hybrid mode. Also, doing more hybrid sections has large implications for the number of classrooms that an institution needs.
In addition, there may be important differences in demographics between students who opt for one mode over the others, and that certain student demographic groups may be more successful with certain teaching formats. For example, Xu and Jaggers (2013), and Cavenaugh and Jacuemin (2013) suggested that student demographics can be different for different modes.
The present study used the entering characteristics of students, a large sample of many instructors, and the final mean course grade achieved for a large business college over several years, to see if the benefits (including negative benefits) of hybrid and online over AIP depends on the characteristics of the entering student. While there are many types of hybrid and online teaching modes, the present study examined traditional hybrid (or flipped) teaching, not rotational hybrids. The online sections in the present study were all asynchronous, not synchronous. The present study analyzed secondary data to determine whether the entering characteristics of students, a large sample of many instructors, and the final grade achieved for a large business college over several years. The purpose of the research was to see whether there was a connection between the demographics of the students, the model of instruction, and the success of the student in the course.

Research Questions
Thus, the research questions that the research examined were: 1. Is there a difference (both demographic and previous academic achievement) in students learning in different modes? 2. How do different demographic characteristics affect student outcomes in different modes? This was measured as the difference in mean course final grades between different groups. Outcomes in this study included the final course grade for the section.

Literature Review
The literature will report on how previous research has examined differences in the demographics of students taking different modes. We first examine the larger research output that does not include hybrid courses, then the smaller research that includes hybrid courses.

Online to AIP Comparisons
Many studies, with sample sizes ranging from very small to very large, have compared the outcomes of online versus AIP courses. The following is a selection of some of the most recent and more relevant studies. Stern (2004) examined online and AIP instruction for one course and concluded that the online instructional delivery mode worked as well as AIP if online instructors had enough time to plan and implement their courses. Sapp and Simon (2005) compared grades for online and AIP writing courses. Their findings showed that more students thrived (defined as A or A-) in AIP courses than online courses (32% to 52%). Summers et al. (2005) examined grades for online versus AIP for a statistics course. They found no significant difference between modes of teaching. Kelly (2009) reported that she could find no significant difference between student grades for online and AIP modes. Kelly did not control for entering GPA. Dell et al. (2010) found no differences between online and AIP sections of a graduate human development and an undergraduate psychology course. Ni (2013) found that there were no significant differences in outcome between online and AIP classes. Amparo et al (2018) used a very large sample (96,000 students) across two institutions to compare online and AIP results. They found that AIP students outperformed online students in course final GPA. Blau and Drennan (2017) used student's perceptions to compare different teaching formats and suggested that universities find ways to increase perceived favorability of online and hybrid courses for those that prefer AIP.

No Examination of Student Type
None of these previous studies examined pre-course university GPA self-selection (e.g., Do more academically able students prefer a particular mode?) Further, most of these studies failed to examine differences in pre-course GPAs or any demographics of students. Generally, these studies mentioned above all found no significant difference in final course grades or that online courses achieved worse final course grades than AIP ones.

Examination Included Student Type
Cavanaugh and Jacuemin (2013) used a large sample size (5,000 courses) in one institution. They found no significant difference overall between online and AIP classes. They did find that students with good pre-course GPA did better those who did not. Online courses increased the effect of pre-course GPA. They also found that students who usually had high grades tended to do online courses, as the mean pre-course GPA was 3.41 for online students, while only 3.02 for AIP students.
Xu and Jaggers (2014) researched a very large data set of online and AIP courses (500,000 student-course sets). They did allow for differences in pre-course GPAs. They found that males, younger students, Black students, and those with lower pre-course GPAs did worse in online courses, while females and Asians had no significant differences, and older students did better in online courses. They also looked at subject matter and reported that computer science, communication, and health had no significant differences. All others had AIP doing better than online courses. The social sciences, business, law, and nursing showed the biggest differences. Teaching mode affected starting students more adversely than continuing students were. Nguyen (2015) summarized research comparing AIP and online teaching modes. He found that generally research considers online learning is better but that there were problems with much of this research. Amro et al. (2015) showed that for their algebra courses, AIP students got higher grades than students studying online did. Although they looked at age and gender factors, they did not look at pre-course GPAs to see if the students were similar in academic ability.
Bief and Brams (2016) compared student performance in online and AIP courses. They encountered mixed results; some studies showed the AIP course were better and some the online courses. Sun and Chen (2016) did a review of 47 papers comparing online and AIP teaching modes. They concluded that online teaching works as well as or better than AIP if done properly. That is well-designed content, motivated interaction, and well prepared and supported instructors.
Most studies did not examine the effect of demographic factors. However, Cavanaugh and Jacuemin (2013) found that students that earned higher grades in traditional settings tended to choose online courses. Xu and Jaggers (2014) showed that the difference between online and AIP depends on race, gender, previous GPA, and age. In fact, they also showed that older students did slightly better in online courses. These two studies hinted that demographics and pre-course GPA might affect course outcomes. Blau et al. (2019) used the students' intent to transfer as an output measure.

Studies that did not Examine Student Types
Several studies looked at comparing hybrid to either or both of and online modes. Reasons, et al., (2005) examined the three teaching formats and concluded that online was better in achieving a higher final course grade than hybrid. McFarlin (2008) examined grade results for hybrid and online sections. McFarlin found that student learning, as represented by grades, increased in hybrid and online sections compared to AIP sections. Lovern (2010) found no significant difference in outcomes between online, hybrid, and AIP sections of the same course. Much previous research did not examine pre-course GPA self-selection. Son et al. (2016) looked at a lab class that was offered in the three formats. These researchers concluded that grades were highest in a hybrid mode, and lowest in a pure online format. Mansour and Mupinda (2007) studied students' experiences rather than outcomes in online and hybrid classes. They found that students preferred hybrid classes, but some students preferred online courses. This maybe reflected the students' learning style. Senn (2008) reported on student perceptions the three modes for one course. He concluded that students felt that hybrid sections were more difficult for this technology heavy course. Larson and Sung (2009) looked at hybrid sections, as well as online and AIP. Unfortunately, they used student perceptions of learning effectiveness not actual learning achieved as a variable. They did not look at whether student self-select types of course by their pre-course GPA. They showed that students preferred hybrid to online and online to AIP. Sackett (2009) compared the three modes' outcomes based on the training that the instructors had had. He found that online learners were older and had better computer competency. Kemp and Grieve (2014) studied student preferences and outcomes between AIP and online activities. They found no difference in learning outcomes but found students preferred online for written assignments and AIP for discussions. Goerke (2018) examined the three modes of training for one Air Force course. She found no differences in student satisfaction between the three modes. Cathorall et al. (2018) assessed student performance in hybrid and online classes. They found no difference in student grades, but higher student evaluations in online courses.

Studies that Examined Student Types
Brau et al (2010) reported on completion and success results in a course transitioning from AIP to hybrid and online modes. They found that completion rates increased significantly as did success rates. They did not think this was due to better students entering online and hybrid sections. Hybrid sections had higher completion rates than online sections. Price et al (2016) looked at effect of factors on student performance and satisfaction across modes. They looked at age, sex, interaction, clarity, control, and motivation. They found little correlation between age or sex/gender and student outcomes. They found that course design (participant interaction, learner control, and course clarity) did affect student outcomes. Mode of instruction had no significant effect. Kim and Keuegar (2017) compared hybrid and AIP courses. They concluded that using two modes, AIP and online, in the same course can be challenging to instructors. Baum and McPherson (2019) examined learning in online and hybrid sections, taking account of the academic weakness of entering students. They suggested that students with weak academic backgrounds and other risk factors, including socioeconomic status, struggle in online classes.
Taken together, previous research investigating student demographic differences showed such differences were sometimes liked to outcomes. In addition, they found that hybrid sections often achieved better outcomes to either online or AIP sections.

Methodology Data Set
Kennesaw State University (KSU) provided every student-course record in KSU's Banner system from Fall 2015 to Summer 2019 for all Coles College of Business undergraduate courses. The analysis did not use later data available because of COVID-19. When the pandemic closed campus buildings, all AIP and hybrid sections went completely online within 4 days. Many of our instructors had never taught online before. The administration told instructors to give the students benefit in grading for the stresses of lockdown. KSU has offered synchronous online, plus rotating hybrids, as well as standard hybrid since the initial shutdown. Many instructors had to teach online for the first time with little or no training. Thus, the course grade awarded during the shutdown because of the pandemic were skewed upwards and not useful for determining comparison of teaching modes. Therefore, data after the Fall 2019 semester was not included.
The researcher removed from the data set all student-record data that had no grade awarded, or had a grade of I (incomplete), S (satisfactory), or U (unsatisfactory), as these grades did not give an indication of student learning.
Each student-course record set originally consisted of the following: 1. An arbitrary random number instead of student name. The researcher deleted this column from the working database as not useful. 2. Course grade in letters. This was converted to numbers; A = 4, B = 3, C = 2, D = 1, F = 0. 3. Previous overall university GPA of student at the start of course. This was missing for some students. Previous GPA varied from zero to 4. Starting transfer and freshmen students would have no previous GPA. 12. This analysis did not use Instructor ID. There were 232 instructors in this analysis. There were 118,280 student-course data records for the analysis.
Dataset Characteristics. Table 1 shows the basic properties of each variable in the data set. N = 118,280 *Hybrid student-section records are a small (2.23%) part of the overall data set.

Statistical Analysis
The statistical analysis used the final grade awarded for the course as the predicted variable and all the other variables as predictor variables. The F-test for the final grade for modes was 739.85, and the t-test between AIP and Hybrid was 31.57, between AIP and online was 33.12, and between hybrid and online was 15.53. Therefore, the differences in final grades between all modes was highly significant.
The research then found the correlations in Table 2 and ran regression analyses on the main data set in Tables 3 and 4. The highest correlation is between course grade granted to student in the course and the student's previous GPA. It also shows that there is high correlation between course level and online and hybrid modes, as in upper division there are more hybrid and online proportionally. There is a high correlation between summer terms and hybrid sections, as there are more online sections in the summer proportionally. Most of these extra online and hybrid sections are for junior and senior courses. There is low correlation between age and online but not hybrid modes. This is probably because older students do more online courses. Female students tend to achieve higher grades in all cases. There are low correlations between course level and final grade. There is also a negative correlation between course grades and previous GPA with claiming a Black racial identity. This correlation analysis showed nothing unexpected.
Next, the study reports the regression analysis in Table 3. This regression supported the correlation analysis. The largest predictor of a student's course final grade was the student's university GPA at the start of the course (Previous GPA). However, hybrid mode was the second biggest correlator, with AIP a negative correlator. Online was neutral. This suggests that students in the hybrid mode had reported higher course grades than in online courses. In turn, students in online courses received higher grades than students in AIP courses. Students reporting to have an international status followed those who claimed an Asian identity were received higher course grade than those identifying themselves as white. Those who reported as being Black, received lower grades. As previous GPA was the best predictor of final course grade in this study, Table 4 shows the results of predicting final course grade by previous GPA only. Some student-course records had missing previous GPA, so they were eliminated from this sample. When using only previous GPA as a predictor of course GPA, the regression had an adjusted R 2 of 90.9%,

Analysis of Mode Effect
The initial analysis suggested that the teaching mode was related to the final course grade, making additional analysis was necessary. As Year and Age had no noticed effect on results, the study deleted that information from the following Table 5 results. The summer term and level columns are missing from most of the following tables, as the study did not consider them major factors. Separating results by teaching mode shows that the hybrid mode leads to higher course grades over online mode (8.34%), and online over AIP (8.38%). Although previous GPA for hybrid is higher than online (1.56%), which is higher than AIP (1.81%), it appears that hybrid mode leads to higher course grades for similar previous GPA. However, these results are for all students.

Analysis for Sex
To examine the effect of different demographics, the analysis showed in Table 6 the results by various student characteristics, including the sex of the student. The overall reported sex balance was 57.7% male to 42.3% at KSU. Although a lower percentage of students reported to be females than males at KSU, more females reported enrolling in more online courses than male students; 26.2% of female student-courses were reported as being online versus 17.5% for male. More male students (2.39%) reported enrolling in hybrid than female (2.02%). Table 7 summarizes Table 6. This shows that Black and Hispanic have the lowest proportion of males in hybrid sections. Asian, Black, Hispanic, and Multi-ethnic/Multi-racial have the lowest proportion in online sections.

Analysis by Course Level
The study then investigated if the results varied by course level and Tables 8, 9 and 10 show the results.  For senior year students, there are almost as many online student-courses as AIP. For first-year students (known as freshman at KSU), there are very few online courses. Both hybrid and online increase with level as a percentage of year's total student-courses.

Figure 2. Final Course Grade for all Students by Course Level and Modes
Grades increase slightly with level for all modes except hybrid. In senior and junior year students, hybrid modes have students that received higher final grades than grades received in online courses. Online course grades were higher than AIP. Data from hybrid courses in second year (known as sophomore year at KSU) were too small to use. The overall mean final grade for all student-courses is almost exactly a B (3.008).

Analysis for Term
There are far more online courses as a percentage in Summer than in Fall or Spring. The analysis looks in Table 11 at whether courses in summer were different to the rest of the year in outcomes.   There is a far larger percentage of online sections in summer (41.75% versus 19.7%). Hybrid declines from 2.33% in spring to 1.18% in summer, while online increases. Final mean course grades for online and AIP both very slightly increase in summer, whilst hybrid mean course grades decline from 3.45 to 3.15. There does not appear to be differences in mode percentages or mode course final grades between Fall and Spring terms, so the study does not use that data further.

Analysis by Discipline
The study investigated whether the discipline influenced final grade with different modes, whose results are in Table 14.   In all disciplines with hybrid sections except economics, the mean final grade for hybrid is superior to online or AIP. In economics, hybrid is superior to AIP but not to online.

Analysis by Race/Ethnicity/International Student Status
The study considered in Table 17 whether different groups were better doing certain modes for their courses. Some students were "N/A" or unknown for reported group and thus the study deleted them. The resulting total data set contained 115358 student-course records.  Tables 18 and 19 summarize Table 17. International students finished the most hybrid courses proportionally, whilst they enrolled in the fewest number of online courses. This may be because United States Visa regulations require 12 credits of "in person" classes, such as AIP or hybrid. Students taking courses in the hybrid mode received the highest grades, except for those reporting an identity as Hispanic. For that group online students received higher grades than those learning AIP. The study uses the white race mean as the base case in the next table (Table  20) as it is the largest group. The biggest differences in mean grades are highlighted in bold. International students and Hispanic students received grades closes to white students in AIP courses. However, for those reporting as Black, the grades they received came closest to white students in hybrid sections. Analysis Using Previous GPA Previous university GPA before the start of the course is an indicator of academic ability. So, one would expect students with high previous university GPA to get better final course grades. Table 21 shows the effects of previous GPA.  Table 22 provides additional information about the findings from Table 21.  Table 23 and Figure 1 summarizes Table 22.

Figure 3. Course Final Grade Data vs Previous GPA for all Students by Instructor Mode
For all previous GPA, the hybrid is best for mean final grade is best then online, then AIP.

Results Using Only Multi-Mode Teachers
All data were deleted with instructors that only taught in the AIP mode, leaving 53,556 studentsections. Table 24 shows the results from the analysis without these instructors. In the table, the same results emerge as in previous analyses. That is, that students in hybrid courses received higher grades than in online courses. Further, both online and hybrid courses result in students receiving higher grades than in AIP courses.

Results Using Only Those Instructors Who Taught Hybrid Mode
The researcher then looked at data sets with instructors who taught hybrid, with 9834 data sets with 16 instructors (about 7% of all instructors). With this data in Table 25, hybrid is a far larger proportion of courses.  Table 26 further disaggregates information from Tables 24 and 25. Generally, instructors who taught at least one hybrid courses allocated higher grades in all modes than those that did not. With hybrid teaching instructors, there was little difference between online and hybrid results. Even so, both groups allocated higher grades than those who taught in AIP courses only.

Summary of Results for Course Final Grades
Previous GPA -The biggest predictor of a student's final grade in a course was their previous university GPA at the start of the course.
Sex -Generally, female students tend received higher final grades than male students in all formats and courses.
Race/Ethnicity/International Student Status -International students (who could claim multiple races and ethnicities) received the highest final course grades. Asian students in at KSU without international student status received the next highest grades. Students reporting their race as Black received lower grades than other groups. Students in hybrid courses received the highest final course grades across races, ethnicities, and status as an international or domestic student. The advantage in terms of higher grade received was highest for Black students and least for Hispanic students.
Mode -Overall, grades were higher in hybrid courses than online courses. Both hybrid and online grades were higher than AIP grades. However, hybrid courses tended to be more available in upper division courses.
Discipline -Only half the business disciplines taught courses in the hybrid mode, which may have affected results. However, in all disciplines with hybrid courses, the hybrid mean course grade was higher than that for online or AIP.
Based on these findings, more courses should be available in the online and hybrid modes, but especially the hybrid mode. There also might be benefits to using the hybrid mode that go beyond the scope of this student. If higher education institutions want to increase hybrid teaching, it seems reasonable to provide additional support to instructors to learn to teach in hybrid modes. Additional research might examine the types of supports.
While this study showed hybrid mode teaching resulted in higher grades than in AIP courses, it did not show why. One theory could be is that hybrid courses enable more interactive teaching than AIP. Another theory is that students took advantage of other affordances of online and hybrid courses that include a more fluid sense of time and deadlines. There is also the possibility that the instructors who give higher grades tend to be those who teach hybrid courses. In any case, additional research is also needed to determine why the grades are higher in these courses. Is because there is better teaching and learning? Is it because teachers set different expectations for learning in different modes? Is it because there are fewer threats to individuals' claimed identities in courses where one does not have to be in the AIP space? Or is there some other reason?
It would be interesting to do similar studies with other universities and colleges to see if KSU's patterns are similar or different. If done with several other teaching institutions and one found similar results, then one could make generalizable conclusions about the effect of teaching mode on course grades.

Limitations
This study has several limitations.
1. The use of previous GPA to represent the academic ability of an incoming student is a convenient assumption. However, that is how most students rate their learning. 2. The use of course final grade to represent learning from a course is a common approximation of learning, but admittedly imperfect. 3. This analysis did not consider other factors like how many online or hybrid courses the student had done before the course, how many online or hybrid courses the student took at the same time, or whether the student was only taking online courses or mixing F2F with online and hybrid courses. 4. In the main study, hybrid student course records were only a small proportion of the total data. 5. The study did not examine differences between instructors. However, many instructors grade harder than others for the same course. Hybrid teaching instructors may grade higher than those teaching other modes. 6. The data for this case study comes from one university. Other universities and colleges may show completely different patterns. 7. Variables for Race/Ethnicity and International Student Status were all grouped together for this analysis, which might obscure some of the nuance between students who claim these characteristics. 8. Since KSU only collects information about sex using a binary of Male/Female. Data and analysis for those who might claim a non-binary sexual/gender identity was unavailable.

Conclusions
The base data set has only a very small proportion of all student-course records from hybrid sections. This may mean that the results are heavily biased towards online and AIP modes. However, due to the large number of student-course records, this analysis can provide useful information that might cause other universities to consider their own patterns. The analysis also showed there was little difference in type of student who did each mode, except those students with more experience in the academy tended to do more online courses. This research basically replicates most of the previous studies with larger student populations, but with more information of how student types affect the results.

Declarations
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author received approval from the ethics review board of Kennesaw State University, USA for this study.
The author received no financial support for the research, authorship, and/or publication of this article.