The Impact of a Structured Protocol on Graduate Student Perception of Online Asynchronous Discussions

Instructors of online courses face unique challenges to ensure student interaction with course material. Sometimes, even the most exciting content is insufficient in an attempt to engage students. Online, asynchronous discussions offer promise as a means to increase student-to-student and student-to-content interaction and, ultimately, student satisfaction with online courses. The modification of structured discussion protocols designed for use in face to face environments offers instructors of online courses an efficient method of adding purpose and structure to asynchronous discussions. This research employed a quasi-experimental, nonequivalent group design to examine students' perception of asynchronous discussion before and after applying a structured discussion protocol that included a clear statement of purpose, directions for participation, and a grading rubric. Results from the data analysis indicated that student perception of online asynchronous discussions improved when a structure was added. Results also showed a lower level of dissatisfaction when discussions were structured.


The Ruthlessly Efficient Online Learner
Online learning has become a standard method of delivery for undergraduate and graduate courses over the past decade. In the 2016-2017 school year, 76 percent of all degree-granting institutions offered courses online, and about 50 percent of institutions provided at least one program entirely online. About one-third of degree-seeking students enrolled in at least one online course during the same academic year, and about 15 percent of those enrolled in degree programs were offered entirely online (Xu & Xu, 2019). The number of students who participate in online learning continues to increase year after year and grew to over 6.3 million for the fall semester of 2016 (Seaman et al., 2018). The increasing demand for and supply of online courses and fully online degrees is a result of the many advantages of remote learning, such as increased flexibility for working students or students who reside in remote locations (Zimmerman, 2012). The global pandemic that began early in 2020 is likely to impact further the number of students who access college courses online. Once the public health crisis ebbs and allows for the safe return to face-to-face teaching, there may be students who opt to continue to access courses online because of ongoing health concerns or simply convenience, further increasing the prevalence of online instruction.
Online instruction may be delivered either synchronously or asynchronously. In synchronous online courses, instruction and class activities occur in real-time using technology tools such as live streaming. Synchronous instruction requires student attendance at a specific time and date. Conversely, asynchronous online instruction occurs without real-time interaction and includes pre-recorded videos, discussion boards, and email. Asynchronous instruction is the traditional method of delivery for online courses (Watts, 2016) and remains popular because students can access a course at any time from any location (Lee & Choi, 2011). The advantages of asynchronous courses are particularly significant for adult graduate students who complete degrees while balancing full-time jobs, families, and personal lives.
Despite the benefits, students in both synchronous and asynchronous online courses often report dissatisfaction emanating from feelings of disconnection and social isolation (vanTyron & Bishop, 2009). Additionally, students who participate in online instruction face a higher risk of poor academic results, with low student retention (Holley & Oliver, 2010) and an increased likelihood of low grades (Bordeau et al., 2018) as primary concerns. Instructors of online courses also face unique challenges that are inherently different from the typical obstacles in traditionally taught face-to-face courses. Sometimes even designing the most exciting content proves to be insufficient in the attempt to engage students. In Small Teaching On-line, Darby and Lang (2019) state, "online students are becoming ruthlessly efficient in how they spend their time on classwork" (p. 56). Even when students know that the content or activity is essential, they sometimes engage at a low level and do just enough to meet minimum expectations. Darby and Lang (2019) suggest that it is the instructor's responsibility to think differently about motivating student engagement with course material to overcome the effects of the ruthlessly efficient learner.

Online AsynchronousDiscussions as an Engagement Strategy
The College of Education at the public university where this study occurred offers a variety of online course options at the undergraduate and graduate levels. However, beginning in the 2016 school year, the certification program for entry-level educational leaders was moved entirely online with a requirement for asynchronous delivery to meet the needs of students who seek the certification while employed as full-time teachers and administrators. Most of the professors who teach in the program are experienced in online instruction. Nevertheless, department discussion often focuses on supporting students who engage in what Darby and Lang (2019) describe as "ruthlessly efficient" behavior by increasing student engagement with their professor, classmates, and course content. Faculty members express a shared belief that students in the school leadership program have a strong desire to master course content and have adopted "ruthlessly efficient" behavior as a coping strategy to manage course work in addition to their busy lives, rather than as a means to avoid challenging tasks. The professors have a common understanding that delivering instruction in a fully online program where all students are full-time working adults with a variety of professional and personal obligations, necessitates ongoing investigation to identify strategies that students find worthwhile and engaging.
Faculty members in the department examined the literature to identify strategies to engage online learners more fully. Abundant evidence suggests that structured asynchronous discussions are a promising strategy that, when used appropriately, can increase student-to-student interaction and student engagement with course material in an online setting.

Relevant Scholarship
The purpose of the asynchronous online discussion is for students to engage in a conversation about course objectives by responding to an initial question and their classmates' ideas. Online discussions differ from traditional in-class discussions because students can respond to prompts on a timeframe of their choosing rather than contemporaneously (Thompson, 2006). The added time to develop a response often results in improved student writing and critical thinking skills because students have the opportunity to create content, research various ideas, and then write a short passage with relatively low stakes. Additionally, being exposed to other students' writing may serve as a model for students with lower writing ability (Aloni & Harrington, 2018). When students interact with one another and their instructor on discussion boards, there is an increase in social presence (Cho & Tobias, 2016) and a reduction in learner isolation (Banna et al., 2015). Kurucay & Inan (2017) found that students who collaborate with other students in online courses earned higher grades than students who worked alone. Research also exists to support the premise that students who actively participate in online discussions earn better grades than those who are less active (Davies & Graff, 2005;Kent et al., 2016).
Despite the body of research that indicates the promise of online asynchronous discussion as a tool for student engagement, many professors are reluctant to use them regularly because they can result in low student participation, participation in a shallow manner, or negative student feedback (Caspi et al., 2006). Studies reveal a variety of reasons for low student engagement in discussions, including confusion about the purpose and value of the discussion and a lack of understanding of the professor's expectations (Kim, 2013;Lee, 2013). The level of accountability in a discussion is also related to the level of student participation. Student participation diminishes when the professor does not provide deadlines, and when the discussion itself is not graded or not a component of the overall course grade (Pena-Schaff & Altman, 2015). In research regarding student opinions of online discussions, a variety of concerns have been reported, including the time-consuming nature of discussions, low levels of classmates' engagement, rude responses, and lack of clear guidelines for feedback and grading (Clinton & Kelly, 2019). Students also express dissatisfaction with the overwhelming time it takes to read posts, classmates who dominate the discussion, or mismanagement of the discussion by the professor (Rovai, 2007).
After considering the benefits of online asynchronous discussion to improve student engagement and student outcomes, several professors who teach in the educational leadership certificate program at the university where this study occurred, agreed to engage in research to gather data about the efficacy of various discussion strategies. One approach tested was the use of structured discussion protocols designed for use in face-to-face environments that have been modified for use in online courses. The School Reform Initiative defines discussion protocols as "structured processes and guidelines to promote meaningful, efficient communication, problem-solving, and learning" (SRI, n.d.). When used correctly, protocols allow equal participation from all group members, and therefore, no one voice or interest group dominates the conversation. Directions for the implementation of protocols in face-to-face environments are widely available. Still, there is limited information about the modification of these protocols for use online and whether they alter students' perceptions of discussions.

Research Question & Hypothesis
Although the use of structured discussion protocols has the potential to impact various aspects of online discussions, this study sought to understand their effect on student perception. For this study, the School Reform Initiative's Final Word protocol was modified for use in an online asynchronous discussion. The research was conducted to determine the differences in graduate student perceptions of asynchronous online discussion when students were told the purpose of the discussion, the modified Last Word protocol was used to structure the discussion, and a grading protocol was in place, compared to when a discussion format with minimal structure was used. The hypothesis tested was that the use of a structured protocol that included a specific purpose for the discussion, directions for participation, and a grading rubric would increase positive student perception of online asynchronous discussions in a graduate course. The literature regarding asynchronous online discussions was used to create the theory of action that if the use of a structured protocol resulted in more positive student perception of discussions, professors would be more likely to use them student-to-student interaction and student to content engagement would, therefore, increase.

Research Design
The data collected for this study was primarily quantitative with an added element of qualitative data analysis for the corroboration of initial findings. The quantitative portion of the research was structured with a quasi-experimental, nonequivalent design to examine students' perception of asynchronous discussion before and after applying a structured discussion protocol. The qualitative data were analyzed using content analysis of student responses to open-ended questions.
The researcher examined the students' perceptions enrolled in four sections of the community engagement course required for initial school leadership certification in the southeastern state where the university is located. Two sections from the summer and fall semesters of 2018 formed the comparison group (N=95), and two sections from summer and fall 2019 (N=84) formed the treatment group.
A quasi-experimental design was used for the analysis of the quantitative data because participants could not be randomized into comparison or treatment groups. Randomization of participants in groups is preferable, but not possible when the comparison and treatment groups consist of students enrolled in graduate courses. This lack of randomization introduces the possibility of significant differences between the two groups that may explain differences in outcomes unrelated to the intervention. While the quasi-experimental design is typical in the social sciences, it does increase the threat to internal validity.
The researcher attempted to mitigate the risk posed by the quasi-experimental design by examining the comparison and treatment groups' demographics. Demographically, the comparison and treatment groups were very similar. About half the students in both groups were classroom teachers, and the other half held leadership positions such as assistant principal or instructional coach. In both groups, slightly more than half of the students reported their race as Caucasian, a third as Black/African American, with others choosing not to self-report. Approximately 80% of participants in both the comparison and treatment groups were female, and about 20% were male.

Measures
Data was gathered from the results of the Student Evaluation of Instruction (SEI) administered at the end of each semester by the university for all online courses. These anonymous evaluations include both statements that students rate using a Likert Scale and open-ended questions. The SEI was conducted online using the CoursEval web-based assessment system, separate from the course delivery platform. The following data from the Likert Scale questions were examined: • Student numerical rating (1-5) regarding the statement "Class discussions and/or other activities helped me to understand the subject matter." • Student numerical ratings (1-5) of the statement "The instructor is receptive and responsive to the sharing of ideas during course discussions." • The following data from open-ended questions were examined: o Student comments from open-ended questions that included the word "discussion." Comments were found in following questions: "Comment on the evaluation methods utilized-fairness, difficulty, ease, etc."; "Explain any positive changes you would like made to this class"; and "Comment specifically on the value of texts, class materials, assignments, and class activities." o Content analysis was used in order to categorize student comments from open-ended questions as positive, neutral, or negative. Content analysis was appropriate as an "unobtrusive technique that allows researchers to analyze relatively unstructured data in view of the meanings, symbolic qualities, and expressive contents they have and of the communicative roles they play in the lives of the data's sources" (Krippendorff, 2013, p. 49) SEI data was used with the understanding that there has been recent literature questioning the effectiveness of student evaluations of instruction to measure the overall effectiveness of the professor. Stark and Freishtat (2014) examined teachers' ratings from a statistical perspective and found evidence that student ratings of teacher effectiveness are negatively related to direct measures of effectiveness. However, Stark and Freishtat suggest that the most problematic issues in student evaluations are overall omnibus ratings of effectiveness. Despite this flaw, some value can be found in questions about specific instructional methods and student comments. These problems with the validity of student evaluations were considered in the design of this study. A deliberate decision was made to include only the questions and comments that specifically dealt with discussion as an instructional strategy. Furthermore, this study seeks to understand student perception of discussion using a protocol, and so the use of student evaluation data was found to be an appropriate measure. Stark and Freishtat (2014) also indicated that student evaluation data are not reliable when low numbers of students participate. In this study, student participation rates in the SEI were high for comparison and treatment groups. The comparison group included three sections of the Community Engagement course delivered in the summer and fall semesters of 2018. Of the 108 students enrolled, 95 completed the SEI, an 87.9% completion rate. The treatment group consisted of two sections of the Community Engagement course delivered during the summer and fall of 2019. Of the 87 students enrolled, 84 participated in the SEI, a completion rate of 96.5%.
The Mann-Whitney U test was used to analyze numerical ratings from two Likert Scale questions regarding class discussions posed to students on the SEI. The Mann Whitney U test is a rank-based nonparametric test that can be used to determine if there are differences between two groups on a continuous or ordinal dependent variable. The design and data met all four assumptions necessary for this test: • One dependent variable (student perception) was measured on an ordinal level (Likert Scale).
• One independent variable that consists of two categorical, independent groups (students who participated in asynchronous online discussions with and without a structured protocol).
• Independence of observations, meaning there is no relationship between the observations of each group of the independent variable or between the groups themselves.
• Upon visual inspection of the data, the distribution of scores for both groups of the independent variable had the same shape.

Participants
Data was gathered from students enrolled in a graduate-level course focused on community engagement for school leaders. This course is a requirement for the initial school leadership certificate in the state where the university is located. The leadership certificate program includes six courses offered entirely online, with no option for face-to-face instruction. All students in the program are full-time teachers or entry-level school leaders who have previously earned a minimum of a masters' degree. As mandated by the university, all course elements must be asynchronous, unless otherwise noted in the course description. During the study period, there were a total of between 390-430 students enrolled in the initial school leadership certificate program each semester.

Procedures
Student perceptions of an online discussion with the minimal structure were gathered from a total of 95 who completed a Student Evaluation of Instruction (SEI) after a community engagement course for school leaders during the summer and fall 2018 semesters. Students were given a short scenario related to a specific reading assignment and the following directions: Given the readings in this module and your professional experience, what guidelines and practi ces would you use to discuss the issues in the scenario with internal and external stakeholder groups? After reading your classmates' posts, what advice do you have that they may not have previously considered?
The discussion took place over one week, and the professor replied to student posts each day and offered redirection when the discussion became redundant. Students were required to have three posts by the end of the week with no additional requirements about when the posts needed to be completed. The discussion was "whole group," and all students in the course were in the same discussion, rather than being assigned to small groups.
Perceptions of online discussions with a clearly stated purpose, a structured protocol, and rubric were gathered from 86 students who completed the SEI after a community engagement course in summer 2019 and fall 2019. The Final Word protocol from the School Reform Initiative was modified for use in an asynchronous online discussion. Students were informed that the purpose of the discussion was to provide an opportunity to work in a small group to explore a specific course reading, to clarify their thinking regarding the reading, and to acquire a deeper understanding of the text through dialogue with peers. Students were placed in groups of four or five for a discussion period that lasted one week. The following directions were given: 1. Each student will choose a passage from the identified reading that they found particularly interesting, meaningful, or engaging. Each student will post that passage in a new discussion thread without explanation about why they chose it. In other words, each student posts only their passage and the page on which it is located. These posts must be made on Sunday, Monday, or Tuesday before noon.
2. Beginning on Tuesday at noon until Friday at noon, students will reply to their classmates with their reactions and thoughts about each of the passages posted by the other group members. Each student reply to a minimum of three classmates during this timeframe. As indicated in the rubric, these posts should be substantial and reflective. Ideas for posts include but are not limited to: 3. Finally, each student will go back to their original discussion post, read the reactions of their classmates, and then make their final post or "the last word." In this post, students will reveal their original interests in the passages and reflect on what they learned from reading the interpretations from their classmates. This post is due by Sunday at 11:59 pm.
The professor monitored but did not post in small group discussions unless there was a significant problem, such as an inappropriate post. Students were provided with a grading rubric.

Analysis of Student Responses to Likert Scale Questions Using Mann-Whitney U Test
Question eight on the SEI asked students to rate their responses on a five-point Likert Scale to the following statement: "Class discussions and/or other activities helped me to understand the subject matter." Response choices were: 1, strongly disagree; 2, disagree; 3, neutral; 4, agree, and; 5, strongly agree. It is important to note that no other changes were made to class activities during the periods when data was gathered. A Mann-Whitney U test was run to determine if there were differences between student responses to this statement during semesters when the Final Word protocol was used in asynchronous online discussion versus semesters when the minimal structure was used in the online discussion. Distributions of the student responses were similar for groups who participated in discussions using the Final Word protocol and those who did not, as assessed by visual inspection. Student responses to this statement were statistically significantly higher in students who participated in discussions with the structure using the Final Word protocol (Mean = 4.65) than in those who participated in discussions with minimal structure (Mean = 4.39),U =4,602,z=2.066,p=.039.
Question twenty-one on the SEI asked students to rate their responses on a five-point Likert Scale to the following statement: "The instructor is receptive and responsive to the sharing of ideas during course discussions." Response choices were 1, strongly disagree; 2, disagree; 3, neutral; 4, agree, and; 5, strongly agree. A Mann-Whitney U test was run to determine if there were differences between student responses to this statement during semesters when the Final Word protocol was used in asynchronous online discussion versus semesters when the minimal structure was used in the online discussion. Distributions of the student responses were similar for those who participated in discussions using the Final Word protocol and those who did not, as assessed by visual inspection. Student responses to the statement were statistically significantly higher in students who participated in discussions with the Final Word protocol (Mean = 4.88) than in those who participated in discussions with minimal structure (Mean = 4.45), U =5,087.5, z=4.096, p=.000. • The discussion posts were a little confusing at first and could use more clarification at the start of the course.

Analysis of Student Responses from Open-Ended Questions Using Content Analysis
• I would like to see the discussion post change and not limited to a specific period. I believe that when a new module begins, the discussion should begin as well.
• Discussions should be held in smaller groups.
• I believe the discussions could become a bit confusing at times, and I feel that the evaluation of input on those discussions was a bit subjective.
• The way discussions are handled should be changed.
• I would like the discussion changed to the more traditional style or a video chat.
• I would like to see online video discussions to act a substitute as discussion boards.
• I felt that the feedback and grading on the discussion questions were subjective.
• The teacher instructions for discussions were not clear. The discussions served more as busy work, and a greater emphasis was placed on if we addressed each other and formatted correctly than it did the actual content. Having only two discussions were pointless to the overall learning of the course.
• The discussions were difficult because we had to go back to them multiple times over multiple days...even submitting responses hours apart did not count. It was difficult to reengage in the discussion posts in a meaningful way. Also, the discussion topics were limited and it sometimes felt like we were having to creatively restate the exact same point.
• The structure of the discussion posts are not reflective of an actual classroom discussion and felt awkward to respond to. Typically, in the classroom setting one would respond directly to a point a person made without addressing several people in one response. If the format can be changed to reading through the post and responding to one post directly that would be helpful.
Data was gathered from the open-ended questions posed to students on the SEI. To determine student perception of asynchronous online discussions with minimal structure, all comments that included the word "discussion" were gathered from student responses from the community engagement course in the Summer and Fall of 2018. Twelve of the 108 students enrolled in the course mentioned "discussion" in the open-ended comments. Content analysis of the responses revealed that 11 of those comments were negative, zero were neutral, and one was positive. All verbatim comments are provided in Table 1.
Similarly, to determine student perception of asynchronous online discussions using the structure of The Final Word protocol, all student comments from the SEI that included the word "discussion" were gathered from the community engagement course in Summer and Fall of 2019. Four of the 84 students who completed the SEI four mentioned "discussion" in the open-ended comments. Content analysis revealed that two of those comments were negative, one was neutral, and one was positive. All comments are provided in Table 2. • I didn't care for the discussion assignments.
• The course would be better if you eliminate discussions.
Neutral Comments: • I liked the discussions but not having to rely on a classmate to respond. I only received one response when I should have had two. This made it challenging to respond with as much substance as was excepted of me.
Positive Comments: • I loved all of the discussion. I truly wouldn't change that at all.

Discussion
The data gathered from the Likert Scale questions on the Student Evaluations of Instruction indicate that student perception improved when the purpose of the discussion was clearly stated, the structure of the Final Word Protocol was used, and grading expectations were made clear with a rubric. There was a statistically significant increase in the degree to which students agreed with the statement, "Class discussions and/or other activities helped me to understand subject matter" among students who experienced the discussion with the protocol than students who did not. The validity of this result is somewhat problematic because the question includes "other activities" in addition to discussions. However, this issue is partially mitigated because, in the courses from which data were gathered, no changes were made to class activities other than discussion. This result demonstrates that this particular discussion protocol allowed students to feel that the discussions aided in their understanding of the subject matter, which is a critical component of success for any course.
Similarly, student agreement with the statement, "The instructor is receptive and responsive to the sharing of ideas during course discussions," was statistically significantly higher when the structure of the Final Word protocol and rubric was used compared to discussions with little structure. This result is of interest, given that the protocol called for minimal instructor engagement in the discussion. The instructor only posted when there was a problem and otherwise left the discussion to the students to manage. This result may be attributed to the level of specificity in directions about what and when to post and the use of a grading rubric available to students before the discussion. These elements may allow students to feel confident in their understanding of what to post and how it will be assessed. Discussions were graded using the rubric, and each student also received written feedback from the professor. Students may have also felt that the rubric and written feedback provided a level of responsiveness that they found useful.
Content analysis of student responses to open-ended questions corroborated the quantitative data from the Likert Scale questions. Student comments indicated a lower level of dissatisfaction when discussions were structured with the Final Word protocol compared with discussions with minimal structure. There were 11 negative comments from a total of 95 students who completed the course evaluation when discussions had limited structure and only three negative comments from a total of 84 students who completed the evaluation when the Final Word protocol was used.
In other words, about 10% of students commented negatively when no structure was in place, compared to 3% when the final word protocol was used. There was only one positive comment in both groups, which may indicate that even with structure, students do not see the discussion as one of the most positive or favorite elements of the course.
The data from open-ended questions is limited because the SEI does not explicitly ask for comments about discussions. Instead, students determined whether it was enough of a positive or negative course element to elicit comments.
The results of this study echo and supplement the themes from the literature over the past several years. Aloni and Harrington (2019) reported that overcoming a lack of student participation in online discussions is possible if professors communicate the purpose and value of the discussion, set clear expectations for responses, and design a discussion structure. The Final Word and other protocols typically used for face to face discussion offer promise as a method to implement structure quickly with simple modifications to revise the directions for use online. Based on work with undergraduate students, Smith (2015) recommended using a grading rubric that awards points for the length of posts and numbers of replies so that students who take the time and effort to follow directions are rewarded.
The structured discussion in this study included a developed rubric to correspond to the Final Word protocol. While it may be reasonable to believe that the rubric enhanced the students' perception of the discussions, no students specifically mentioned the rubric in their SEI comments. It may be useful for future research to examine further the impact of this aspect of the structure in discussions.
In a study that considered how to improve student attitude regarding online discussions, Clinton & Kelly (2019) found that undergraduate students who were explicitly told the purpose of the discussion in a video from the instructor found the discussions to be more useful than students in a control group. Interestingly, Clinton & Kelly (2019) also found that stating the discussion purpose did not impact students' thoughts about how interesting or enjoyable they found the discussion. Although the Final Word study described in this paper occurred with graduate students who had already earned masters' degrees, similar results were found. Results from the qualitative data did indicate that students had less of a negative perception of discussion when the purpose was clearly stated as a part of the Final Word protocol. Still, the incidence of positive comments did not increase.

Recommendations
This study's results indicate that student perception of online asynchronous discussion did improve when the structure was added to discussions with a specific statement of purpose, specific directions for participation modified from the Final Word protocol, and a grading rubric. The population for this study was students who chose to enroll in a school leadership certificate program while also holding full-time jobs as teachers or school leaders. This study was initially undertaken as a part of a larger effort to restructure online courses' elements to create conditions in which busy graduate students more fully participate in courses and move beyond the minimum expectations of the "ruthlessly efficient online learner" as described by Darby and Lang (2019). The study results support the use of modified protocols to add structure to the discussion, but more work remains for professors of online courses and researchers.
This study was limited to a population of graduate students, so replicating with undergraduates would help eliminate the assumption that less experienced students would respond in the same manner. Additionally, participants in this study were working K-12 teachers and leaders who were told that discussion protocols such as the Final Word might be useful strategies in their classrooms. The perception of discussions using protocols from students who do not have a similar practical or professional use may differ. This study was also limited by the fact that the open-ended questions in the Student Evaluation of Instruction (SEI) did not ask students to comment on class discussions specifically, and one of the Likert Scale questions asked student perceptions of class discussions and/or other activities. A data collection tool, more specifically tailored to online asynchronous discussions, may yield more precise results.
It was clear that student perception of online asynchronous discussions improved when the structure was added, but this study did not examine the effect of structure on the level of student engagement with the course material or the impact on student mastery of that material. Further research is necessary to address the theory of action from this study, which posited that if a structured protocol resulted in more positive student perception of discussions, then professors would be more likely to use them, and student-to-student interaction and student to content engagement would, therefore, increase.
Finally, this study investigated the impact of only one specific protocol, but many more exist and are available from The School Reform Initiative and other open-source providers. Given the positive effects of the Final Word protocol, other protocols provide promise as methods that could similarly enhance online asynchronous discussions. Previous literature states that students benefit from a clear purpose and directions in discussions. Modification of existing discussion protocols provides the professor with a template from which to work. Creating discussions that students perceive to be effective may also of interest to instructors who teach online for the first time because of the global pandemic. The practice of modifying discussion protocols initially developed for use in face-to-face environments may be a simple method for instructors with limited time and experience with an online course to use a strategy that students find engaging.