Good practices in asynchronous e-learninG — a short Guideline document for polish medical teachers — a pilot study

INTRODUCTION: E-learning is gaining popularity also in medical education. It offers students unlimited access to educational materials, helps meet their individual preferences by adapting various learning styles, and is considered to be at least as effective as traditional lectures. However, this can only be true provided that e-learning is of good quality. Short guidelines may be used to familiarise medical teachers with good practices in e-learning, but they should meet the needs of their users, and some areas may require more attention. They should be identified, and medical teachers should be provided with additional resources covering them. This study aimed to develop a short guideline for Polish medical teachers and determine potentially troublesome areas. MeThODS: A detailed review of the literature was performed to create a guideline on preparing and conducting e-learning classes. The most important items from it were listed as an evaluation template and pre-tested on a sample of 10 e-learning courses in a search for areas requiring more attention. ReSUlTS: Half of the courses did not provide students with a syllabus, and none of them clearly defined intended learning outcomes. Also, adult learning concepts were not introduced satisfactorily. Only seven out of 10 courses used activities at all, and they often tested simple knowledge reproduction, were limited to poorly-written test questions, and placed at the end of lessons. CONClUSIONS: In this pilot study three potentially troublesome areas were identified: defining learning outcomes, application of adult learning theory, and choice of activities. Key wORDS: e-learning quality, e-learning guidelines, medical teachers Disaster Emerg Med J 2020; 5(2) ADDReSS fOR CORReSPONDeNCe: Marek Dąbrowski, Chair and Department of Medical Education, Poznan University of Medical Sciences, 7 Rokietnicka Street, 60-806 Poznan, e-mail: maro.dabrowski@gmail.com INTRODUCTION Growing demand for healthcare specialists in Poland directly translates into increasing numbers of medical students enrolled each year. Simultaneously, no medical school is able to expand its capacity indefinitely. Sooner or later it will have to face problems like limited financial resources and availability of lecture halls, and a shortage of clinical hospitals, patients, or academic teachers and thus risk the quality of medical training. Meanwhile, international studies show many benefits of e-learning methods and their increasing popularity also in medical education Disaster anD emergency meDicine Journal 2020, Vol. 5, No. 2 2 www.journals.viamedica.pl [1,2]. E-learning is highly valued by students and is believed to be at least as effective as traditional lectures while offering lower costs and more convenience and flexibility in terms of time and space [2,3]. What is more, there are also reports suggesting that e-learning may contribute even more efficiently to students’ gain in knowledge, skills, and attitudes, and retention thereof, than traditional lectures [4,5]. Also, the philosophy behind it differs from the traditional approach [4,6]. After all, just because someone has sat through a lecture, does not mean they have learned anything. According to the andragogy theory proposed by Malcolm Knowles [7,8], internal motivation is vital in adult learning. In order for e-learning to work, students have to become more active participants of the learning process and assume responsibility for it [4,9]. This in turn may increase their motivation for learning and help shaping positive habits in terms of continuous professional development [4,10]. Consequently, one of the most important assets of e-learning is that it offers students unlimited and free access to educational materials that can be viewed by them whenever and wherever they want [11]. It is essential because learning can often be spontaneous, students may at some point just feel like doing it, and providing them with meaningful resources may help to make the best of every single moment like that [4,12]. Another advantage of e-learning is that it can adapt various learning styles, multimedia, tools, and activities, in order to best match individual preferences of students and maintain their engagement in the learning process [4]. Moreover, it gives students more control over technical and organisational aspects of their learning process. They can tailor the amount of time spent on different tasks to suit their individual needs, rewind, or pause when they feel tired [13,14]. Obviously, e-learning has certain limitations that have to be acknowledged and analysed before its introduction. Especially in medical education e-learning never can nor never should be seen as substitute for practical training or contact with patients. However, it may serve as an attractive and effective way to provide students with the necessary theoretical knowledge before such activites [6]. Moreover, according to Cook [15], e-learning is not always more cost-effective than traditional learning methods, and its use should be optimised in order to keep it low cost and low tech but still good enough in terms of quality. The success of e-learning methods is also highly dependent on student’s self-discipline and time management skills [14]. Its whole concept is based on assumptions that students will be ready to actively participate in it while academic teachers will be ready to adopt more student-centred approach towards learning and lead good-quality e-learning classes [9]. Therefore, it seems reasonable to evaluate attitudes and readiness of both these groups towards e-learning whenever it is introduced on a wider scale at a given institution. Finally, positive outcomes of e-learning can only be observed if it is of good quality [12]. As a result, it seems vital to promote good practices in e-learning among teachers. Despite its popularity in other countries, there is a paucity of reports on its use in Polish settings, and e-learning still constitutes quite a novelty in Polish medical education. Taking all of the above into consideration, this study aimed to develop a guide summarising good practices in designing and teaching e-learning classes, to be used by Polish medical teachers. It is planned to be distributed among academic teachers of the University and to be used independently by them whenever needed (just-in-time learning). However, particular areas may require additional attention, so the secondary aim of this study was to try to identify them in a pilot study in order to best suit the needs of medical teachers. Asynchronous e-learning (where content is available all the time and students display it at their own pace and schedule) was chosen first due to its convenience, lower cost, and high availability for students. MATeRIAl AND MeThODS Study protocol In order to achieve the aforementioned objectives, the study protocol included three major steps. Firstly, the review of the literature was performed in order to determine good practices in preparing and teaching e-learning classes. The results of the literature search were then used to create a short document on the topic in Polish. Finally, the document was pre-tested on a sample of e-learning courses to determine the aspects that may need increased attention for example in the form of workshops. The review The review aimed to determine good practices and guidelines dedicated to preparing and teaching e-learning classes. A detailed literature review on Piotr Przymuszała et al., Good practices in e-learning — a pilot study 3 www.journals.viamedica.pl the subject was performed in October and November 2018 to identify all studies and guideline documents that concerned quality and good practices in e-learning. For this purpose, PubMed and Google Chrome databases were searched using the keyword ‘e-learning’ in combination with the following keywords: ‘quality’, ‘good practices’, and ‘guidelines’. All detected research studies, review articles, and good practice guidelines from 2008 to 2018 were analysed by the first author on the basis of their content. First, a screening of titles and abstracts allowed the exclusion of search results that clearly did not correspond with the aim of the study (e.g. studies on the influence of e-learning on participants’ knowledge, skills, or quality of care in different morbidities). In cases of doubt or when abstracts were not provided (guidelines), a cursory analysis of content was performed for inclusion or exclusion criteria. Additionally, reference lists from all eligible studies and guidelines were reviewed to retrieve further sources. All manuscripts and guidelines obtained as described above were analysed according to identical inclusion and exclusion criteria. They were included when they related to the topic of the study (presented good practices, guidelines, and good tips in e-learning, participants expectations, etc.), were published in English between 2008 and 2018, and full-text could be accessed. Search results were excluded when at least one of the aforementioned conditions was not met. Finally, the literature search produced 30 manuscripts (research studies, review articles, frameworks — Tab. 1) and 32 eligible guideline documents. Manuscripts and guidelines were further analysed in search of specific examples of good practices in e-learning, which was followed by the formulation of a summary document in Polish aimed at helping Polish medical teachers in preparing and conducting e-learning classes. The tool Despite a long history of e-learning use in many countries, it is still only starting to gain popularity in Poland, especially in medical education. Therefore, some academic teachers may lack experience and remain uncertain how to design, prepare, and lead e-learning classes problems that were often reported to the authors of this study. As a result, a decision was made to formulate a short document guiding academic teachers step by step through the process


INTRODUCTION
Growing demand for healthcare specialists in Poland directly translates into increasing numbers of medical students enrolled each year. Simultaneously, no medical school is able to expand its capacity indefinitely. Sooner or later it will have to face problems like limited financial resources and availability of lecture halls, and a shortage of clinical hospitals, patients, or academic teachers and thus risk the quality of medical training. Meanwhile, international studies show many benefits of e-learning methods and their increasing popularity also in medical education www.journals.viamedica.pl [1,2]. E-learning is highly valued by students and is believed to be at least as effective as traditional lectures while offering lower costs and more convenience and flexibility in terms of time and space [2,3]. What is more, there are also reports suggesting that e-learning may contribute even more efficiently to students' gain in knowledge, skills, and attitudes, and retention thereof, than traditional lectures [4,5]. Also, the philosophy behind it differs from the traditional approach [4,6]. After all, just because someone has sat through a lecture, does not mean they have learned anything. According to the andragogy theory proposed by Malcolm Knowles [7,8], internal motivation is vital in adult learning. In order for e-learning to work, students have to become more active participants of the learning process and assume responsibility for it [4,9]. This in turn may increase their motivation for learning and help shaping positive habits in terms of continuous professional development [4,10]. Consequently, one of the most important assets of e-learning is that it offers students unlimited and free access to educational materials that can be viewed by them whenever and wherever they want [11]. It is essential because learning can often be spontaneous, students may at some point just feel like doing it, and providing them with meaningful resources may help to make the best of every single moment like that [4,12]. Another advantage of e-learning is that it can adapt various learning styles, multimedia, tools, and activities, in order to best match individual preferences of students and maintain their engagement in the learning process [4]. Moreover, it gives students more control over technical and organisational aspects of their learning process. They can tailor the amount of time spent on different tasks to suit their individual needs, rewind, or pause when they feel tired [13,14].
Obviously, e-learning has certain limitations that have to be acknowledged and analysed before its introduction. Especially in medical education e-learning never can nor never should be seen as substitute for practical training or contact with patients. However, it may serve as an attractive and effective way to provide students with the necessary theoretical knowledge before such activites [6]. Moreover, according to Cook [15], e-learning is not always more cost-effective than traditional learning methods, and its use should be optimised in order to keep it low cost and low tech but still good enough in terms of quality. The success of e-learning methods is also highly dependent on student's self-discipline and time management skills [14]. Its whole concept is based on assumptions that students will be ready to actively participate in it while academic teachers will be ready to adopt more student-centred approach towards learning and lead good-quality e-learning classes [9]. Therefore, it seems reasonable to evaluate attitudes and readiness of both these groups towards e-learning whenever it is introduced on a wider scale at a given institution. Finally, positive outcomes of e-learning can only be observed if it is of good quality [12]. As a result, it seems vital to promote good practices in e-learning among teachers. Despite its popularity in other countries, there is a paucity of reports on its use in Polish settings, and e-learning still constitutes quite a novelty in Polish medical education. Taking all of the above into consideration, this study aimed to develop a guide summarising good practices in designing and teaching e-learning classes, to be used by Polish medical teachers. It is planned to be distributed among academic teachers of the University and to be used independently by them whenever needed (just-in-time learning). However, particular areas may require additional attention, so the secondary aim of this study was to try to identify them in a pilot study in order to best suit the needs of medical teachers. Asynchronous e-learning (where content is available all the time and students display it at their own pace and schedule) was chosen first due to its convenience, lower cost, and high availability for students.

Study protocol
In order to achieve the aforementioned objectives, the study protocol included three major steps. Firstly, the review of the literature was performed in order to determine good practices in preparing and teaching e-learning classes. The results of the literature search were then used to create a short document on the topic in Polish. Finally, the document was pre-tested on a sample of e-learning courses to determine the aspects that may need increased attention -for example in the form of workshops.

The review
The review aimed to determine good practices and guidelines dedicated to preparing and teaching e-learning classes. A detailed literature review on the subject was performed in October and November 2018 to identify all studies and guideline documents that concerned quality and good practices in e-learning. For this purpose, PubMed and Google Chrome databases were searched using the keyword 'e-learning' in combination with the following keywords: 'quality', 'good practices', and 'guidelines'. All detected research studies, review articles, and good practice guidelines from 2008 to 2018 were analysed by the first author on the basis of their content. First, a screening of titles and abstracts allowed the exclusion of search results that clearly did not correspond with the aim of the study (e.g. studies on the influence of e-learning on participants' knowledge, skills, or quality of care in different morbidities). In cases of doubt or when abstracts were not provided (guidelines), a cursory analysis of content was performed for inclusion or exclusion criteria. Additionally, reference lists from all eligible studies and guidelines were reviewed to retrieve further sources.
All manuscripts and guidelines obtained as described above were analysed according to identical inclusion and exclusion criteria. They were included when they related to the topic of the study (presented good practices, guidelines, and good tips in e-learning, participants expectations, etc.), were published in English between 2008 and 2018, and full-text could be accessed. Search results were excluded when at least one of the aforementioned conditions was not met. Finally, the literature search produced 30 manuscripts (research studies, review articles, frameworks -Tab. 1) and 32 eligible guideline documents. Manuscripts and guidelines were further analysed in search of specific examples of good practices in e-learning, which was followed by the formulation of a summary document in Polish aimed at helping Polish medical teachers in preparing and conducting e-learning classes.

The tool
Despite a long history of e-learning use in many countries, it is still only starting to gain popularity in Poland, especially in medical education. Therefore, some academic teachers may lack experience and remain uncertain how to design, prepare, and lead e-learning classes -problems that were often reported to the authors of this study. As a result, a decision was made to formulate a short document guiding academic teachers step by step through the process. In order to make it more user-friendly, the document was kept relatively short and amounted to 10 pages. It is divided into five chapters, namely: 1) Course planning and creation, 2) Lessons and choice of learning material, 3) Presentation of learning material, 4) Use of multimedia, and 5) Activities, feedback, and evaluation. Each chapter contains one or two pages. Additionally, a checklist was added at the end to help teachers with fast revision of a prepared e-learning course. The initial version of the document was formulated by the first author based on the results of the literature review. It was subsequently discussed with the second and last authors until consensus was reached. This agreed version was later evaluated using the Delphi method with a panel of three independent experts on the subject of e-learning, which allowed the formation of its final version used in this study.

Study settings
A pilot study was conducted to determine the extent to which individual good practices from the tool are met on a sample of e-learning courses from our University. The most important items from the tool were initially chosen and listed as an evaluation template by the first author. They were later thoroughly discussed by the first, second, and last authors until consensus was reached between them. The final version of the template was used to analyse the e-learning courses. Each item was graded according to a four-point Likert scale from 0 to 3 (where 0 -criterion not met at all, 3 -criterion met fully). Ten e-learning courses were chosen proportionally in order to form a sample of all e-learning courses taught at the University. When only one e-learning course was taught at a given faculty (e.g. physiotherapy or paramedics) it was also included in the sample. Courses from the same faculties were chosen randomly. All courses were evaluated by the first author in order to ensure consistency.
The study protocol was presented to the local Bioethics Commission. As the study protocol did not involve patients or human participants, a positive opinion of the Bioethics Commission was not necessary under the Polish legal system.

ReSUlTS
Ten e-learning courses were analysed in this pilot study. Among them, there were four courses dedicated to medicine students, two courses dedicated to nursing students, two courses dedicated to dietetic students, one course dedicated to physiotherapy students, and one dedicated to paramedic students. Given the aims and qualitative character of the study, the courses were analysed collectively because small sample sizes would make any comparisons between the faculties unreliable. Detailed results of the study are presented in Table 2 as numbers of e-learning courses graded on 0-3 Likert scale for each good practice item. Additionally, the most important findings are described below.
Among the analysed e-learning courses, half of them did not provide students with any syllabus and assessment criteria, and only four e-learning coursed met this criterion fully. What is worse, none of the 10 e-learning courses clearly defined their intended learning outcomes (ILO). As a result, items related to ILO could not be assessed (Tab. 2).
The item 'students have control over their learning' was rated in the following way: 0 means no control; 1 means that students could return to previously viewed slides or lessons, but all newly displayed slides and lessons had to be viewed in a predetermined order with a fixed amount of time spent on each; 2 means that students still had to display slides and lessons in a predetermined order, but they could skip quickly through them; and 3 means full control of students. References to students' previous knowledge and experiences were not included at all in six courses, and only five courses showed at least a small attempt to promote critical thinking, student's reflection, and problem solving. Most courses demonstrated practical application of knowledge, but only two did it moderately well, and none of them showed that this criterion was met fully. Given the usual student's attention span of 25-30 minutes, the time required to complete was also assessed. In five courses no lesson lasted longer than 30 minutes. Proportionally for every lesson exceeding that limit, one point was taken.
Learning material was sorted in order in most of the courses, and most of the slides were limited to only one topic. The text on the slides was predominantly easy to understand. However, occasionally sentences were too long and therefore complicated. No particularly difficult words or unexplained abbreviations were detected. Text was completely free from errors in all examined courses. All courses also achieved maximum scores for using user-friendly colour schemes and appropriate font styles (e.g. utilising Polish diacritical signs) and sizes (comfortable for reading). The layout was also consistent in all presented cases.
All courses utilised the lector's voice. However, only six of them used a friendly, real person's voicein five out of six cases this voice additionally attempted to encourage students' reflection, and these were also the cases in which the audio material was not distracting. Meanwhile, four studies used a cold, automatic voice that was simply reading the text with no time for reflection. None of the courses used hyperlinks. Multimedia were used in all of the courses to different extents (mainly illustrations and pictures, but also graphs, diagrams, and tables). No animations or videos were present in the analysed material.
The multimedia used were generally of good quality, and in most cases they were used purposefully. However, occasionally it was difficult to assess the meaning and purpose of some materials (e.g. a diagram with no information about what the student should pay attention to), and sometimes purely decorative motives were also present. Most courses did not include references in text or in multimedia. Course is free from stereotypes and uses gender-neutral language 0 0 0 10 -Seven out of 10 courses used activities to engage students, but only in some of them activities were diversified and evenly distributed throughout the entire course. For the remaining courses, activities were mostly placed at the end of lessons and included test questions. Of note, all test questions were formulated correctly in only one course. Most of the activities tested simple knowledge reproduction and only three courses used (to different extent) activities promoting higher-order thinking. Most activities provided instant feedback to students after completion. However, no course could be awarded with the highest grade in this category because none of them provided a full and detailed explanation of correct and false answers.

Good practices in e-learning
No course provided all information potentially important for users. In this category the following were included: short content description, welcoming message for participants, teacher contact details, list of lessons with learning outcomes assigned to each of them, estimated amount of time required to complete each lesson, last actualisation date, and technical support contact data. Four courses provided information about a forum assigned to them and invitations for students to visit it. On the other hand, six did not contain any such information and fora remained unexploited. Courses were generally intuitive, and there were no major problems in this area. All of them used gender-neutral language and were free from stereotypes. Most courses tried to acknowledge the needs of disabled people (possibility to increase font size, possibility to read text out loud, avoiding red and green, transcript of lector's voice).

DISCUSSION
To our best knowledge, this is the first Polish study trying to develop a tool promoting good practices in e-learning as well as attempting to assess its quality in search of areas that may require additional attention (e.g. in form of workshops). Since this was only a pilot study on a small sample size, it should not be regarded as detailed assessment of e-learning quality at our university. Its aim was instead to pre-test the guideline document developed earlier in order to determine the exact needs of medical teachers as e-learning content creators. In this context medical teachers should also be considered as adult learners -they learn how to design and lead good quality e-learning classes. Therefore, any intervention (guidelines, workshops) should also ap-ply the andragogy theory described above [7,8]. To enhance learners' intrinsic motivation, there needs to be a clear indication as to why they are learning something and how is it relevant for them. Guideline documents are useful in terms of their convenience -teachers can use them whenever and wherever they want as a quick and efficient reference source to acquire or systematise basic knowledge, unlike workshops for example. This is especially important in terms of problems with long-term retention of knowledge and busy schedule of medical teachers due to numerous other professional obligations [39]. In order to make any document useful it has to be relatively short because long elaborations may scare off users rather than encourage them to use them. Consequently, there is no need to go into detail with every single point. Only the most crucial and problematic ones should be dealt with in greater attention. They can be described more thoroughly in the document or serve as a topic of a workshop or an e-learning course. If a given institution has adequate financial means, more workshops or e-learning courses can be organised, including less burning issues. In this way a teacher can individually asses which one to attend. However, it still seems a reasonable practice to provide them with a short summary guideline document on the topic and indicate which aspects should be dealt with more caution.
Our pilot study results show that medical teachers at the university had no or few problems with aspects like making text simple and understandable, explaining rare words or abbreviations, keeping text free from spelling, punctuation, and grammar errors, making portions of text not too long, using a balanced colour scheme, comfortable font styles, and sizes, and maintaining a consistent layout throughout the entire course. On the other hand, improvement could be made in regards to providing students with important information, use of forums and activities, as well as applying rules of andragogy theory (giving students control over their learning, providing them with opportunities to use their prior knowledge and experiences, critical thinking, reflection, problem solving, and demonstrating practical application of knowledge). Although, due to lack of data from similar studies in Poland, it is hard to compare our results with the literature, one striking observation can be stipulated. Teachers generally managed well in aspects they were accustomed to while preparing slides for traditional classes, but showed problems in ar-eas more specific to student-centred education and e-learning. Nevertheless, aspects like the possibility to communicate with the teacher and each other (e.g. on a forum), instant feedback, teacher and technical support, use of different learning styles, interactivity, and values of adult learning principles are still important for students and maintaining their motivation, and therefore their adequate quality should be assured [9,11]. Furthermore, none of the courses clearly defined its learning outcomes, which is surprising given their role in Polish medical education [40]. It is the learning outcome, not the process of education itself, that should be emphasised. Learning outcomes are useful because they provide a template for a lesson and allow this plan to be adhered to, from the choice of learning material to activities in order to assess whether learning has really occurred [4,9,38]. Unfortunately, due to a lack of defined learning outcomes, this process could not be followed in any of the examined courses. Meanwhile, learning outcomes are also extremely important from the learners' point of view because they provide them with aim, guidance, and information about what to focus on [2,11,38]. The aim should be clear in order to motivate students, otherwise the whole course is just content [11]. Last but not least, e-learning courses should not be discouraging to their users. They cannot be too long, distracting, or force students to view content they already know [2,11]. As presented above, these aspects should also be worked on.

lIMITATIONS
We acknowledge our study had several limitations. First of all, the sample size was small. However, as mentioned above, this was only supposed to be a pilot study in order improve the guideline document and identify areas that probably require more attention while introducing e-learning on a wider scale at the University. Moreover, for a similar reason, e-learning courses from only one Polish medical university were examined in this study, and therefore further studies are required in order to determine whether similar problems are present at other Polish medical schools. Finally, we are also aware of the risk of selection bias because two faculties were represented by only one course each. However, as emphasised above, the aim of this study was not to determine detailed frequencies of individual deficiencies or flaws. We rather tried to estimate what areas might require additional attention in order to help teachers overcome those difficulties. Therefore, inclusion of at least one existing course from every faculty seemed more important and justified.

CONClUSIONS
E-learning can only be effective if it is of good quality. Therefore, it seems vital to promote good practices in designing and teaching e-learning classes among teachers. This is especially true in countries like Poland, where e-learning has recently been introduced on a wider scale and awareness of its concepts and quality may be limited. Short documents with guidelines may constitute a meaningful and convenient way of introducing some basic concepts. The guide presented in this study, despite its relative compactness, covers a broad area of course creation, and its popularisation can draw teachers' attention to particular elements they were not aware of before. However, any educational intervention should meet the needs and expectations of their potential users. Particular areas may require more attention, and they should be identified as soon as possible. In the presented study three such areas were identified: defining learning outcomes, application of adult learning theory, and choice of activities. They can be elaborated by providing medical teachers with additional resources in the form of workshops, e-learning courses, or additional reading materials -according to their individual preferences.