The Effects of an Online Teaching Material Integrated Methods on Students’ Science Achievement, Attitude and Retention

The International Journal of Technology in Education (IJTE) is a peer-reviewed scholarly online journal. This article may be used for research, teaching, and private study purposes. Authors alone are responsible for the contents of their articles. The journal owns the copyright of the articles. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of the research material. All authors are requested to disclose any actual or potential conflict of interest including any financial, personal or other relationships with other people or organizations regarding the submitted work.


Introduction
The importance of the integration of technology into education has emerged once again with the emergence of unexpected extraordinary situations such as interruption of face-to-face education because of COVID-19 pandemic in many countries. For this reason, it is paramount that technological applications such as virtual classroom, distance education, and online concept teaching, which are not as common as face-to-face education, are supported by new studies. As technology develops, technologically enhanced environments must be integrated with teaching programs to support learning. Online learning environments provide meaningful interactions to learners (Woo & Reeves, 2007). With the internet and web-based technologies, computer-based concept maps provide more opportunities such as limitless multimedia and collaborative learning via online knowledge sharing (Cañas, Ford, Novak, & Hayes, 2001;Korur, Toker, & Eryılmaz, 2016;Wang, Cheng, Chen, Mercer, & Kirschner, 2017). Usage of advanced technologies in education urges researchers to find out how these technologies affect learning Jonassen, Howland, Marra, & Howland, 2008;International Journal of Technology in Education (IJTE) 23 Nisyah, Pramuda, Mundilarto, Kuswanto, & Hadiati, 2019).
Usage of technology-based concept mapping on interactive boards promotes students' attitudes toward science courses and is effective in improving their achievements (Hwang, Wu, & Kuo, 2013). For example, with the help of an innovative Online Advance Organizer Concept Teaching Material (ONACOM), seventh graders learned the concepts of ‗Light' unit with an easier, more meaningful, more permanent and entertaining approach (Korur et al., 2016). It is crucial to investigate whether or not the ONACOM integrated teaching methods decreases cognitive load emphasized by Sweller (1994), supports dual coding stated by Clark and Paivio (1991) and eases meaningful learning defined by Ausubel (2000) for primary school students. This quasi-experimental study was used to investigate the effects of two ONACOM-integrated instructional methods, the expository learning (EL) and problem-based learning (PBL), in teaching ‗force and motion' on the learning outcomes of fourth graders.

Background
The idea of bringing concepts and digital multimedia contents together, are based on various theories. It is crucial to understand students' conceptual learning and acquisition of conceptual knowledge, meaningful learning theory, cognitive theory of multimedia learning, and neuroscience of learning (Schunk, 2012). Ausubel (2000) stated that learning new concepts and acquiring new knowledge become more meaningful when these are associated with former concepts. Hence, meaningful learning requires a well-organized knowledge structure as well as advanced correlation amongst concepts (Novak, 2002). In this context, computer-aided concept mapping or interactive tools might help students understand abstract topics (Schaal, Bogner, & Girwidz, 2010).
Learning is promoted by understanding the hierarchical relationships and linkages between concepts (Ausubel, 2000). diSessa (2002) indicated that there are two different kinds of mental entities; ‗p-prims' that are small, simple and numerous knowledge elements and ‗coordination classes', that are large and complex elements. In coordination classes, there are several mental connections, one of which arise in the mind related to a fact or an event defined as -learning a new word‖ by diSessa and Sherin (1998). In fact, learning a word is similar to learn a new concept, especially in science learning, in the human mind (Ausubel, 2000;diSessa & Sherin, 1998;Novak & Cañas, 2006). Novak and Cañas (2006), similarly, defined that concepts are perceived regularities or patterns in events or objects, or records of events or objects, designated by a label. Based on this idea, concept maps are teaching tools that identify the concepts, given in boxes, and the linking words, relations between them in order to promote conceptual understanding. Mayer (2009) based the cognitive theory of multimedia learning on (i) Baddeley's (1983) model of working memory concerning selection and organization of knowledge and requirement to bring them into conformity with prior knowledge, (ii) Clark and Paivio's (1991) dual coding theory concerning the requirement to support verbal concepts with visuals or vice versa, and (iii) Sweller's (1994) cognitive load theory which stated that excessive cognitive load may affect learning negatively. Supporting learning environments with technological applications, i.e. smartboards and multimedia, is a prominent aspect that is crucial to decrease the cognitive load

Yilmaz & Korur
24 on students' mind and increase their interaction and motivation levels (Sweller, 1994). (iv) The information processing theory, which supports Mayer's cognitive theory of multimedia learning, revealed that students' learning increases when audio materials and animations are used together (Schunk, 2012). It was also emphasized that these two learning channels have limited information capacity when used separately, but using both learning channels eases the activity of learning (Mayer, 2009;Wu, Zhou, & Duan, 2013).
In neuropsychological theory of Hebb (2002), the left hemisphere processes textual stimuli and the right hemisphere processes visual stimuli. For effective learning, both hemispheres must be used coordinately so that the cognitive load per hemisphere could be decreased (Clark & Paivio, 1991). Teaching concepts is not only important to transfer the concepts from short-term memory to long-term memory, but it is also crucial to construct better connections between them to activate a working memory that can plan and direct students' behavior (Chiou, Tien, & Lee, 2015). Since using visuals and audio with graphic organizers makes both cerebral hemispheres simultaneously active, it helps information to be coded to the long-term memory (Paliç & Akdeniz, 2012). With meaningful learning, students achieve better in comparison to rote learning, and they also reach better retention and retrieval of knowledge capacity especially when they activate their working memory and accumulate new knowledge in their long-term memory (Chiou et al., 2015;Novak & Cañas, 2008). Concept maps may bring knowledge into long-term memory towards meaningful learning (Heinze-Fry & Novak, 1990;Martínez, Pérez, Suero, & Pardo, 2013). The structure of concept map help students link between their selfconstructed knowledge and their cognitive structures (Cañas et al., 2005). Concept maps support meaningful learning, as well as problem-solving, creative and dynamic thinking skills (Cañas et al., 2003;Derbentseva, Safayeni, & Cañas, 2007;Goss, 2009). A meta-analysis study of Nesbit and Adesope (2006) showed that concept mapping is more effective in learning than commonplace activities such as reading text, attending lectures and class discussions in achieving knowledge retention and knowledge transfer. Concept maps are useful for learners in a broad range of educational levels, subject areas and settings. According to another metaanalysis study of Horton et al. (1993) concept mapping increased students' achievement while also strongly improving their attitudes towards the subject area they learned.
Some concept mapping tools such as KMap and CmapTools (http://cmap.ihmc.us/cmaptools/) enable the user to benefit from a rich internet environment. The materials, such as Cmap Tools, Inspiration, KMap, and ONACOM enhance students' learning processes in science, provide better understanding on science concepts, make concept mapping easier and more entertaining and integrate teachers' and students' efforts in creation and development of concept maps (Asan, 2007;Briggs et al., 2004;Gaines & Shaw, 1995;Korur et al., 2016;Novak & Cañas, 2006). Köse, Akkuş and Gezer (2014) also reported that computer-based concept maps had a positive effect on students' attitudes towards science. Digital concept mapping tools are enable the learner to reach more detailed information, discover and test it by making cognitive control of the ideas and the learning process (Fatemeh, Ahmad, & Mohammad, 2011). As a contribution to related literature, the specific aim of this study is to evaluate the knowledge retention effects of ONACOM integrated two teaching methods in terms of students' achievement gains.

Review of Studies Investigating Effects of Online Teaching Material on Students' Science Achievement
Technology has brought many innovations in teaching the knowledge of science, and improved the vision of both teachers and students towards learning science (Akerson & Abd-El-Khalick, 2005). Through many aspects of learning science, well-designed technology integration into inquiry-based learning methods can enhance students' learning of science (Abdelraheem & Asan, 2006). Inquiry-based learning may be used with many significant technologies that support modern learning environments such as interactive books, internet-based drawings, simulation environments, educational videos, electronic worksheets, multimedia and hypermedia systems. Inquiry-based learning is more effective than non-inquiry-based learning methods in academic achievement in science and mathematics courses (Kogan & Laursen, 2014;Wilson, Taylor, Kowalski, & Carlson, 2010). The effects of the inquiry method on learning may be increased with technology support (Lee, Linn, Varma, & Liu, 2010;Liu, Liu, Pan, Zou, & Li, 2019;Merritt, Lee, Rillero, & Kinach, 2017).
However, inquiry-based teaching methods such as PBL are very effective in knowledge construction, but they are not widely-used because of consumption of too much time and energy, in addition to being too slow and too expensive (Costenson & Lawson, 1986). On the other hand, the EL method which was emphasized by Ausubel (2000) is one of the indispensable methods of teachers in education and currently used in many educational environments with a high percentage. So, when EL is enriched with newly developed technologies, more affordable and quality learning environments may be created at a lower cost (Ercan, Ural, & Ateş, 2016;Korur et al., 2016). In this respect, Clark (1994) and Kozma (1994) debate teaches us the media (ONACOM) were integrated into two common teaching methods (inquiry and expository) which were both used widely and effectively in real-classroom settings in order to understand whether the change in students' achievement gains was due to the method applied or the media (ONACOM).

26
The ONACOM is not only a multimedia-integrated online concept mapping tool, but it is also an advance organizer enabling rich contexts on the internet to be used and an online learning environment that has semantic and social network structures (https://cidkom.mehmetakif.edu.tr/Whats.aspx). In a previous unique experimental study of Korur et al. (2016), a unit about ‗light' was taught to 300 students from seventh grade by using ONACOM both in the experimental group (inquiry-based learning) and the comparison group (expository learning method). In comparison group, ONACOM was used as a good advance organizer in EL method stages which was proposed by Ausubel (2000). In the experimental group ONACOM was used as an online information source for active learners in 5E method stages which were developed by Bybee et al. (2006). Korur et al. (2016) indicated that, regardless of the method used, ONACOM was found to be effective in both groups because it showed an increase in students' achievement and attitude scores.
Furthermore, ONACOM is found effective not only on students' achievement and attitudes but also on concept teaching and elimination of misconceptions in science (Korumaz, 2018). In this study, although a similar teaching method with Korur et al. (2016) was used in the comparison group, it was aimed to examine how ONACOM has an impact on student outcomes when integrated into a different inquiry method, which is problem-based teaching. This study was carried out with fourth graders, whose tendency towards technology is thought to be different than seventh graders and found worthy of research in this context. Hence, this paper aimed to reveal the effects of integrating the ONACOM into two different commonly used learning methods, inquiry-based (PBL) and expository learning on the achievement, attitudes and retention of fourth graders in the ‗Force and Motion' unit. Therefore, this quasi-experimental study investigated the following research questions (RQ): RQ1: Do the fourth graders' post-intervention scores achieved in the unit of 'force and motion' achievement test and attitude scale toward the 'force and motion' depend on being in the groups that were taught either with ONACOM-integrated PBL and with ONACOM-integrated EL when their respective pretest scores and their science grades in the previous semester (PSGA) were controlled? RQ2: What are the students' gains in terms of achievement, attitude and knowledge retention from the two systematic applications of the ONACOM-integrated methods, PBL and EL?

Design of the Study
Science classes at fourth graders at two different schools were included in this quasi-experimental study which aimed to investigate the relative effectiveness of the ONACOM integrated two separate teaching methods on students' posttest achievement, attitudes and knowledge retention levels towards force and motion concepts. 10 groups from the two schools in total were divided into experimental and comparison groups and the students in the groups used similar worksheets. They took the same attitude and achievement tests at the beginning, at the end and a month after the end of the study. The same instructor taught the unit to the groups within the same teaching time in natural classroom settings. Furthermore, ONACOM was integrated into both the expository and problem-based learning methods for both groups (see Table 1). FMATT: Force and motion attitude scale; it was applied as pretest and posttest.

Study Group
The accessible population was nearly 900 students at 14 public primary schools in Burdur City Center. The two most crowded schools in the city were selected by including all fourth graders in the study. Within the current Turkish education system, it is not possible to assign students randomly to the groups; instead by consulting the participants' teachers for one school, three classrooms were selected for the experimental group, and three classrooms were selected for the comparison group. For the other school, two classes were selected for each of the experimental and comparison groups. The prerequisite permission was granted from the commission of the educational institute of the local public university and the Provincial Directorate of the Ministry of National Education. The students' socioeconomic backgrounds and their ethnic origins were similar in both schools.
Likewise, their science textbooks, the classroom conditions (including projectors and computers) and technical conditions in computer laboratories were not quite different for both schools.
Demographic variables were also compared in both groups to confirm the homogeneity of the groups.

A Force and Motion Achievement Test (FMACH)
FMACH was developed specifically for this study. First of all, a 56-question pool and a table of specifications were formed in accordance with the learning outcomes stated in the national science education curriculum.

28
While determining which questions measured which cognitive level in Bloom's revised taxonomy, we paid attention to supporting each learning outcome under these cognitive levels with an equal number of questions.
Views and criticisms from eight experts of the field (two academicians, two primary school classroom teachers, three science teachers and one Turkish teacher) were gathered for a draft version (4 pages, 23 questions). A pilot study was conducted with 240 students who were previously taught the ‗Force and Motion' unit. After data entry was completed, the discrimination and difficulty indices for the items were analyzed by using the item analysis program ITEMAN (http://iteman.software.informer.com/4.1/). Since the results of the 1st, 2nd and 6th questions in the analysis regarding their discrimination indices were too low and there were other questions that measure the similar learning outcomes, they were removed from the draft. This version of the FMACH were finalized by the experts and it consisted of 20 questions with two parts. In the first part, 8 questions were true/false questions, while in the second part, 12 questions were multiple-choice questions. The Cronbach's alpha reliability coefficient was found to be =.745. After a month, the same FMACH was applied as a retention test. The reliability coefficient of the retention test was found as =.700. An example of a question (originally in Turkish) in the test is given in Figure 1. FMATT was adapted from the study by Taşlıdere and Eryılmaz (2012) by substituting the unit title ‗Light' for ‗Force and Motion' for determine to the students' attitudes toward force and motion concepts. The face validity of the items was determined by utilizing the opinions of the same experts that review the FMACH. FMATT contained 24 items in a Likert-type scoring (5 = strongly agree, 4 = agree, 3 = neutral, 2 = disagree, and 1 = strongly disagree). Each student had a chance to obtain a maximum of 120 points from the scale if the numerical scores of the items were added.
The construct validity of the scale was also analyzed by an explanatory factor analysis by applying it to the same 240 students participating in the pilot study for the achievement test. The suitability of the data for factor analysis was determined by Kaiser-Meyer-Olkin (KMO) and Bartlett's sphericity tests. The KMO value was found to be .892 and could be confirmed to be ‗very good' (Büyüköztürk, 2010). The Bartlett's sphericity test outcome was  2 (276) = 3084.981, p< .05. Therefore, it could be confirmed that the data from FMATT were appropriate for factor analysis. The eigenvalues of the five factors were 1.0 or higher. 63% of the total variance was explained by these factors, which was concluded as high (Büyüköztürk, 2010). The loadings of the items in the five factors were between .470 and .799, which were above the critical limit of .300. The reliability  (Pallant, 2001), it can be said that the results for both the FMACH and FMATT are reliable.

Implementations in the Experimental and Comparison Groups
Three separate ONACOM concept maps were prepared as a good advance organizer for the following subjects in the ‗Force and Motion' unit: (1) ‗Motion and Motion Types', (2) ‗Moving and Stopping Objects' and (3) ‗Effect of Force on Objects'. Major parts of digital content were formed by the first author after taking opinions of experts and then uploaded onto the ONACOM concept maps. In the experimental group, ONACOM was integrated into the stages of the PBL method which was discussed in a constructivist framework by Savery and Duffy (1995): (a) problem, (b) discussion and production, (c) solution of the problem and explanation and (d) evaluation. In the comparison group, it was integrated into EL stages which were designed by Ausubel (2000):  The ONACOM concept map about the ‗Moving and Stopping Objects' topic in the ‗Force and Motion' unit is given in Figure 3. Even if the same concept maps were used in the groups, the usage of the maps was different depending on the methods applied in the groups. For instance, while the teacher began the course by reading an interesting story from the concept map in the comparison group, in the experimental group, they began the course with a problem status written on a worksheet without opening the concept map. An example excerpt from the worksheet of the students is given in Figure 4. Likewise, in the comparison group, concept maps were mostly used under the supervision of the teacher at three stages of the method, especially in presentation and question-answer sessions, while in the experimental group, they were used mostly at two stages of the method. Additionally, in the experimental group, the students were allowed to open the contents in the maps and look inside on their own for 5-10 minutes. In the implementation, various stages were followed depending on the two different teaching methods (see Table 2).
In both groups, worksheets were distributed to the students throughout the course in accordance with the stages of the methods used with ONACOM. During this process, the students answered the questions in the worksheets. In the comparison group, the teacher presented the advance organizer and subject contents by using ONACOM (see Figure 5). At the end of the learning process, the students were asked to fill in the blank concept map in the worksheet for strengthening their cognitive organization. The students in the experimental group played a more active role in the process than those in the comparison group. In the experimental group, the students worked in small groups, used ONACOM for 5-10 minutes freely, produced solutions to the problems and carried out a discussion session about these solutions (see Figure 6).

Data Analysis
The data for five experimental groups in two schools were combined into a single PBL group dataset, and the data for five comparison groups were combined into a single EL group dataset. Initially, descriptive statistics were calculated to investigate the similarity between the groups. To analyze the research questions, multivariate analysis of covariance (MANCOVA) was used to analyze the posttest data for the combined groups. As stated by Tabachnick and Fidell (2007), MANCOVA may be applied if there are one or more co-variates with multiple dependent variables. In this study, the co-variates were PSGA, pretest achievement and pretest attitude scores (gender did not meet the criteria of a covariate).
While the independent variables were the two methods (PBL and EL), the dependent variables were the attitude posttest, achievement posttest and retention of knowledge scores. The analysis was carried out by using SPSS 20. The assumptions of the t-test and MANCOVA were analyzed. Raw data were analyzed in terms of multivariate outliers by Mahalanobis distance values that are evaluated based on Chi-squared critical values, and there were no significant multivariate outliers at p <.001 for the data (Byrne, 2013;Field, 2005).
The normality of the data was tested by the skewness and kurtosis values (see Table 3). They were in the range of -1.5 to +1.5, indicating that there was no violation of normal distribution for the data (Kuannan, 1998). The preliminary assumptions of MANCOVA, which are homogeneity of regression, linearity, sphericity, equality of variance, multicollinearity and independence of observations, were also tested. No significant violations were found.
By utilizing the estimated values for power, error types and effect sizes (d and η 2 ), the sample size was evaluated (Field, 2005;Fraenkel, Wallen, & Hyun, 2012). The probability of making a Type-I error was set to .05, and the probability of making a Type-II error was set to .01. Therefore, the estimated power of the study was set to 1.00-.01= .99. By considering these values and accepting a medium effect size (lower limit for Cohen's d=.5), the estimated sample size was calculated as N = 121 in accordance with Cohen's (1988, p.381) approach. The sample of this study, which included 220 students, met the estimated size.

Findings
The students' outcomes, achievement and attitudes for varying teaching methods, pretest, posttest and retention test scores were analyzed by using descriptive statistics. Gain scores for each group were determined by calculating the differences between test scores. The results obtained from the descriptive statistics were summarized according to the groups (see Table 3). Analysis of the descriptive statistics as seen in Table 3 indicated that the mean achievement posttest and retention test scores of the experimental group were greater than those in the comparison group, while the mean attitude posttest score was greater in the comparison group than that in the experimental group. In general, both groups had good levels of achievement and retention (between 70 and 85) and positive attitudes (between 101 and 120).

The Effectiveness of Two Different Interventions
The first research question of the study was: "Do the fourth graders' post-intervention scores achieved in the unit of 'force and motion' achievement test and attitude scale toward the 'force and motion' depend on being in the groups that were taught either with ONACOM-integrated PBL and with ONACOM-integrated EL when their respective pretest scores and their science grades in the previous semester (PSGA) were controlled?" MANCOVA was conducted to evaluate the effectiveness of the two different interventions as it can equate groups of one or more independent variables -i.e., the students' PSGA, achievement pretest score and attitude pretest scoreand can be used for more than one dependent variablei.e., the achievement posttest score, attitude posttest score and achievement retention test score. The PSGA, attitude pretest score and achievement pretest score were the independent variables that were met the criteria for being a covariate. MANCOVA was applied, and the results of the analysis are summarized below (see Table 4). Notes. N = 220. Hypothesis SD = 3, Error SD = 213; * p < .05.
According to the results, after adjusting for the pre-intervention scores (PSGA, attitude pretest score and achievement pretest score), there was no significant difference in the treatments between the groups on the collective dependent variables (i.e. attitude posttest score, achievement posttest score and achievement retention with these three dependent variables, and the covariates (PSGA, attitude pretest score and achievement pretest score) were controlled, the relative effectiveness of both ONACOM-integrated methods was similar.

Students' Achievement, Attitude and Retention Gains in Both PBL and EL Groups
The second research question of the study was -What are the students' gains, achievement, attitude and retention from the two systematic applications of the ONACOM-integrated methods, PBL and EL?‖ The question was explored for each dependent variable by using a series of paired-sample t-tests. Paired-sample ttests results were given below. The results are explained for each variable under separate headings.

Achievement Scores
The achievement test scores of the experimental group were analyzed, and there were significant differences between the mean values of achievement pretest score (M = 11.00, SD = 2.99) and achievement posttest score 80), and the statistical power was high (.99). Therefore, it was concluded that the effect of the EL method on achievement was large, and there was a significant increase (14.60%).

Attitude Scores
The attitude test scores of the experimental group were evaluated in terms of attitudes, and there were significant differences between the mean values of attitude pretest score (M = 100.92, SD = 16.41) and attitude posttest score (M = 109.81, SD = 12.24; t(106) = -6.265, p< .05). The effect size was medium (Cohen's d = .61).
The mean score difference was 8.89 points in the experimental group. In the comparison group, the data for the attitude score were analyzed, and there was a significant difference between the mean values of attitude pretest score (M = 96.77, SD = 17.72) and attitude posttest score (M = 108.30, SD = 10.78; t(112) = -6.780, p< .05).
The mean score difference was 11.53 points.

Retention Scores
In this study, pretest and retention test data analysis for the experimental group showed significant differences between the mean values of achievement pretest score (M = 11.00, SD = 2.99) and achievement retention test score (M = 14.96, SD = 3.10; t(106) = -12.100, p< .05). The large effect size (Cohen's d = 1.30) and high statistical power (.99) illustrated that the effect of the PBL method on achievement in force and motion concepts was large since there was a 3.96-point mean difference (a significant increase of 19.80%) from the pretest to the retention test. In the comparison group, the analysis indicated a significant difference between the mean values of achievement pretest score (M = 11.72, SD = 3.39) and achievement retention test score (M = 15.49, SD = 3.00; t(112) = -12.243, p< .05). The effect size was large (Cohen's d = 1.17), and the statistical power was high (.99). There was a 3.77-point mean difference (a significant increase of 18.80%) from the pretest to the retention test. In conclusion, both methods integrated with ONACOM were quite effective on retention of knowledge for force and motion concepts.

Discussion
In this study, it was primarily analyzed that the fourth graders' post-intervention scores depend on being in the groups that were taught the unit either with ONACOM-integrated PBL or with ONACOM-integrated EL. One of our main rationales of this investigation was that Korur et al. (2016), emphasized when ONACOM was integrated into the methods in a coherent way, ONACOM-integrated methods, including the inquiry-based and expository learning methods that were used in any classroom conditions might provide positive contributions on students' science learning. The research design was partly similar to that used by Korur et al. (2016) in a way that the conditions for both groups were matched instead of applying the selected teaching material only for the experimental group. However, unlike the study of Korur et al. (2016), in this study, the grade level of the application groups was different (4th graders) and a month after the posttest intervention, the achievement test was applied as a retention test to examine the students' knowledge retention levels. During the PBL applications in experimental group, the students actively used ONACOM as a rich information portal which was supported with various multimedia contents while producing solutions to problems related to the force and motion. The comparison group was lectured by using the ONACOM-integrated EL method. At the EL stages, ONACOM was used as an innovative advance organizer, which provided conceptual learning meaningfully. ONACOM's well-designed integration into both methods; not only had a positive effect on primary school students' achievements and attitude towards the lesson, but also support their knowledge retention.
The retention test was carried out a month after the implementation of an achievement test, it was found that the mean scores were significantly higher than the pretest scores in the Force and Motion unit in terms of both groups. Moreover, this increase was even higher than the mean posttest scores. One of the reasons might be that, after the implementation of the posttests, the students continued to use ONACOM with their own usernames and passwords. Using and analyzing concept maps and digital content on these maps through ONACOM ensure students to keep constructing knowledge and supporting their learning on an advanced level. This is one of the reasons why the retention scores of the students were higher than their posttest achievement scores. On the other hand, in this study, the difference in the mean retention scores of the experimental group was higher than those in the comparison group. Therefore, our findings confirmed that, with inquiry-based learning, the retention levels of students become higher than those in other methods (Fazelian & Soraghi, 2010). Likewise, the results were also supported by the findings of various studies in the related literature indicating that computer-aided multimedia learning environments support knowledge retention (Daşdemir & Doymuş, 2016;Pilli & Aksu, 2013;Pöhnl & Bogner, 2012).

International Journal of Technology in Education (IJTE)
37 In the constructivist perspective of the PBL method, ONACOM helped the experimental group students make coordination between concepts for producing -a scientific concept‖, defined by diSessa (2002), in their minds.
By integrating it into the EL method, ONACOM worked as a perfect "advance organizer" emphasized by Ausubel (2000). The concept maps in ONACOM provided students with the learning of science concepts meaningfully. For concepts where students have difficulty in understanding, they can use ONACOM to control and renew their knowledge. Therefore, ONACOM-integrated PBL method for teaching force and motion unit for fourth graders did not have a significant effect on the students' achievement and attitude in comparison to a similar group of students involved in ONACOM-integrated EL about the same topic. The ONACOM was used not only in experimental group but also in the comparison group by integrating it into different widely-used methods (PBL-EL). This feature of the study was somewhat different from the studies that used media integration only in experimental group and reveal a significant treatment effect on students' learning in the relevant literature Hwang et al., 2011;Pöhnl & Bogner, 2012). In addition, in related literature, there are studies indicating that inquiry-based learning is more effective than non-inquiry-based learning methods in academic achievement in science education (Kogan & Laursen, 2014;Wilson et al., 2010).
With defensible methodology, it is not surprising that finding significant differences between experiment and control groups by using media only for experiment group (Clark, 1994). On the other hand, in this study, MANCOVA results were not significant. In terms of showing the effectiveness of ONACOM, it was systematically integrated into both methods. It has been a supportive finding rather than a disadvantage to demonstrate the superiority of ONACOM usage in both groups. For this reason, in the further t-test analysis of the achievement, attitude and retention scores, there was a significant difference between the pre-and postintervention scores. Both ONACOM-integrated methods had a significant effect on the post-intervention scores, i.e. posttest achievement, posttest attitude and retention scores of the students. A probable reason for the score gains in both groups may be the well-designed integration of media (ONACOM) into the methods.
Consequently, ONACOM enhanced the students' academic achievement and attitudes, regardless of the teaching methods, similarly to the findings of Korur et al. (2016). This is perhaps the reason why similar results were found in the experiment and control groups in previous studies (Asan, 2007;Chiou et al., 2015;Çepni, 2009).
ONACOM helped the students in both groups construct knowledge and support meaningful learning as Korumaz (2018) reported that ONACOM is an effective tool for students' conceptual understanding. Another reason might be that ONACOM may enable students' obtained knowledge by using both hemispheres of the brain in the learning process, as emphasized by Hebb (2002). In ONACOM, pedagogically, concepts were given as to support the dual coding process described by Clark and Paivio (1991), and the multimedia contents in it may decrease students' cognitive load as explained by Sweller (1994). The technological tools such as simulations, visualizations, computer-based presentations and multimedia-based tools provide students with detailed learning and address different learning styles as reported by Parnafes (2007) and Reigeluth et al. (2016).
Furthermore, when online concept teaching tools are used as a part of the teaching method, they support the learning process and strengthen students' comprehension and understanding of science as stated by Korur et al. (2016) and Koretsky et al. (2014).
Additionally, in this study, the mean attitude posttest scores were significantly higher than the mean pretest

Yilmaz & Korur
38 scores in both groups. There might be more than one reason, one of which is that multimedia contents in ONACOM concept maps made the learning process more enjoyable, and so, they had an influence on the students to improve their attitudes towards the courses. Another reason might be that each student who participated had their own profile pages and was able to look into concept maps at any time. In the literature, there are similar studies which showed that being active during the learning process and being a part of the process had a positive effect on students' attitudes (Altun & Emir, 2008;Hwang et al., 2011;Korur et al. (2016); Köse et al., 2014;Wu et al., 2012). In this study, the increase in the mean attitude score of the comparison group (11.53 points) was a little bit higher than the increase in the experimental group (8.89 points).
A reason for this might be that the students in the experimental group had just encountered the PBL method. On the other hand, the students were more familiar with the teaching activities in the expository teaching group since they had taken part in activities related to expository teaching before. This study clearly illustrated that integration of technology into the teaching process increased the students' attitudes towards the course in both groups. The findings of this study coincided with the conclusions of similar studies by Asan (2007), Chiou et al. (2015), Çepni (2009) and Korur et al. (2016).
In this study, with ONACOM integration, the EL method became method more suitable for its objectives where students learn concepts better by obtaining more detailed information from multimedia contents. The PBL method became a more student-centered and effective method. ONACOM was used by the students as a freely available online information source to produce alternative solutions for problems related to force and motion.
On the other hand ONACOM concept maps was created by researchers firstly and shared to the teachers. It may be considered a limitation related to the implementation that teachers did not prepare the concept maps for the unit freely. This is a necessary element at the beginning since ONACOM is a new portal for the teachers.
Based on the recommendations of Clark (1994) and Kozma (1994), in this study, there was no control group that was taught by a teacher who used a teaching method emphasized by the curriculum as usual, without integrating ONACOM. This may be considered a significant limitation but is not a disadvantage according to Clark (1994), who argued that the general approach to the effectiveness of media studies should be toward integrating the media into both experimental and comparison group applications. The aim of this study was to understand whether the change was due to the method applied or the media (ONACOM) and to compare well-planned applications of these two methods.

Conclusions
In accordance with ethical concerns, the required permissions were received prior to the experimental implementation. Furthermore, internal validity factors (Hawthorne effect, implementation and instruments) and external validity factors (generalizability and ecological validity) that could be a threat for the design were checked. No threat that could affect the internal and external validity of the study was determined. ONACOM is an innovative online teaching material and concept mapping tool, through which desired multimedia course contents can be uploaded to concepts. Within the context of this study, concept maps were formed in ONACOM for the 4th-grade science course under the unit named ‗Force and Motion'. Then, multimedia contents were

International Journal of Technology in Education (IJTE)
39 uploaded onto the concepts. By using these concept maps formed for the unit ‗Force and Motion', the experimental group was lectured by using the problem-based learning method, while the comparison group was lectured in accordance with the expository approach. The ONACOM that was consistently integrated into both methods increased the students' achievement and attitude and supported their knowledge retention for any classroom condition by implementing the process considered in this study. One reason of this result might be that ONACOM concept maps and multimedia contents such as images, videos, animations, simulations and stories support and ease learning science concepts in the ‗Force and Motion' unit. The other reason might be that, concept maps and multimedia contents in ONACOM enable dual coding, help decreasing cognitive load and support working memory, and thereby, as Mayer (2009) stated, it is an effective material in multimedia learning. Another reason might be that, with the great harmony of ONACOM with the essential structures of both teaching methods, it helped the students configure scientific information and construct new knowledge easily in addition to the effects of the methods themselves. The methods integrated with ONACOM were different from the ones the students were accustomed to, so, they were more interested in the topics of the course. There might be reasons particular to the students in the experimental group as:  Using ONACOM freely at the PBL stages, students played a more active role in the learning process, felt more responsible about their learning and achieved better in science.
 ONACOM enabled students to manage the learning process according to their own knowledge construction and they liked to learn on their own.
 The various multimedia contents used by the students gave them different perspectives, enabling them to develop creative solutions to problems in learning force and motion concepts.
There might be reasons particular to the students in the comparison group as:  ONACOM provided students with conceptual learning meaningfully as an innovative advance organizer.
 Online concept maps and multimedia contents in ONACOM provided students with learning of concepts of the -Force and Motion‖ unit in detail and helped them construct the connections between the concepts correctly in their minds.
 ONACOM made EL, which is a conventional method, fun, collaborative, more effective and attractive.
It is a claimable conclusion that if ONACOM is consistently integrated into any method with stages indicated in this study, considering the grade level and the science unit, it will be effective, because ONACOM fitted the essential structures and purposes of both methods. ONACOM reduced the possible differences between the PBL and EL methods as much as possible and yielded effective results in both groups. It was revealed that welldeveloped innovative software such as ONACOM will be effective on learning regardless of the course and method chosen by the teacher when applied systematically as in the case of this study.

Recommendations
The effects of online teaching materials may be examined in longitudinal studies. Because changing attitudes requires more time, long term activities should be designed to make changes in students' attitudes toward science. It may be stated that students should be encouraged to use computers and related technology effectively in classrooms for supporting their learning. In today's world where we experience pandemic conditions, we will better understand the importance of online software and continue to improve learning-teaching processes as long as we are healthy.

Note
This study is part of the master's thesis written by Emre YILMAZ supervised by Associate Professor Doctor Fikret KORUR from the Institute of Educational Sciences of Burdur Mehmet Akif Ersoy University.