Paper—An Inquiry into Students’ Metacognition and Learning Achievement in a Blended Learning Design An Inquiry into Students’ Metacognition and Learning Achievement in a Blended Learning Design

In blended learning, the borders between classroom and online learning have been blurred by technology. It is approximately applied in the fourth industrial revolution era such as higher education. This study is aimed to investigate the progression of undergraduate students' metacognition of blended learning and their achievement in the environmental chemistry course. Quantitative method was employed using a quasi-experiment, while applying a control design of non-equivalent post-tests group. The population was fifty-nine students in an environmental chemistry course at one of the public universities in Indonesia. Twenty-nine students were randomly selected to form the experimental class while the remaining thirty formed the control. The experimental class experienced blended learning, while the control had conventional learning. At the end of the course, the metacognition of both classes were analyzed with the Testlet instrument which measures their cognitive achievement. The result showed different progress regarding student’s metacognition. In addition, it showed the same trend caused by various factors, such as difference in learning experience and unpreparedness of students in blended learning. Keywords—Blended learning, environmental chemistry, Testlet, metacognition


Introduction
In this fourth era of the industrial revolution, everyone can access technology better, including generation Z which was born from 1995-2010 [1]. They were generally characterized by high technological ability [2] and will start taking over various sectors of employment in the next few years. Generation Z has obtained a decent education and technical abilities which require the participation of teachers to create advance learning conditions. Therefore students, especially education majors, must be familiar with technology-based training, can be called also as blended design.
Blended learning combines face-to-face classroom with computer-based learning, both online and offline [3]. As an applied design, it supports both the fourth industrial revolution and 21st century learning techniques. The advantages of blended design in higher education are the flexibility of time and place; encouraging students to be more independent and responsible for their learning, and choosing the most effective strategies according to their characteristics [4]. There are different regulations regarding the implementation of blended learning and they determine the degree at which the two stated learning procedures are implemented. If an institution wants to execute a blended design, it should contain the following four quadrants as presented in Figure  1 [5].

Fig. 1. The Quadrants in Blended Learning
Implementation of blended design pays attention to characteristics for material with a broad range and little calculation. A preliminary study showed that 30.24% of students disagreed when the metacognition was delivered with blended design [3], simply because the role of lecturers cannot be replaced by online learning. Hence, environmental chemistry course is one subject that can be delivered with this type of learning design.
Environmental chemistry is currently one of the compulsory courses for chemistry education majors. Initially, it was an elective but along with the increased number of students each year, it became compulsory. As an overly applicative subject, environmental chemistry has a comprehensive scope of material and due to its extensive nature, lecturers do not have time to deliver the entire material in the class. However, blended design tends to unravel the solution to this problem. It has resulted in the collaboration of students at the University of Hawaii, Honolulu, in USA with those of Kansai University students in Japan [6]. Students' performance to conventional learning was showed following its implementation at Timisoara University [7]. This study implemented blended design for the environmental chemistry course. However, metacognition ability is one essential skill required by students. This skill can be interpreted as a person's awareness of their knowledge and how they are able to regulate it [8].
Metacognition was developed in 21st-century learning and was first introduced in 1970 by Flavel as "thinking about thinking" [9]. Based on these definitions, cognitive knowledge and regulation are the two recognized components of metacognition. In addition, the research at Meghalaya, India, concluded that metacognition influences students' performance [10], irrespective of gender and settlement. From this study, there is increasing demand of this skill and it urgently needed to develop. Students are expected to know the weaknesses and strengths of their skill's indicators, the lower can be improved, while the higher ones can be further developed. The standard instrument for measuring metacognition is the Metacognitive Awareness Inventory (MAI). The MAI contains several questions measuring indicator of metacognition skill. Students may provide incorrect answers when instruments such as questionnaires are used as the measuring skills. Although, they can give the correct answers but not following the actual question. This condition makes the measurement results less valid. There are available resources that claim to measure students' cognitive abilities but they are not aware of the procedural steps of implementation. Blended design can help to measure cognitive skills.
Online learning enables new and unique ways of capturing and representing metacognitive knowledge, such as graphical reification and (abstracted) replays and reviews. Additionally, they can improve traditional approaches, like self-explanation and collaboration, by presenting them in a more dynamic and exciting form. Computers have potential as metacognitive tools. For example, through metacognition ability to record interactions with users, they can become powerful reflection tools and having captured the actions of the student performing the task, these can be played back, adequately abstracted, and structured. It helps to create awareness of the learning process while enhancing performance on the learning task through reflection and problem-solving paths.
As collaborative devices, they can be programmed to support group planning, monitoring, and evaluation of the learning process. Students in a small group can look back over their solution paths and compare them with other members. This should trigger a reflection on which learning procedure to improve [11]. Furthermore, "Learning Companions," acting as peers encourage the students to reflect and articulate their actions [12]. Progress of metacognition in the online platform can be traced by computer learning, collaboration, graphic reification, task-reply, reviews, and selfreflective activities. On the other hand, reflective prompts or guided questioning requires learners to justify their ideas and make them explicit [12] [13].
The Testlet instrument contains stems with several questions [14], which was initially developed to measure students' cognitive achievement. However, in the last few years the instrument has been developed to measure various skills, such as science process [15], evaluating reading comprehension [16], and generic science [17]. It combines the advantages of multiple-choice and essay questions [18], with the ability to speed up the lecturers in correcting answers and still be able to analyze students' answers.
In this study, the Testlet instrument was used to measure students' metacognition. It has ten stems, and each contains one main and two additional questions. It was administrated online in the university courseware. Once the students submitted all of the answers, they immediately obtained the score results. Integrated Testlet was used, where an answer to the main question supports the additional ones. Hence, anticipating the students to cheat or guessing the solutions.
This study explores two research questions, based on the explanation above: (1) Is there any significant difference in the students' metacognition between experimental and control class? (2) In what extent are the differences in students' metacognition between two classes? 2 Methodology

Research design
This study used a quantitative approach [19] and the data were presented to describe students' metacognition and cognitive achievement. The population was fiftynine students in the environmental chemistry course. Twenty-nine students were taken randomly as the experimental class, while thirty formed the control. The same lecturer taught them and the Testlet instrument, which measured both their metacognition and cognitive achievement. They both remained constant and unaltered. The research design is shown in Table 1. Blended learning in the experimental class was delivered in eight meetings. Five for face-to-face, and three for online-based learning. This group was adjusted to the respective institutions. The model determined the percentage of blended learning as 60% face-to-face and 40% online-based courses. In the experimental class, five meetings of face-to-face learning were delivered by a lecturer in the classroom while the other three meetings were carried out in the following order: Online discussion, independent learning about waste management, and assessment with Testlet instrument. Overall, computer-based used Moodle's Learning Management System (LMS). The details of blended learning in the experimental class are presented in Table 2. --Conventional learning in the control class was conducted with direct approach (face to face) and a project task in which they observed a treatment plant. The assessment was given with the same Testlet instrument as in the experimental class and it was carried out online at the same place and time.

Research instrument
The instrument used in this study is a set of Testlet provided online on Moodle, consisting of ten stems, where Testlet contains three questions. The total items within the Testlet are thirty and they each contain an indicator of metacognition. They include; planning, evaluation, declarative, procedural and conditional knowledge. Students from both classes worked at the time and place and got the score when they finished working on all thirty questions. Thereafter, their responses were analyzed to obtain the score of the Testlet.
The instrument measured the cognitive skill of students for environmental chemistry courses and the scoring guidelines used a common multiple-choice method. This study was devoted only to discuss Testlets for measuring students' metacognition and achievement. Integrated Testlets were used, where, the answer to one item affects the other within the same stem. The guidelines are presented in Table 3 [8]. Table 3. Scoring guideline of the integrated Testlet Scoring Guideline Score In one stem, the first question is answered incorrectly. 0 In one stem, only the first question is answered correctly.
1 In one stem, only the first and second questions are answered correctly.
2 In one stem, all of the questions are answered correctly. 3

Data analysis
To answer the research question, the Testlet score was examined statistically by analyzing the posttest score. Data from both classes were analyzed for normality, homogeneity, and Mann-Whitney U (MWU) tests using SPSS software version 20.

Test of normality
Normality test is essential before statistical analysis [20]. It showed when a data is normally distributed or not. This type of test is carried out using SPSS software's test such as the Kolmogorov-Smirnov, Shapiro-Wilk, Lilliefors, Chi-Square, and Jarque-Bera [21]. Thjs study used the Shapiro-Wilk normality test, which was first developed in 1965 [22]. This method was chosen because it was valid for small samples, even less than 20 samples [23]. This study had 59 participants: 29 in the experimental class, and 30 in the control. The results of the normality using the Shapiro-Wilk test are presented in Table 4. A data is normally distributed when the significant value (p-value) > 0.05. Based on Table 5, the significant value of the experimental class is 0.703 (p > 0.05) and can be concluded that the data was normally distributed. However, the significant value for the control class is 0.018 (p < 0.05) and it is not normally distributed. To this extent, the statistical analysis used is non-parametric.

Test of homogeneity of variance
The homogeneity test aims to analyze the variance. It is one of the requirements in carrying out the Mann-Whitney U test, executed if the variance of the two groups is homogeneous. The data were not normally distributed and Levene's test was conse-quently used as the homogeneity analysis. Levene's test results are presented in Table  5. Based on Table 5, Levene's test showed that the variances for students' metacognition was F (1.57) = 0.067, p = 0.797 (p > 0.05). This value indicates that the variance of the two groups is homogeneous, and the Mann-Whitney U test can be carried out.

Analysis of students' metacognition
Significant difference tests with SPSS can use parametric and non-parametric analysis. However, before the test, normality and homogeneity of the data should be analyzed. Homogeneous and customarily distributed data can do a significance test with parametric analysis. Results obtained from this study were homogeneous, but one of the groups was not normally distributed. Therefore, the significance test used was non-parametric analysis. An example is Mann-Whitney U, and the results are shown in Table 6.  (2-tailed) .415 Table 6 shows the value of Asymp. Sig (2-tailed) of 0.415 (p > 0.05) indicates that there is no significant difference between students' metacognition in the blended learning design and conventional teaching method. This result showed statistically, blended or conventional learning did not affect the subjects' metacognition. This finding is supported by the percentage of achievement indicators for each group presented in Figure 2.

Fig. 2. Percentage of metacognition's indicators
Another finding showed the achievement of metacognition indicators in the experimental class is better than that of the control, as illustrated in Figure 2. Students with experienced blended learning design had more comprehension in planning, evaluation, declarative, and procedural knowledge. The highest score is the indicator of evaluation and about 46% of students achieved this level compared to only 40% in conventional learning.
Virtual synchronous and asynchronous quadrants require students to be active, independent, and able to practice their decision-making abilities in blended learning. Decision-making is one of the factors needed in metacognition, such as, in planning, procedural, and conditional knowledge indicators. This finding supports previous research blended learning increases metacognition awareness in students [24].
While analyzing the achievement of each indicator, another test was also conducted, determining the number of students' metacognition level which can be categorized into underdeveloped, still very risky, starting to developed, well-developed, and very well-developed [25]the comparison of the level of metacognition from the two groups presented in Figure 3.

Fig. 3. The comparison of metacognition level
Results of statistics showed high scores in the experimental class. Although, it gave lower results than the control when viewed in whole. In the control class, one of the lessons learned was by giving project assignments related to waste management materials, but was delivered online with multimedia in the experimental group. It showed students have to do some experiment, in all techniques of learning. Stumpf's research on the retention capacity in humans showed it would last 90% when obtained by direct experiments and only 50% when acquired through media such as learning videos or multimedia [26]. Information obtained from video or multimedia will also be reduced by 15% after 3 hours and 35% after three days [26]. This study can be a consideration for the development of the next blended learning [27].

Conclusion
The implementation of blended learning design in the environmental chemistry course resulted in a positive impact on students' metacognition. Although there were no significant differences between the two classes statistically, students in the experimental class achieved a higher score in metacognition indicators. Blended learning enhances students' experience through varied online resources, and project assignments can be given together.
The Students in the experimental class with experienced blended design had higher achievement in planning, evaluation, declarative, and procedural knowledge and they strengthened their metacognition. The ones in the control class only dominated in the conditional knowledge domain and they had more practice for taking advantage of all facilities provided in blended learning. Previous research showed, in terms of facilities, students used the Internet and computers for an average of 7 hours/day. Campus Wi-Fi supported its implementation [28]. It was aimed at determining student's readiness in implementing this design approach. Blended learning is successful and basically designed for undergraduate students. It can be used as reference for other cours-

Metacognition Level
The Control Class The Experimental Class es and as a complement method in the higher education institution [30]. As for recommendations, this research should familiarize students by applying it frequently to yield maximum results, both in terms of cognitive and skills development.