Assessing Multidimensional Energy Literacy of High School Students: An Analysis of Rasch Model

The research aims to assess multidimensional energy literacy among high school students. The research employed case study method. Participants involved in this research were 30 students (17 male and 13 female students) who had studied the concepts of energy. The data collection instrument was a questionnaire consisting of 25 multiple-choice questions for cognitive aspects, and 17 and 10 statement Likert-scale items for affective and behavioral aspects respectively. The data gathered were analysed using Rasch model. The results of analysis by using Ministep 4.4.1 in the form of Wright map display indicate that the literacy level of high school students on energy is discouragingly low and the teacher has not address students’ energy literacy in learning process optimally. In addition, scores on affective aspect are higher than cognitive and behavioral aspects. These findings provide information for future curriculum development, teaching and learning that can improve the energy literacy of high school students.


Introduction `
Issues on energy consumption have drawn serious concerns from the global community (e.g. [1][2]). Enerdata [3] places Indonesia on the top 11 position among the countries with highest energy consumption in the world. Such high level of energy use ha770s an impact on rapid global climate change (e.g. [1,4,5]). To reduce energy consumption, one is required to at least understand how energy can be conserved or, in other words, energy literate (e.g. [6][7][8]).
Energy literacy was first introduced as a term in the 1970s [17] [18]. Initially, energy literacy only had two important aspects reviewed, namely knowledge aspect and affective aspect [10,18]. In its development, energy literacy has been viewed as a multidimensional ability that includes three aspects: cognitive aspect, affective aspect, and behavioural aspect (e.g. [1,2,7,12,19,20]). Therefore, energy literacy does not stand alone but is a combination of various aspects (see Figure 1). Within this framework, education plays a role in improving one's literacy on energy use in an attempt to use it wisely and efficiently (e.g. [1,2,5,[8][9][10]).
The concept of energy is generally taught in physics at high school. Many countries have conducted researches related to energy literacy at schools as a pre-emptive measure to global energy issues, as those in USA and Canada by Barrow & Morrisey [10]; in USA by DeWaters & Powers [2] and Demeo, et al. [11]; in Taiwan by Yeh,et al. [12] and Chen, et al [7]; in UK by Cotton, et al [13]; in Malaysia by Lay, et al [14]; and in Indonesia by Sukendar and Setiawan [15] and Pradana, et al [16]. However, investigations on energy literacy in Indonesian schools, both on students' level of energy literacy of  In measuring the three aspects, an instrument commonly used in assessing students' energy literacy is the modified version of the Energy Literacy Questionnaire (ELQ) [20][21]. The current research also used the modified version of ELQ as the main reference (e.g. [14,[21][22][23]). In addition to questionnaires, the current research employed a series of interview session with teachers about learning processes carried out either inside or outside the classroom. The interview was used in data triangulation when analysing the results of ELQ.
In analysing the data on instruments to measure all aspects of energy literacy, researches on energy literacy in schools have relied heavily on parametric and nonparametric tests using Microsoft Office Excel software [2] [24] and SPSS [1,2,10,13,14,[22][23][24]. Aside from those two ways, there are other alternatives that can be used to analyze response analysis; one of which is Rasch model. The Rasch model allows researcher to gather a more comprehensive and elaborate result. It was designed by Georg Rasch for constructing tests with items and persons parameter (e.g. [25][26][27]). Georg Rasch developed an analysis model of item response theory (IRT) [28][29]. Then, this model was popularized by Ben Wright [27,29] in the form of the Rasch model application for educational assessment using computer software designed for the Rasch model application [28,31]. The Rasch technique allows the data to be expressed on an interval scale, people measure on the logit scale (intervals) to be calculated, and items are measured on the same logit scale to be calculated [25]. Rasch model will create a hierarchical relationship between the respondent (person) and the items used. Interval scale logit unit for persons and items are same, these two things can be compared directly so to produce more complete information about the tests performed and the ability of students who take the tests. The output table section is on the menu bar that will be used in the analysis of this study. Such complete and detail analysis is still rarely found for literacy research, particularly that on energy literacy.
In response to the problems elaborated above, the research is aimed at investigating the level of high school students' energy literacy from cognitive, affective, and behavioural aspects. These three aspects will be analysed with the Rasch model, assisted by Ministep 4.4.1-software used in Rasch analysis. The results of the software analysis is then combined with the results of the interview on the learning process and used learning media. The research is expected to provide information for future curriculum development and to serve as the basis for teaching and learning models to improve energy literacy among high school students.

Participants
Participants involved in this research were 30 students that had major in the natural sciences at one of the private high schools in Bandung, Jawa Barat, Indonesia. The sample consisted of 17 male students and 13 female students who had studied energy concept in Physics. On the age average students, the age was taken as the research sample was 16 years old.

Research design
The design of this research was a case study. This case study used a single-case design. Beforehand the participants were taught about energy concept by a physics teacher. Then students were asked to fill in a series of Energy Literacy on-line questionnaires. Afterwards, the teacher was interviewed and asked a number of questions to confirm findings related to teaching and learning activities in the classroom about energy literacy. Students' responses were then analyzed using the Rasch model with Ministep 4.4.1 software and then triangulated with the teacher's responses from the interview sessions.

Instruments
The instrument used in this research is a modified version of the Energy Literacy Questionnaire (ELQ). This modified version of ELQ was developed by DeWaters and Powers [19][20]. The questionnaire consists of 56 questions that cover cognitive aspects (38 questions), affective aspects (17 questions), and behavioral aspects (10 questions). The instrument is declared valid and reliable with internal consistency of each aspect of the original ELQ version, measured using alpha reliability coefficient, with the results obtained was .79 (cognitive aspects); .83 (affective aspects); and .78 (behavior aspects). All results are acceptable in terms of internal consistency.
For the current research, the Energy Literacy Questionnaire (ELQ) was further adapted to be in line with the content of Indonesian national curriculum and the context of Indonesian society. The questionnaire was then further validated by physics content experts from Physics Education Department of UPI, Bandung and a linguist from UPI Language Center, Bandung. The instrument consists of 25 multiple-choice questions for cognitive aspects, 17 statement items with Likert scale for affective aspects, and 10 statement items with Likert scale for behavioral aspects. Table 1 shows questions distribution of energy literacy questionnaire for three aspects. The cognitive aspect score is given code 'M' for male student and 'F' for female student, while 'C' code was given for the number of items. Twenty-five questions of cognitive aspect were given in multiple choice form. Meanwhile, the affective aspect items were given 'A' code. The instrument of affective aspect uses Likert scale statements. Each answer is given score 1-5, with "1" for "strongly disagree" and "5" for "strongly agree". As for behavioral aspect, each item is given 'B' code. Similar to the affective aspect, behavioral aspect is delivered in the Likert scale statements. Each answer is given score "1-5", with "1" for "never" and "5" for "always". The three aspects were developed based on the indicators in Table 1, where energy-saving is one of the topics in question (see Figure 2).
In addition to the questionnaire, interview with structural questions was conducted to obtain information needed about learning process and learning media used.

Research analysis
The participants' responses were analysed by utilizing Ministep 4.4.1., a software that analyzes using Rasch model. The output tables used in this study are  -20). Student reliability shows consistency of students answer. The criteria for individual suitability are shown by outfit mean square (MNSQ) and outfit Z-standard (ZSTD) with .5<MNSQ<1.5 being the acceptance value for MNSQ and -2.00<ZSTD<+2.0 being the acceptance value for ZSTD [28].

Cognitive aspect
The results of the Rasch statistical analysis on cognitive aspect from  Figure 3. Figure 3 displays the value of person reliability .59 and .62, which suggests that the reliability for a person is categorized as "sufficient". In the meantime, person measure -.56 shows that the average ability of the students is lower than the item difficulty (set by default at .0). The Cronbach alpha value of .58 falls to "bad" category. The outfit mean-square value of 1.00, this value falls into acceptable category, which indicates the excellent conditions for measurement. Moreover, the outfit standardized Z value of .03 is categorized acceptable too, which suggests that the data has a logical estimation.  5 Findings above mean that cognitive aspect falls into "low" category. This result is similar to findings from previous research results (e.g. [2,20,32]). Other findings (e.g. [14,23,33]) confirm that cognitive aspect obtains the lowest score when compared to other aspects. Another analysis related to the cognitive aspect is shown in Figure 4. Cognitive aspect scores depicts that student 05F (female student) has the highest ability and student 27M (male student) has the lowest ability. Another information is that the most difficult question is C23, with C01 the easiest question. Item C23 asks the students about the changing of energy sources, and almost all students fail to answer the question.

Affective aspect
The results of the Rasch statistical data on affective aspect from Table 3.1 of Ministep 4.4.1. is shown in Figure 5. Figure 5 depicts the value of person reliability of .14 and .31, of which the reliability for a person included in the lowest category. Yet, person measure at .87 shows that the average ability of students is higher than item difficulty (set by default at .0). The Cronbach alpha value of .29 is categorized as bad. The outfit mean-square value of 1.01 indicates that it is good condition for measurement. Furthermore, the outfit standardized Z value of -.07 has a logic estimation as it belongs to acceptable category. This finding is similar to previous research (e.g. [14,23,33]) stating that the affective aspect obtains the highest score of the three aspects. Another analysis related to the affective aspect is illustrated in Figure 6. Affective aspect scores shows that students F18, M12, M22, and M28 have the highest ability and F25 (female student) has the lowest ability. Additional related information in the figure is that the lowest item score is for A04. Item A04 is a statement about how they use energy that produces solutions to global energy issues; almost all students get low score. On the other hand, the item with the highest score is A03. Item A03 asks students' opinion about the importance of saving energy, and all students receive high score. The scores indicate that almost all students strongly agree that saving energy is important.

Behavioral aspect
The summary of the Rasch statistical analysis on behavioral aspect from  Figure 7. Analysis on behavioral aspects shows that it is not better than affective aspect,   Figure 7 shows the value of person reliability .43 and .54, which the reliability for a person belongs to the lowest category. Nevertheless, person measure at .27 suggests that the average ability of students is slightly higher than item difficulty (set by default at .0). The Cronbach alpha value of .43 is categorized as "bad". The outfit mean-square value of 1.00, this value is in the acceptable category, which implies that it is in excellent conditions for measurement. The outfit standardized Z value of -.01 is in the acceptable category, which means the data has a logical estimation. Further analysis related to the behavioral aspect is shown in Figure 8.  From the students' scores of behavioral aspect, student M02 has the highest ability and students F08 and M03 are the lowest. Additionally, the lowest item score is B09's. Item B09 is a statement about their behavior in promoting energy-saving by encouraging their families to buy energy-saving lamps. Almost all students receive low score in this respect. On the contrary, the item with the highest score is B04. Item B04 asks about the students' behavior when the computer in their home is not being used; all students obtain high scores. In that, all students always turn off the computer when they do not use it.

Interview
The teacher, who was knowledgeable on energy literacy, was also interviewed. The teacher believed that energy education was very important in schools. However, the teacher's practices of energy literacy were not apparent in the class. The teacher did not use a specific learning model to increase students' energy literacy and did not include global issues about energy in the learning materials. Learning media used was in the form of video and laboratory equipment. In the terms of energy literacy promoting activities, the school stakeholders had encouraged students to do energy-saving habits, such as turning off lights and air conditioners when not in use. However, in practice some students ignore them.

Conclusion
In general, the multidimensional energy literacy of high school students with Rasch model for the three aspects were categorized low. These findings provide information that the learning process carried out by the teacher has not been able to improve students' energy literacy. Therefore, the learning process must be changed, both in terms of learning models and learning media used by teachers. In addition, the curriculum should not only emphasized cognitive aspect, but also other aspects such as affective and behavior.