A COMPARISON OF USING L1 (INDONESIAN) AND ENGLISH TO ANSWER ENGLISH LISTENING COMPREHENSION QUESTIONS

English test results show that a lot of Indonesian students obtained low scores in listening. Students say they understand what speakers say in an English listening test but they have difficulties in answering the questions in English. This shows that students’ English receptive skills are higher than students’ productive skills. This research intends to find out what the difference in results is when students answer English listening comprehension questions in L1 (Indonesian) and English, and what the students’ perception is on the use of Indonesian language in English listening test. This research uses the mixed method along with a listening test, questionnaires, and interview as data collection tools. The findings show that these learners have difficulties to answer the test in English. The test answered in Indonesian received better scores than in English. Moreover, the questionnaires and the interview result show positive feedback on the use of Indonesian language. This shows that the learners actually have good listening skills and that their low-test scores are mostly due to their difficulty in expressing their thoughts in English.


INTRODUCTION
Listening is a skill needed in everyday communication and is an integral part of a communication process that allows human beings to understand a message and provide appropriate feedback. Nowadays, listening has even become a subject of study at school and college, and needs to be mastered by students of first, second, and foreign languages.
In Indonesia, listening comprehension has been set by the government as part of the curriculum for foreign language learning. As an official subject for elementary, middle, and high school, listening comprehension is specifically important for high school students as it is one of the English language skills tested in the National Examination.
It is unfortunate, however, based on Susilawati & Fitriah's study findings, Indonesian students' listening test scores are still very low. Their listening comprehension grades are below the standard of mastery learning criteria (KKM) (Susilawati & Fitriah, 2015). They have difficulties in learning the skills, whether due to technical aspects of the listening assessments or issues in understanding the materials.
According to Révész and Brunfaut (2013, p. 33), English learners face numerous challenges in listening. These challenges vary depending on a number of factors, which include task-related characteristics such as task input and task response, or lexical features of a text needed to respond. Underwood (1989), as cited by Gilakjani and Sabouri (2016, p. 127), also discusses several challenges to the effective listening comprehension process: (1) listeners cannot control the speed of a speaker's speech; (2) listeners cannot have words repeated; (3) listeners have limited vocabularies; (4) listeners may not have contextual knowledge of a particular conversation; and (5) listeners cannot concentrate. As a result, learners may not be able to respond to or answer questions asked during the listening comprehension test.
In addition to these many challenges, one in particular persists, and that is listeners' limited response when answering questions in a listening test. They do understand what speakers say but they have problems in responding or answering what speakers ask. This indicates that they may have good receptive skills but limited productive skills, which results in low-test scores.
Related to the use of the first language on English language learning, there are some researchers who support this practice. Shuchi and Islam (2016) posit that the use of the first language on second or foreign language learning in a proper level will not disturb the learning process. The results of some studies have shown that the use of L1 can support and facilitate the process of teaching and learning of the target language itself (Cook, 2001;Tang, 2002;Wells, 1999). Moreover, Sharma (2006) also stated that the use of the learner's first language does not ruin the nature of the target language but rather emphasizes the function of the specific second or foreign language learning.
Specifically, the use of L1 in listening tests is supported by Cook (2001, p. 416), who posits that the first language can be utilized to learn a second or foreign language and one way to do this is by implementing it in a test. This statement is also supported by some previous studies. Filipi (2012) studied the effect of using the first language in foreign language listening tests in Australia. She found that the tests given to the learners were difficult and had the issue of fairness. This issue happened because the tests mostly had to be answered using the target language which was still learned to be mastered by the learners. In her study, questions of the foreign language listening tests were written in the first language to answer the case of issue of fairness. She discovered that the use of the first language helped learners to answer the global information type of questions because they could describe their comprehension more. The study conducted by Filipi has a similarity to the present study, which is both focused on the use of the first language in answering foreign language tests.
A similar study was conducted by Amirian, Zareian, and Suma (2016) from Hakim Sabzevari University. They studied the effect of using the first language and target language on learning the incidental vocabulary through a story. They used the vocabulary tests and compared tests answered in the first language, target language and both the first and target language. The instruments used and procedures applied in the study were nearly the same as the present study. Both studies compared the tests answered using the L1 and the English language.
In addition, Bartlett (2017) also studied the use of first language on second language learning in Japan. Based on his findings, the students showed that they need the first language to learn the second language because it helps them to comprehend complex language and to confirm that their understanding was correct. This was believed by the students to improve their ability to comprehend the target language.
The previous studies show that the first language could be utilized to learn the second or foreign language by implementing it on a test as what was stated by Cook (2001). Since there has not been any study conducted to gain a deeper understanding about this in Indonesia, the researchers of this study conducted a research in which Indonesian high school students were asked to answer a listening test in L1 (Indonesian) and in English. The purpose is to find out whether the cause of the problem is learners' poor listening skill or learners' poor productive skill. In addition, the study also wanted to know the students' perception on the use of Indonesian language in English listening test. The two research questions in this study are 1. What is the difference in results when students answer English listening comprehension questions in L1 (Indonesian) and English? 2. What is the students' perception on the use of Indonesian language in English listening test?

Research Design and Instruments
A mixed-methods research design is adopted in this study, meaning that both quantitative and qualitative methods were collected and analyzed (McMillan & Schumacher, 2001).
The basic assumption of this research design is that "the uses of both quantitative and qualitative methods, in combination, provide a better understanding of the research problem and question than either method by itself" (Creswell, 2012, p. 535). We thus integrate the findings and draw inferences using both the qualitative and quantitative approaches (Tashakkori & Creswell, 2007).. The data for the quantitative method are taken from listening tests scores and questionnaires, while the data for the qualitative method are taken from open-ended questionnaires and interviews. This research is conducted using an ex post facto quantitative research design with a focus on criterion group design (Nunan & Bailey, 2009, p.70). Ex post facto study or after-the-fact research is a category of research design in which the investigation starts after the fact has occurred without interference from the researcher. In this study, the researcher only compares the results of listening tests with answers written in Indonesian and in English in four eleventh-grade English classes.
For the qualitative data, the researcher collects data using descriptive survey with pre-and post-test questionnaires. In the pretest questionnaire, each learner is asked to explain his/her history of English language learning with a particular emphasis on listening. In the post-test perception questionnaire, learners are asked to describe their views toward the use of Indonesian language in the listening test. The researcher also conducts interviews to add supporting data to the post-test perception questionnaire.
The research sample comprises 120 science students in four eleventh-grade classes. They are chosen because they have longer histories of learning English compared to the tenth graders and this make them a better fit for the ex post facto research design. The sample is collected using purposive sampling technique mostly because the sample has been provided and the researcher does not have authority to conduct random sampling.

Procedures
The research was conducted for one month, from the first to the fourth week of April 2019 at High School X in Bekasi, West Java. A trial was conducted in advance to test the validity and reliability of the research instruments, which include English listening tests and perception questionnaires. The researcher uses Pearson Product Moment to measure the validity of the tests and questionnaires and Cronbach's alpha to measure the reliability of the two instruments. Two types of open-ended listening tests are used in this study: one with the answers in English, and the other with the answers in Indonesian. Both tests are analyzed using descriptive and inferential statistics by comparing means and the t-test for difference between groups. The study also implement inter-rater reliability with Cohen Kappa statistics to ensure the strength of the listening tests' validity. The perception questionnaire is designed based on learners' frequency rate (Dörnyei, 2003;Schweers, 1999) using the Likert scale.
After the trial was conducted, the two types of listening tests were given to each class that participated in the study. The test with answers written in English was given first. A week later, the second test with answers written in Indonesian was given. The perception questionnaire was distributed after the second test on the same day. In the fourth week of April, interviews were conducted with 12 students from each class based on their test scores: the highest, the average, and the lowest.

FINDINGS AND DISCUSSION
The discussion on research findings begins with an explanation of the trial results on the English listening test and the perception questionnaire. It is followed by an analysis of the pretest questionnaire, the listening tests with answers written in English and Indonesian, and the perception questionnaire and interviews.

Trial Test Results
The trial listening test was held in two classes that are not part of the research sample to ensure the validity and reliability of the study. The results can be seen as follow: Based on Table 1, both listening tests with answers written in English and Indonesian in two trial classes are valid. The rvalue in the two tests are 0.584 and 0.778. There is a difference in the coefficient value between the classes, but both numbers show rvalue higher than rtable of 0.468, which means the tests are valid. In terms of reliability, both tests show that they are reliable with Cronbach's alpha of 0.721 and 0.873. Therefore, the tests used in this trial can be replicated and used as research instruments.
To support the validity and reliability of the listening tests, the researcher also uses inter-rater reliability. The Kappa values of inter-rater reliability for these tests demonstrate a great consistency. In the test with answers written in English, the Kappa values are 0.660 and 0.682, while the test with answers written in Indonesia has Kappa values of 0.762 and 0.669. All values show a great consistency and substantial agreement, which means every student in the trial classes received consistent scores in the listening tests with answers written in English and Indonesian. It can be concluded that both tests are similar in their level of difficulties and that they can be used as research instruments.

Trial Results of Perception Questionnaire
Pearson Product Moment is used in this study to measure the validity of the questionnaire by looking at the value of each question and comparing it to the total value of the questionnaire. If rvalue is more than rtable, then the question is valid; but if rvalue is less than rtable, then the question is not valid.  Table 2 shows that based on Pearson Product Moment, rtable is 0.320 with a df value of 38 and an alpha value of 0.05. In question 2, 3, 5, 6, 10, 11, and 12 rvalue is more than rtable, while in question 1, 4, 7, 8, and 9 rvalue is less than rtable. This means those five questions are not valid, and they need to be revised before they are used as part of the research instruments.

Pretest Questionnaire Analysis
The questionnaire provides descriptive information about students' prior learning experiences in listening comprehension. Because the study uses an ex post facto research design, it is important for the researcher to understand students' learning history. This design focuses on conditions that are already present and then collects data to investigate a possible relationship between these conditions and subsequent characteristics or behaviors. In this design, the independent variables or the presumed "causes" already occurred before the study began. Therefore, the researcher needs to explain these independent variables, which are students' prior learning experiences from when they first learned listening comprehension skills from middle school to the present.
In the pretest questionnaire, there are two categories: students' English learning experiences and their listening learning experiences.  Table 3 provides a brief description of the high school students' English learning experiences. 120 students state that they have been learning English since they were in middle school. They also learnt the same skills and English components in middle school. Reading (41.7%) is the most frequent skill learnt by these students, while speaking and writing are in second place with 25%. Only 8.3% of students state that they learned listening frequently, even though it is an important skill to master in the target language. In terms of English components, students claim that vocabulary (50%) is the most frequently taught in middle school, followed closely by grammar (41.7%).
The following answers reflect students' English learning experiences in high school. Reading (33%) remains the most frequently learned skill by the students, while listening continues to be at the bottom of the list of skills learned with 12.5%. However, in high school, vocabulary no longer becomes the component most frequently taught. Instead, grammar (67%) is the focus. Based on the data, students claim that reading is the easiest skill to learn, while listening is the most difficult.
Questions about students' preparation in taking listening tests are also asked. Based on the questionnaire, listening tests are often given in middle school. Around 10.9% of students said that the tests are given regularly, while 39.1% said they are given quite frequently. The total percentage of listening tests given in middle school is 50%. In high school, listening tests are conducted more frequently and the total percentage is 91.7%, based on the number of students who take the tests very often, often, and quite often.  Table 4 represents students' views of their learning experiences in listening. Some 52.5% of students reported that listening is quite easy, though quite a significant number (47.5%) claimed that it is difficult.
In middle school, listening is not often taught. About 49.2% of students claimed that it is a skill seldom taught, and 17.5% stated that it has never been taught. In high school, the skill is taught more often than in middle school, and around 88.3% students agreed with this statement. Some 52.5% said it is taught quite often, while 35.8% say it is taught often.
The pretest questionnaire provides an idea of students' experience learning English in general, and more specifically in listening. Based on the data around 33% of students are taught listening skills in middle school quite often, and 50% of students said they take listening test frequently. In high school, 88.3% of students are taught listening skill, and the frequency of listening test given is as high as 91.7%. Therefore, it can be concluded that the independent variables in this study are the students' English learning experiences, since they have taken listening tests in the past prior to this study.

Listening Test Analysis
This section answers the research question by comparing the results of two listening tests: one with the answers written in English and the other with the answers written in Indonesian. By comparing the results, the researcher endeavors to better understand the extent students' listening comprehension skills.
The listening test scores are compared and analyzed using descriptive and inferential statistics. Descriptive statistics is used to compare means and to determine the mean difference between the two sets of tests in four different classes. A paired sample t-test is used. Both tests have been measured for their validity and reliability using Pearson Product Moment and Cronbach's alpha. The study also implements inter-rater reliability with Cohen Kappa statistics to ensure the listening tests validity. Overall, the tests are valid and reliable, and the Kappa values of interrater reliability for these tests demonstrates great consistency. Moreover, normality in each class is assessed prior to inferential statistical analysis.
Based on the descriptive and inferential statistical analysis of the listening tests in Grade 11 of Science Class 1, 2, 3 and 4, the results of test with answers written in Indonesian are higher than the results of the test with answers written in English. The listening test scores means in Table 5 show that the test with answers written in Indonesian has a higher mean score than the test with answers written in English. The difference exists across four different classes. In Science Class 1, the mean of the listening test scores with answers written in English is 47.3, while the mean of the same test with answers written in Indonesian is 57.8. In Science Class 2, the mean for the listening test with answers written in English is 53.93, while the test with answers written in Indonesian is 57.46. The same can be found in Science Class 3 (50.6 versus 57.6) and in Science Class 4 (49.5 versus 55.8). The means of the two sets of tests are also statistically different when they are analyzed using the paired t-test.
The paired t-test is used to clearly illustrate the means differences between the two test results, as shown in Table 6 below. The difference can be seen in the significance values across four classes that are less than 0.000, compared to the alpha value of 0.05. Moreover, the difference between the test with answers written in English and the test with answers written in Indonesian can also be seen in the t-value across the four classes: −8,355, −4,938, −10,151, and −8,889. The t-value is negative because it is smaller than the value of the t-table for df 29, which is 1.699. This result of the paired t-test across four classes shows a significant difference among the means of tests with answers written in English and in Indonesian. This shows that the students' listening comprehension answering the test in Indonesian is higher than the students who answered in English. This finding supports Filipi's (2012) research findings.

Perception Questionnaire and Interview Analysis
In this section, data from the perception questionnaire on learners' views of the listening test are analyzed along with the qualitative descriptions and interviews. The students share their views on the use of Indonesian language in the English listening test. For high school students in Indonesia, being able to answer an English listening test in their own language is considered novel. Thus, the researcher is interested in understanding students' perceptions of the new test model for listening and the effect it has on their listening comprehension. A descriptive survey with open-and close-ended questions was used to answer the second research question on students' perception of the new model.
During the study, a perception questionnaire was distributed to 120 eleventhgrade high school students in four science classes: Science class 1, 2, 3, and 4. Each class has 30 students, and each student filled in the questionnaire and returned it to the researcher, which makes the response rate 100%. The questionnaire was designed using the Likert-type scale and was analyzed to provide a statistical result.
The questionnaire consists of three parts: the students' perception to the listening test answered in English and Indonesian, students' perception to challenges in answering the test in English and Indonesian, and students' perception of their listening comprehension skills. Based on the students' responses to the questionnaire, it can be concluded that the students have a positive view on the new model of test that uses their own language to answer the English listening test. In terms of challenges in answering a listening test, the students reported that it is easier to answer test in Indonesian than in English. When they are asked about their comprehension level in taking a listening test with answers written in English and a test with answers written in Indonesian, the students claimed that they have better comprehension when they answered the test in Indonesian.
The students' view of the use of Indonesian language in an English listening test can be seen clearly in Figure 1. As many as 66 students (or 54.58%) reports that they could answer the test in Indonesian, while 54 students (or 45.42%) reports they could answer the test in English. The difference in students' perception is not significant, it is only 9%. This shows that the students do not find it strange to answer an English listening test in Indonesian. This finding is in line with Barlett's (2017) study in which the students support the use of both L1 and the target language. Furthermore, based on the results of the interviews with the students, the study finds that they think using Indonesian helps them to better express their listening comprehension in a test. The two statements above support the notion that the students find the use of Indonesian language to answer the listening test helpful. The use of Indonesian helps to overcome their limited productive skills in English.
In terms of the students' perception to challenges in answering the listening test, 42%, or 51 students stated that they find it difficult to answer test questions in Indonesian. While 58%, or 69 students claimed that answering test questions in English is more challenging. These numbers are supported by the findings from interviews with the students.

N2
: Based on the statistical data in Table 3.3 and interviews with students 1, 4 and 5, it can be concluded that the students find it easier to answer the listening test using Indonesian. They reported that quite often they do not know how to express their thoughts and answer questions in English, even though they understand the questions and know the answers. A number of students emphasized this by saying "I often do not know how to answer them in English even though I understand the questions and know the answer." They will also say "I feel like it's easier to answer in Indonesian than in English," or "… but answering in Indonesian is easier than in English." This demonstrates the challenges they face when answering test questions in English compared to answering test questions in Indonesian. Finally, the students also shared their views on their listening comprehension skills. The students found that they can comprehend the listening test easier when they can answer it in Indonesian and not in English. This statement is supported by 67 students (or 55.7% of the respondents), who claim that they have a more positive view of their listening skills when answering the test in Indonesian. Only 53 students (or 44.3%) claims that they have a positive view of their skill when the test is answered in English.

CONCLUSIONS AND SUGGESTIONS
Based on the findings of the study on Indonesian language use in English listening test, a few conclusions can be made. First, high school students are able to answer the listening comprehension questions better when they answer the listening test in Indonesian instead of English. This means that their listening comprehension is actually higher as it is reflected on the results of the tests in Indonesian and English. Implicitly, this also means that they need to improve their productive skills in order to respond in English. This conclusion is supported by the students' test scores. The students' test scores when they answered the test questions in Indonesian are higher compared to their test scores when they answered the test questions in English. The differences in their test scores are shown in the means of each class scores. The researchers have conducted a thorough calculation using paired t-test to determine the significance of the mean difference in the test scores.
Second, based on perception analysis on the use of Indonesian language in English listening tests, the students claimed that they did not find it strange to answer the questions in Indonesian, even though they had never done it before. Despite this fact, in the context of second or foreign language teaching and learning, the use of L1 is highly avoided.
Third, the findings of this study are useful for the students, English teachers, and language teaching context. For the students, the findings give a clear description of their receptive and productive skills in the context of listening test. For the teachers, the findings inform them about their students' listening skills; and this will help them identify the listening skills that still need to be improved. For the second or foreign language teaching and learning context, the findings confirmed the findings of the previous studies (Amirian, Zareian & Nour, 2016;Cook, 2002;Sarma, 2006) that the use of first language can be implemented in a useful way.
For further research, some recommendations should be considered. Firstly, Indonesian should not be fully utilized in a listening assessment. However, the use of first language can be justified if teachers want to have a better understanding of their students' listening comprehension skills. Secondly, related to the type of questions used in a listening comprehension test, it is advisable not to use openended questions as this type of question assesses not only learners' receptive skills but also productive skills by asking students to write their answers in English. If a researcher only wants to measure students' listening comprehension skills, it is better for him/her to use selected response questions. Lastly, other researchers can also compare the use of both types of questions on testing students' listening comprehension in further research.