Higher-order thinking word problem-solving errors made by low-performing pupils: Comparative case study across school types in Malaysia

The achievement gaps existed among the three types of Malaysian elementary schools with different instruction mediums. Thus, this study sought to compare the types of errors made by the low-performing students from different school types in solving word problems involving higher-order thinking skills by conducting a multiple case study. The study involved 18 low-performing students from a National Primary School (NPS), National-Type Chinese Primary School (NTCPS), and National-Type Tamil Primary School (NTTPS) in Malaysia. The students’ errors in solving eight word problems were identified qualitatively through conducting four sessions of diagnostic interviews, followed by the Newman’s Error Analysis. Then, a cross-case analysis was conducted. The findings indicated that the participants of the three school types made reading, comprehension, transformation, process skills and encoding errors. However, the NPS and NTTPS low-performing students made more process skills errors and encoding errors, as compared to the NTCPS low-performing students. The findings imply the need to enhance the procedural fluency of NPS and NTTPS low-performing students for reducing the achievement gaps among the school types.

Introduction
Geometric measurements such as perimeter, area, and volume are the fundamental knowledge required in most fields such as science, technology, and engineering (Smith et al., 2011). These important mathematical topics are usually included in either the geometry strand or the measurement strand in the elementary mathematics curriculum of various countries (Huang & Witz, 2011; National Council of Teachers of Mathematics [NCTM], 2000). After learning the mathematical concepts, students are engaged in solving routine word problems involving geometric measurements so that they can relate the concepts learnt with the real-world applications (Verschaffel et al., 2010).
While real-world problems are often presented in an unfamiliar context and have no definite solution, solving this type of problem involves manipulating the learnt knowledge and synthesizing a new strategy, instead of applying the known strategy. In other words, higher-order thinking skills (HOTS) are highly in demand for solving such problems. To improve students' problem-solving competency, the students are also engaged in solving the non-routine word problems that involve deriving the numerical information which is not explicitly given based on the properties of the geometric shape (Jones, 2002) and the relationship between the formulae (Yeo, 2008). These non-routine word problems are regarded as word problems involving HOTS in this study.
While there is a greater emphasis on problem-solving and HOTS, the result of the Trend in International Mathematics and Science Study (TIMSS) 2019 indicated that the performance gaps between high-performing students and low-performing students are large (Mullis et al., 2020). The low-performing students often fall far behind their counterparts in solving geometric measurement word problems involving HOTS (Satsangi & Bouck, 2015). To support the low-performing students in problem-solving skill acquisition, there is a need to identify the difficulties faced by the students in solving the word problems involving HOTS. In this regard, Newman's Error Analysis could be a promising method to deepen educators' understanding of student difficulties because it can provide a highly specific diagnosis of types of problem-solving errors made (Watson, 1980). As a multi-ethnic country, Malaysia practises a vernacular education system to accommodate the needs of the three major ethnic groups, namely Malays (69.6%), Chinese (22.6%) and Indians (6.8%) (Department of Statistics Malaysia, 2021). The elementary schools in Malaysia are streamed into three school types based on the medium of instruction: Malay-medium National Primary School (NPS); Mandarinmedium National-Type Chinese Primary School (NTCPS), and Tamil-medium National-Type Tamil Primary School (NTTPS). Thus, the three main ethnic groups have equal opportunities to receive education in their mother tongue (L1).
Despite the same curriculum being adopted in the three types of schools, researchers (i.e., Chew et al., 2021;Chin & Chew, 2022b;Sia & Lim, 2020) constantly found that students from NTCPS performed better than students from NPS and NTTPS in solving word problems. This indicates the existence of mathematics achievement gaps in Malaysia due to school type difference besides the learners' ability. To narrow down the gaps, this study sought to compare the errors made by the low-performing students from NPS, NTCPS, and NTTPS in solving word problems involving HOTS. The findings of the study would provide insight into the hindrance which obstructs students to solve word problems from a cultural perspective. This would eventually deepen teachers' understanding of the problem-solving difficulties faced by their students with different cultural backgrounds while the education system in many countries is increasingly diversified due to globalisation (Zhang, 2019).

Newman's Error Analysis (NEA)
Newman's Error Analysis (NEA) is a simple diagnostic procedure introduced by Newman (1977Newman ( , 1983 for identifying errors in solving mathematical word problems. In order to solve the word problems correctly, Newman (1977Newman ( , 1983 claimed that the student must pass through five successive hurdles: (i) Reading (ii) Comprehension, (iii) Transformation, (iv) Process Skills, and (v) Encoding. In the first stage, the students would have to read the word problems. This was followed by comprehending the numerical data, the quantitative relationship of the objects, and the numerical tasks in the second stage. Then, the students would have to transform the numerical data into mathematical sentences by applying the appropriate arithmetic operation to solve the numerical tasks. After that, the students would have to perform the calculation based on the mathematical sentences formulated. Lastly, the students would have to express the final answer in the unit as requested in the word problem. For example, rather than expressing the final answer for the area of the square in cm, the students should express the final answer in the unit of cm 2 .
Anchoring on the claim made by Newman (1977Newman ( , 1983, an error would arise if the students failed to pass through any of the five successive stages in solving word problems. To identify the error made by the students in solving word problems, Newman (1977Newman ( , 1983 proposed a diagnostic procedure that involves conducting an individual interview with the students while they are solving the word problems. Since each Newman's interview prompt corresponds with a problem-solving hurdle (White, 2010), the type of error made by the student could be specified systematically (Watson, 1980). Numerous past studies had applied NEA in their studies to get insights into students' errors while solving various problems. The study conducted by Abdullah et al. (2015) showed that Malaysian secondary students made all the five errors with almost equal proportions when solving mathematics problems involving HOTS. Rather than conducting studies in the secondary school context, the researchers such as Chin and Chew (2022a, b), Singh et al. (2010), as well as Sibanda (2017) also determined elementary students' errors in solving word problems in Malaysia and South Africa. The findings indicate that majority of the students failed to solve the word problems due to comprehension errors (Sibanda, 2017;Singh et al., 2010). On the other hand, Chin and Chew (2022a, b) found that the transformation error contributed the most to the student's failure in solving word problems. Despite the numerous studies on NEA, the past studies only diagnosed the errors made by students with all ranges of abilities in different contexts.

Higher-Order Thinking Skills (HOTS)
Higher-order thinking skills (HOTS) refer to creative, critical, and analytical thinking skills used to solve non-routine problems through manipulating existing knowledge and known algorithms (Puspitasari et al., 2018;Yeung, 2012). In other words, solving the non-routine word problems would require the top three levels of the cognitive domain in Bloom's Taxonomy, namely: analysing, evaluating, and creating (Anderson et al., 2001). The students need to extract scattered information in the non-routine word problems and link them accordingly (analyzing), make suitable inferences based on prior conceptual and procedural knowledge (evaluating), and eventually manipulate the knowledge and devise a procedure to solve the word problem (creating). Thus, non-routine word problems are considered word problems involving HOTS in this study.
Past studies related to HOTS were mostly related to largescale assessments such as TIMSS and Programme for International Student Assessment (PISA). Even though Mullis et al. (2020) found that majority of Grade Four and Eight students could not solve complex problems in TIMSS 2019, Puspitasari et al. (2018) indicated that non-routine word problems could be solved by the Indonesian Grade Eight high-achieving students without any difficulty. In Malaysia, most studies on HOTS were conducted in the secondary school context. Based on the study conducted by Abdullah et al. (2017), majority of Grade 10 students could not solve the non-routine word problems. Likewise, Suseelan et al. (2022) also found that most Malaysian students in elementary schools performed poorly in solving word problems involving higher-order thinking skills. Although various studies related to HOTS have been conducted in the past, the findings were inconsistent.

School-Type Differences in Mathematics Teaching and Learning in Malaysia
The diversity in the Malaysian education system sparked the interest of researchers to study the school-type differences in mathematics teaching and learning. Despite the same mathematics curriculum used (Lim, 2003) and the same teachers' qualifications among the school types (Hamid et al., 2012), the existence of an achievement gap was reported in several studies (i.e, Ghazali & Sinnakaudan, 2014;Lim, 2003;Sia & Lim, 2020). This is because mathematics teaching and learning could be affected by the instruction medium used (Vukovic & Lesaux, 2013). While the Mandarin language has a simpler and consistent number system, the use of the language decreased students' memory load and hence contributed to the effective retrieval of procedural facts learned . Thus, the use of Mandarin language as an instruction medium benefits L1 students in mathematical concept acquisition (Lim, 2003).
Rather than focussing on achievement differences and the impact of instruction medium, several researchers (e.g., Chia & Lim, 2020;Ghazali & Sinnakaudan, 2014;Roscoe & Sriraman, 2011) compared the teachers' beliefs and instructional practice across school types. The NTCPS teachers held stronger beliefs in constructivist teaching in mathematics learning compared to the NPS and NTTPS teachers (Ghazali & Sinnakaudan, 2014). Thus, the mathematics lessons in NTCPS focused on developing students' conceptual understanding followed by procedural fluency. The NTCPS teachers allocated more time to explain the mathematical concepts (Chia & Lim, 2020;Ghazali & Sinnakaudan, 2014;Lim, 2003). After that, the students were given drills and practices to reinforce the concepts they had learned (Ghazali & Sinnakaudan, 2014;Lim, 2003). Besides, the NTCPS teachers actively engaged the students in mathematics by developing problem-solving activities to enhance their problem-solving competency (Ghazali & Sinnakaudan, 2014).
On the contrary, the NPS and NTTPS teachers believed that a teacher-centred learning environment with adequate reinforcement practices was sufficient to support students' learning (Roscoe & Sriraman, 2011). With this perception, more time was allocated for desk instruction in the NPS mathematics classroom (Chia & Lim, 2020). Instead of explaining the mathematical concepts explicitly, the NPS teachers spent more time walking around the class to check their students' work while they were working out the exercise individually in the classroom. The NPS and NTTPS teachers also had low confidence in students' capability in solving word problems (Ghazali & Sinnakaudan, 2014). Since they perceived that the students could not solve mathematics problems independently, they taught them to solve the word problems by applying the learnt algorithm (Ghazali & Sinnakaudan, 2014;Lim, 2003), rather than engaging them to solve the problems in the lessons.

Purpose of the Study
Various studies on assessing students' competency in solving problems involving HOTS and diagnosing students' errors made in solving word problems have been conducted in the past. However, the reviewed studies involved students with different ranges of abilities, and studies that solely focused on low-performing students are considered scarce. Whist Malaysia has a diverse school system, the previous studies mainly focused on comparing the teachers' beliefs, teaching approaches and the impact of language used on students' mathematics learning. To fill the research gaps, this study sought to compare the errors made by the Malaysian lowperforming pupils from different types of schools in solving word problems involving measurement formulae and higherorder thinking skills based on NEA. The research questions addressed in this study are: (1) What are the types of errors made by low-performing pupils in solving word problems involving HOTS and measurement formulae based on NEA? (2) What are the differences in the types of errors made by low-performing NPS, NTCPS and NTTPS pupils in solving word problems involving HOTS and measurement formulae?

Research Design
The study was conducted by employing a multiple case study research design. As advocated by Creswell and Poth (2016), multiple case study involves performing an indepth analysis of the data collected from multiple sources to explore the phenomenon in the multiple bounded systems explicitly. In other words, multiple case studies are appropriate to be used when the cases involve different contexts (Ridder, 2017) because the robustness of the study could be increased through cross-case comparison (Yin, 2018;Miles & Huberman, 1994). In this study, school type served as the boundaries of each case. With school type as the unit of analysis, school-type comparisons of errors made by the low-performing students in solving higher-order thinking word problems could be made. While each case bounded by the school type could be used for confirming or disconfirming the conclusion drawn from each other, the use of multiple case studies would enhance the external validation of the findings.

Population and Sampling
The population of the study was Grade Four students from NPS, NTCPS, and NTTPS because measurement formulae are introduced to the students in Grade Four. Due to the practical constraint, the sampling frame was limited to Penang state, Malaysia. Since the study aimed to compare the errors made by the low-performing students in solving word problems, the sample of the study was selected using stratified purposive sampling (Radhakrishnan, 2014). The sample selection process is shown in Fig. 1. The process began by stratifying the population into three strata based on school type, namely NPS, NTCPS, and NTTPS. Then, one school was selected to represent each stratum. This was followed by selecting the low-performing students as the sample of study through the administration of a problem-solving screening test involving HOTS to all Grade Four students in the three selected schools (NPS: 32; NTCPS: 31; NTTPS: 44). In this study, the lowperforming students were operationalized as the students who scored below 40% based on the guidelines given by the Malaysian Examination Syndicate (2016).
There were 21 NPS students, 19 NTCPS students and 24 NTTPS students who were categorized as low-performing students. Due to the time constraint, only six students who scored the lowest in the problem-solving screening test from each school type were selected as the participants of the study. This was supported by the previous studies (i.e, Baxter et al., 2005;McAuley & McLaughlin, 1992;Methe et al., 2012) that involved a small sample size (n ≤ 6). Since the students received mathematics instruction in their mother tongue and scored at most five marks on the problem-solving test, the participants from each school type were equivalent to each other.

Problem-Solving Screening Test
In this study, the problem-solving screening test served two purposes: (i) identifying the low-performing students, and (ii) eliciting participants' responses to determine their errors. The problem-solving screening test consisted of eight open-ended non-routine word problems involving geometric measurements and HOTS. They were adopted from Electronic Supplementary Material 1 1 published by Menaga et al. (2022) and covered all aspects of geometric measurement included in the Malaysian Grade Four Mathematics Curriculum: (i) perimeter of squares, rectangles, triangles, and regular polygons; (ii) area of squares, rectangles, and triangles, and (iii) volume of cubes and cuboids.
All eight problems required the students to analyse the given pieces of information in the word problems (analyse), identify the appropriate numerical information in the problems (evaluate) and manipulate the necessary measurement formula (create). Since they involved the top three levels of the cognitive domain in the Revised Bloom's Taxonomy (Anderson et al., 2001), they were categorized as HOTS problems. The content and cognitive domains involved and the mark allocation for each item in the test are summarized in Table 1. The scores allocated for each problem ranged from four to six depending on the complexity of the solution. The more complex the solution was, the higher the marks allocated. The scores were given based on the methods used, the calculations performed, and the final answer written. The marking scheme for Item Q1 is shown in Electronic Supplementary Material 2 2 prepared by Menaga et al. (2022).
Initially, the problem-solving screening test was formed in English. Then, it was translated to Malay, Mandarin, and Tamil languages to ensure its usability for the NPS, NTCPS, and NTTPS students respectively. Upon the construction of the problem-solving test, it was sent to two subject matter experts from NPS, NTCPS, and NTTPS, each with at least 10 years of teaching experience for the evaluation of content coverage and item relevance. Together with the problemsolving test, the test specification, the content coverage judgemental form and the item relevance judgemental form were emailed to the experts. The experts were requested to rate the content coverage (the extent to which each item was covered in the syllabus) and the item relevance (the extent to which the items were relevant to the constructs and cognitive domains measured) on the judgemental form with a five-point Likert Scale.
To evaluate the consensus of the experts, the Scale-level Content Validity Index (S-CVI) was calculated for both content coverage and item relevance of the problem-solving test based on the ratings given by the experts (Polit & Beck, 2006). The S-CVI for the content coverage of the problemsolving screening test was 1.00 for all the three types of schools. This indicated that all the items in the problemsolving screening test were covered in the Grade Four syllabus (Polit & Beck, 2006). The item relevance S-CVI for the Malay, Mandarin and Tamil versions of the problem-solving test were .94, 1.00, and 1.00 respectively. All the S-CVI values surpassed .80, indicating that the problem-solving test items were relevant to the construct and cognitive domain measured (Polit & Beck, 2006). After the instrument validation, the Malay, Mandarin, and Tamil versions were piloted at the NPS, NTCPS and NTTPS schools respectively to evaluate their reliability. The estimate of reliability was calculated by obtaining Cronbach's alpha value using SPSS Version 24. With the Cronbach's alpha coefficient (Malay version: .83; Mandarin version: .81; Tamil version: .78) surpassing the reliability common rule of thumb of .70, the three versions of the problem-solving test were reliable (Pallant, 2016).

NEA Interview Protocol
The NEA interview protocol was used to guide the semistructured interview for collecting the qualitative data on students' errors made. The NEA interview protocol consisted of five interview prompts adopted from White (2010). Each interview prompt was used to identify a specific error made in solving the word problems: (1) Reading Error: Please read the question to me. If you don't know a word, leave it out.

Research Procedure
Before the data collection, permission to conduct the study was obtained from the Malaysian Educational Planning and Research Division (EPRD) and the Penang State Education Department. With the permission of the headmaster of each participating school and parents' consent, the research participants identified through the problem-solving screening test were invited to attend four one-to-one NEA interview sessions which were conducted using the students' mother tongue. The four interview sessions with the duration of 30 minutes each were conducted on four consecutive days during school hours. Each interview session began by engaging the student to solve two word problems in the problem-solving test [i.e., Session 1: Q1 and Q2; Session 2: Q3 and Q4; Session 3: Q5 and Q6; Session 4: Q7 and Q8], followed by identifying the student's errors made in each word problem guided by the NEA interview protocol. All sessions were video-recorded with the student's consent and transcribed verbatim into text for data analysis purposes by three researchers who are fluent in Malay, Mandarin and Tamil respectively. After transcription, the transcripts were rechecked with the original audio to ensure the accuracy of the data (Braun & Clarke, 2006).

Data Analysis
The data analysis was conducted by three researchers who are fluent in Malay, Mandarin and Tamil respectively. To address Research Question One, Newman's Error Analysis was conducted. The analysis process began by coding the transcripts based on the coding framework proposed by Watson (1980) as shown in Table 2. The reading, comprehension, transformation, process skills and encoding errors were identified and coded as 'R', 'C', 'T', 'P' and 'E' respectively. Then, the frequency of each type of error made was tabulated. Besides, the mean frequency was calculated for each error type to indicate the average number of errors made by each student. To address Research Question Two, a cross-case analysis was conducted. For each case (school type), the mean frequency of each type of error made was calculated. Then, the cross-case comparison was conducted to determine the differences in terms of errors made.

Types of Errors Made by Low-Performing Students
Based on the coded transcripts of the NEA interview, it was found that the students from all three types of schools made all the five errors, namely reading, comprehension, transformation, process skills and encoding errors. Since each error could be made by the participants once for each item, the total possible errors made by the students from the three types of schools was 144 (3 types of school × 6 students per school type × 8 items × 1 time for each error type = 144 times for each type of error). The percentages of all types of errors based on NEA made by the low-performing Grade Four students in solving word problems involving measurement formulae and HOTS identified in this study are shown in Table 3. In general, transformation errors are the most common errors made by the low-performing students from all three types of schools. All students from NPS, NTCPS, and NTTPS made transformation errors (100%) in solving the eight word problems involving geometric measurement and HOTS. With the guidance given by the researcher, all low-performing students have rectified the reading and comprehension errors made. Yet, they failed to transform the eight word problems into mathematical sentences using correct sentences.
The comprehension error ranked second in terms of its occurrence. A total of 100 comprehension errors have been made by the 18 low-performing students from the three types of schools. Even though their reading errors have been corrected, they failed to understand the terminology used and the diagram shown in the word problems. Thus, they could not explain the problem situation using their own words even with the researcher's prompts. Besides, they also failed to state the requirements of the word problems. With a mean frequency of 5.56, each low-performing student made a comprehension error in about six word problems on average.
The total number of processing skills errors made by the low-performing students was slightly lower than comprehension errors. The 18 low-performing students made 89 process skill errors in solving the eight word problems. In other words, each student made a process skills error in nearly five word problems. Even though they had formulated the correct mathematical sentences for solving the word problems, they made errors in performing arithmetic operations. Consequently, they obtained the wrong answer.
In general, the students made the least errors in reading the word problems. There were only 37 reading errors that had been made by the 18 low-performing students from the three types of schools. In other words, most of the students could recognise and pronounce the words and symbols shown in the word problems correctly. Out of the eight word problems, each student only made a reading error for two word problems on average.

Comparison of Types of Errors Made across School Types
To compare the errors made by the low-performing students from NPS, NCTPS, and NTTPS, the mean frequency of each type of error made was calculated. Then, the combo chart was plotted to illustrate the comparison of the mean frequency of errors made. As shown in Fig. 2, the mean frequency was not equal across school types for most of the types of problem-solving errors, except transformation errors.
The mean frequency of reading and comprehension errors made by the low-performing NTCPS students was higher than that of the low-performing students from NPS and NTTPS. However, there was only a slight difference between the mean frequency of comprehension errors made by the low-performing students from NPS and NCTPS. On average, the low-performing students from NPS (M Comprehension, NPS = 6.17) and NTCPS (M Comprehension, NTCPS = 6.50) did not comprehend about six word problems. Meanwhile, the low-performing students from NTTPS (M Comprehension, NTTPS = 4.00) made comprehension errors for four word problems on average.
With a mean frequency of 3.83, the reading errors made by the low-performing NTCPS students were at least threefold of the reading errors made by the low-performing students from NPS (M Reading, NPS = 1.00) and NTTPS (M Reading, NTTPS = 1.33). In the interview sessions, the low-performing NTCPS students were asked to read aloud the word problems presented to them. However, the read-aloud session was paused several times because the low-performing NTCPS students did not know how to pronounce the words. They only could continue reading aloud after the researcher guided them to pronounce the unfamiliar words. Besides, the low-performing CNTPS students also pronounced the words wrongly. For example, SJKC11 pronounced the mandarin character '该' [correct pronunciation: gāi] wrongly as 'kè' [the pronunciation for the mandarin character '刻']. In contrast, the low-performing students from NPS and NTTPS were fluent in reading the word problems. The long pauses and incorrect pronunciation of words were seldom found in their read-aloud sessions.
The mean frequency of process skills and encoding errors made by the low-performing NTCPS students was lower than that of the low-performing students from NPS and NTTPS. A large difference was observed in the mean frequency for the two types of errors among the three school types. The mean frequency of process skills errors made by the low-performing NTCPS (M Process skills, NTCPS = 2.67) was half of the mean frequency of process skills errors made by the low-performing NTTPS students (M Process skills, NTTPS = 5.17). Meanwhile, the mean frequency of process skills errors made by the low-performing NCTPS (M Process skills, NTCPS = 2.67) was less than half of the mean frequency of process skills errors made by the low-performing NPS students (M Process skills, NPS = 7.00). Most of the low-performing NCTPS students only made computation errors in performing division. However, the process skills errors were made by the low-performing students from NPS and NTTPS in other arithmetic operations besides division. For example, the low-performing NPS student, SK32 obtained 66 by adding six to six in Interview Session 3. On the other hand, the low-performing NTTPS student, SJKT32 answered six when the researcher asked, '5 times what is 35?' during Interview Session 1. With a mean frequency of 0.67, the encoding errors made by the low-performing NTCPS students were less than the low-performing students from NPS (M Encoding, NPS = 4.67) and NTTPS (M Encoding, NPS = 3.17). Specifically, the mean frequency of encoding errors made by the low-performing NPS students was nearly seven times the mean frequency of reading errors made by the low-performing students from NTCPS. On the other hand, the mean frequency of encoding errors made by the low-performing NTTPS students was nearly five times the mean frequency of reading errors made by low-performing students from NTCPS. Most of the lowperforming students from NPS and NTTPS failed to express the final answer with the correct unit. For example, the low-performing NPS and NTTPS students (i.e., SK28 and SJKT32) gave the final answer for the area in the unit of cm, rather than cm 2 during Interview Session 2 conducted in the respective school. However, the encoding errors were rarely made by the low-performing students in NTCPS. In the four interview sessions, most of them expressed the final answers with the correct unit as requested in the word problems.

What are the Types of Errors Made by Low-Performing Pupils in Solving Word Problems Involving HOTS and Geometric Measurement Based on NEA?
This study revealed that the low-performing students from NPS, NTCPS, and NTTPS made all five types of errors, namely reading, comprehension, transformation, process skills and encoding in solving word problems involving geometric measurement and HOTS. These five types of errors corresponded with the four-step problem-solving process model introduced by Polya (2004): (i) understand the problem; (ii) devise a plan; (iii) carry out the plan; and (iv) look back. The reading and understanding errors were made by the low-performing students when they were understanding the word problem. The transformation errors were made by the students when they were formulating the mathematical sentences to solve the word problem (devise a plan). The procedural errors were made by the students when they were performing the calculation to solve the word problem (carry out the plan). The encoding errors Mean Frequency of Errors Made were made by the students when they checked whether their solution made sense or not (look back). In other words, the low-performing students from NPS, NTCPS and NTTPS made errors in all the problem-solving steps proposed by Polya (2004). The most frequent errors were the transformation errors, followed by the comprehension errors, process skills errors, encoding errors and finally reading errors. In fact, the lowperforming students from all three types of schools made transformation errors in all the word problems. This is in line with the study conducted by Abdullah et al. (2015), Chin and Chew (2022a, b) and Newman (1977). The findings are also in accordance with the claim made by Jiang et al. (2020), whereby many learners committed systematic errors at the stage of transformation due to the inappropriate application of the learnt heuristic. However, the findings contradicted the findings of the study by Raduan (2010) which indicated that comprehension errors were the most common errors made by the students in solving word problems. This might be due to the difference in the nature of word problems as well as the sample used in the study conducted by Raduan (2010). Unlike the study conducted by Raduan (2010) which involved students with different ranges of abilities in solving routine word problems in the real-world context, this study only involved low-performing students in solving non-routine word problems with high cognitive demands. These types of word problems might contain extraneous information. Besides, the mathematical relationship of the numerical information was not presented explicitly in the word problems. While the low-performing students are commonly characterized by poor mathematical representation ability (Montague, 2003), transforming the non-routine word problems into the mathematical sentence using correct operations might be a notoriously difficult task for them (Xin et al., 2005).
Reading error was the error least encountered in this study. This is in line with the study by Abdullah et al. (2015) as well as Chin and Chew (2022a, b). This might be because most of the Year Four pupils were able to read all the words or symbols in the word problems. However, they failed to understand the problems and hence could not proceed with the succeeding stages to solve the problem. Also, the word problems in the test were presented in their mother tongue, thus might have made it less difficult for them to read the problems (Ganuza & Hedman, 2017). In addition, the word problems have been validated by the subject matter experts to ensure the language used was on par with the level of Year Four students. The terminologies which the students were unfamiliar with had been replaced using simpler terms. In other words, the word problems were presented using the terms that were appropriate to the student's reading level. Thus, reading errors recorded the lowest percentage as compared to other types of errors.
In general, the students conducted more errors in transformation, process skills and encoding, as compared to reading and comprehension errors. According to Chan and Kwan (2021), students' word problem-solving proficiency was affected by their content knowledge and reading ability. Pivoting on this claim, the reading and comprehension errors were associated with language factors (Fuchs et al., 2018) while the transformation, process skills and encoding errors were associated with the content-knowledge factor (Lin, 2021;Singh et al., 2010). In other words, the proportion of errors made by the low-performing students related to content knowledge in this study was much higher compared to errors related to language factor. This was in line with the study conducted by Chin and Chew (2022a, b) as well as Clements and Ellerton (1996). The high proportion of errors made by the low-performing students on transformation, process skills, and encoding might be due to their low proficiency in mathematics content knowledge (Lin, 2021;Singh et al., 2010).

What are the Differences in the Type of Errors Made by Low-Performing NPS, NTCPS and NTTPS Pupils in Solving Word Problems Involving HOTS and Measurement Formulae?
The findings indicated that there was no difference in transformation errors made by the low-performing students from NPS, NTCPS, and NTTPS. Despite the mean frequency of comprehension errors made by low-performing students from NTCPS being higher than those from NPS and NTTPS, the difference in the mean frequency was not large. This might be due to the nature of word problems solved by the students. While the mathematical relationship among the numerical information was not explicitly presented in the word problems included in the screening test, it could be difficult to be comprehended by the low-performing students regardless of school type. Rather than visualising the mathematical relationship underlying the word problem, they only focused on the numbers and occasionally, the keywords in the problems (Montague, 2003). With insufficient mathematical vocabulary, they might also fail to recognise the keywords or misinterpret the keywords . Consequently, the low-performing students might simply formulate the mathematical sentence with the numerical information given in the word problems by using any arithmetic operations. Due to limited work memory capacity, the low-performing students might also fail to retrieve the schema which associated the keywords with the appropriate arithmetic operation (Namkung et al., 2019). Thus, the students tended to make transformation errors when solving the non-routine word problems regardless of school type.
The magnitude of differences in the reading errors made by the low-performing students from the three school types was considerably large. The NTCPS low-performing students made more reading errors compared to the low-performing students from NPS and NTCPS. This could be due to the linguistic factor. According to Sung and Wu (2011), the Mandarin language is notably more difficult than the Malay and Tamil languages due to the complexity of its writing system. While Malay words consist of letter strings (Yap et al., 2010) and the Tamil scripts consist of vowels and consonants (Nag & Narayanan, 2019), reading the Malay words and Tamil scripts involves pronouncing the letter morpheme or vowels and consonants presented in the script. Unlike the Malay words and Tamil scripts, Chinese words are composed of characters with several strokes occupied in a two-dimensional box-shaped spatial layout (Yu & Reichle, 2017). To read the Chinese words, the NTCPS lowperforming students had to recognize the character and the corresponding morphemic syllables called pīnyīn, instead of reading the word based on letter morpheme or vowels and consonants presented in the script (Chua & Tan, 2015;Yu & Reichle, 2017). While low-performing students are commonly associated with poor working memory (Xin et al., 2005), they tended to make reading errors when solving the word problems in this study.
The findings also indicated that NTCPS low-performing students performed significantly better in terms of process skills than NPS and NTTPS students. This is in line with the study by Chin et al. (2022), Lim and Chan (1993), as well as Sia and Lim (2020). The strong computational skills of NTCPS low-performing students might result from the teachers' practice in the mathematics classroom. As reported by the study conducted by Chia and Lim (2020), Ghazali and Sinnakaudan (2014), as well as Lim (2003), the mathematics lessons in NTCPS commonly began with the explanation of the concepts. Since this activity had the longest time allocation (Chia & Lim, 2020), the teachers had sufficient time to promote students' conceptual understanding. After explaining the concepts, the students would be given drills and practice to reinforce the concepts learned, and hence procedural fluency was built (Ghazali & Sinnakaudan, 2014;Lim, 2003). Thus, students from the NCTPS had better computational skills compared to those from the NPS and NTTPS.
Like the process skills error, the NTCPS low-performing students made significantly lower encoding errors than those from NPS and NTTPS. The findings are parallel with the findings reported by Sia and Lim (2020). This might be due to the different teaching methods used in each type of school. As reported in the study conducted by Chia and Lim (2020), the mathematical concepts were explained explicitly in the mathematics classroom. For example, various concrete examples were used by the teachers in NTCPS to support students' understanding of measurement units such as centimetre, and millimetre (Chia & Lim, 2020). With a strong conceptual understanding, the students were able to identify the errors made when they looked back at the questions and their solutions. Moreover, the teachers in NTCPS gave plenty of additional word problems to the students at the end of every topic and guided them till the stage of writing the final answer correctly (Lim, 2002). This might have helped the low-performing pupils to learn to identify what should be written in the final answer space.

Conclusion
This study provides an insight into the comparison errors made by the low-performing students from the NPS, NTCPS and NTTPS in solving word problems involving geometric measurement and HOTS. Regardless of school type, the lowperforming students made reading, comprehension, transformation, process skills and encoding errors. However, the tendency of conducting problem-solving errors varies among the school types. The low-performing students from NTCPS made more reading errors, while the low-performing students from NPS and NTTPS made more process skills errors and encoding errors. In short, the findings of this study provide case-based evidence on the differences in the problem-solving errors made by the low-performing students and hence explain their weaknesses in solving higher-order thinking word problems. While the study was conducted following the qualitative research paradigm, the findings of this study still need to be confirmed with a quantitative nationwide study for ensuring the generalisability of the findings.

Practical Implications
Despite all the types of errors that had been made by the low-performing students from NPS, NTCPS, and NTTPS, transformation errors were the most common errors which hindered them from solving the word problems involving geometric measurement and HOTS correctly. Even though they fully understood the numerical information and the problem situations as described in the higher-order thinking word problems, they failed to formulate the mathematical sentence for solving the word problems correctly. Thus, it is suggested that mathematics educators should plan for appropriated intervention that may help the low-performing students overcome the transformation errors in solving the word problems. This may eventually enhance their problemsolving competency.
The findings of this study highlighted that more process skills errors and encoding errors were made by the NPS and NTTPS low-performing students compared to their peers in NTCPS. According to Singh et al. (2010), process skills errors and encoding errors are content-knowledge related errors. The low-performing students made mistakes in performing the computation based on the mathematical sentence formulated and failed to write the final answer using the correct units. This indicates the lack of procedural fluency among the low-performing students from NPS and NTTPS. Thus, procedural fluency should be emphasised in NPS and NTTPS mathematics classrooms for enhancing the low-performing students' computation skills. As such, the performance gap among the three types of schools could be reduced.
In addition, the low-performing students from NTCPS were reported for making more reading errors in solving higher-order thinking word problems, compared to the lowperforming students from NPS and NTTPS. The NTCPS students could not recognise the Chinese characters, and hence failed to pronounce the words correctly. This would eventually obstruct their understanding of the problem situation described in the text. Consequently, they failed to solve the word problems. While reading error is a language-related error (Singh et al., 2010), the findings of this study call upon the collaboration among the Mandarin and Mathematics teachers to plan for appropriate remediation that could support low-performing students in rectifying the reading errors made during higher-order thinking word problem-solving.

Limitations and Recommendations
There are several limitations to this study. Firstly, the findings of this study could not be generalised to the entire population due to the use of multiple case studies that follow a qualitative research paradigm. Moreover, the findings of this study might be subjected to errors due to the small sample size of the study in view of the practical constraints such as sampling accessibility arising from the Covid-19 pandemic. To ensure the generalisability of the study, it is suggested to conduct a nationwide quantitative study involving a larger sample size in the future. Besides, it is suggested to use inferential statistics for making statistical inferences on school-type differences in problem-solving errors made so that more meaningful findings could be obtained.


Development and validation of Foreign Language Teacher Support Scale (FLTSS)

Teacher support, as an essential type of social support and an important antecedent of many key outcomes in L2 learning, can significantly contribute to foreign language achievement. Although teacher support has received considerable attention in education and educational psychology, it has drawn scanty attention in foreign language and applied linguistics research. Therefore, the present study aimed to fill in this research gap by developing and validating a domain-specific scale to measure the types of teacher support perceived by English as a Foreign Language (EFL) learners. An initial item pool was constructed based on some generic measures of teacher support and semi-structured interviews with EFL learners. Then, the items were submitted to a panel of experts and their content validity was checked using the content validity index. After a series of exploratory and confirmatory factor analyses, the final version of the Foreign Language Teacher Support Scale (FLTSS) measuring four types of perceived teacher support, namely, emotional, instrumental, appraisal, and informational, was prepared. The results of examining different types of validity and reliability indicated that the scale is suitable for measuring different types of perceived teacher support. Some suggestions for further research are presented.

Page 2 of 16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 To date, the research literature of EFL and applied linguistics has focused on several influential factors relating to learners, teachers, and their interactions to provide suggestions for better learning (e.g., Dewaele, 2007;Ellis, 2008). Teacher support, as an essential type of social support and an important aspect of classroom environment, has been considered as a key factor which can significantly contribute to foreign language leaners' success to the extent that "successful learning may be very difficult, if not impossible" without sufficient teacher support (Piechurska-Kuciel, 2011, p. 84). EFL teachers can create a positive, supportive, and secure atmosphere by regulating emotional and social processes (Furrer et al., 2014) so that learners can "unload emotionally while experimenting with their English" (Sharp-Ross, 2011, p. 110).
Given the importance of teacher support in general and for learning English in particular (Piechurska-Kuciel, 2011) as well as the gap relating to the lack of due attention to this important construct in the research literature of applied linguistics Sadoughi & Hejazi, 2021, 2022 and the unavailability of a domain-specific scale in L2 research, the present study is an attempt to develop and validate a teacher support scale which can measure the types of support perceived by learners during their foreign language learning process. First, a review of previous studies highlighting the benefits of teacher support is presented. Then, the social support model proposed by Tardy (1985) is introduced as the theoretical basis of the study. Next, details relating to the method (e.g., questionnaire development, procedure, and data analysis) are provided. Subsequently, the validity and reliability of the proposed instrument is examined. Finally, a brief discussion of the findings as well as the limitations, implications, and suggestions for further research are presented.

Literature review
Teachers, along with students and materials, are generally known as a principal element in language teaching and learning situations (Hutchinson & Waters, 1987;Richards, 1998). In fact, they play a highly important role in supporting students' learning progress via numerous interactions with them. Teacher-student relationship is considered as a highly important mechanism which can promote students' engagement and motivation (Reeve, 2012) as well as resilience (Hu, 2022) by providing a supportive and positive learning environment Sadoughi & Hejazi, 2021, 2022. Teachers can build a secure and encouraging environment by establishing a constructive relationship with students and providing them with sufficient support (Furrer & Skinner, 2003).
A growing body of research in education and educational psychology has focused on teacher support and confirmed its substantial benefits for students. For example, research indicates that teacher support can significantly enhance students' academic achievement (Niehaus et al., 2012;Wong et al., 2018), motivation (Ahmed et al., 2010), learning strategy use (Yıldırım, 2012), creative thinking (Zhang et al., 2020), self-regulated learning (Perry et al., 2002), task persistence (Pakarinen et al., 2014), engagement (Klem & Connell, 2004;Mckeown & Taylor, 2022;Tas, 2016;Zhao et al., 2019), academic self-efficacy (Aldridge et al., 2012), concentration on learning activities (Boulton et al., 2012), and career development (Wong et al., 2022). In addition, several studies have shown that higher levels of teacher support could encourage learners to devote Page 3 of 16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 considerable effort to their learning process, increase their self-confidence (Pianta et al., 2012;Ucar & Sungur, 2017), and cultivate positive attitudes to the subject-matter (Rice et al., 2013). Furthermore, teacher support can foster students' positive achievement emotions Sadoughi & Hejazi, 2021;Lei et al., 2018), lower their negative achievement emotions (Lawman & Wilson, 2013), promote their task value and academic self-concept (Tas et al., 2019), assist them to fulfill teachers' expectations and reduce their undesirable and distractive behaviors (Wang & Eccles, 2013), decrease and resolve their uncertainties (Rosenfeld et al., 2000), promote their grit , and enhance their belonging to the learning environment (Roeser et al., 1996;Saroughi & Cheema, 2022). Hence, it is highly important for teachers to foster and maintain a friendly and positive relationship with students and offer them substantial support to promote their achievement. Despite the fact that "the language learner [is considered] as an active self-reflective agent of an interaction with the social context" (Taylor, 2013, p. 34), the literature on the role of the support provided by important others, particularly EFL teachers, is still relatively scanty and fragmented. Some studies have shown that teacher support can substantially enhance students' willingness to communicate (MacIntyre et al., 2001;Piechurska-Kuciel, 2015;Wei & Xu, 2022), which is considered as an essential aspect of foreign language success. In addition, teacher support can improve EFL learners' motivation (Bi, 2015;Piechurska-Kuciel, 2013;Vatankhah & Tanbakooei, 2014), enhance their learning experience and engagement (Sadoughi & Hejazi, 2021, 2022, promote their positive emotions Sadoughi & Hejazi, 2021) and affective learning (Sun & Shi, 2022), and reduce their negative emotions such as anxiety by offering them a sense of security in the class (Abu Rabia, 2004;Huang et al., 2010;Jin et al., 2017;Lawman & Wilson, 2013;Piechurska-Kuciel, 2011). Additionally, the results of a study conducted by Ghaith (2002) indicated that perceived teacher support had a positive correlation with academic achievement and cooperative learning among EFL learners. Furthermore, sufficient support from EFL teachers can improve students' resilience, protect them from stressors, and help them overcome challenges and stressful tasks in learning a foreign language (Rahimi & Bigdeli, 2014). Last but not the least, considering the demanding and challenging nature of the lengthy process of learning a foreign language, teachers play a crucial role in maintaining learners' interest and sustaining their effort , which can strongly predict learners' foreign language achievement (Piechurska-Kuciel, 2013).

Theoretical background
Teacher support is deeply rooted in the social support model. According to Tardy (1985), social support could be conceptualized as a multi-faceted construct with five main dimensions including direction (support given or received), description/evaluation (social support described or assessed), disposition (available or utilized), content (instrumental, informational, emotional, or appraisal support), and network of support. Social support could be provided by family members, peers, and teachers for learners (Steese et al., 2006).
This study focuses on EFL learners' perceived types of social support provided by their EFL teachers, which could have several important advantages for them (see Introduction Page 4 of 16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 and Literature review). Generally speaking, teachers can provide different types of support including informational, emotional, instrumental, and appraisal support (Malecki & Demary, 2002). Informational support refers to teachers' provision of information, guidance, or advice on a particular content area. Emotional support is related to empathy, love, belonging, and trust. Instrumental support includes resources such as time, money, or skills. Finally, appraisal support refers to teachers' evaluative feedback and/or instructions for improving students' performance.

This study
Although perceived EFL teacher support has appeared as an important variable in previous studies (e.g., Bi, 2015;Huang et al., 2010;Piechurska-Kuciel, 2011, these studies have mainly used generic measures of teacher support (e.g., Aldridge et al., 1999;Chen, 2005;Griffith, 1995;Johnson & Johnson, 1983;Trickett & Moos, 2002), thus failing to capture the particular types of support which EFL teachers can provide for their learners in learning a foreign language. Considering the nature of teacher support as a malleable (Gehlbach et al., 2012) and multidimensional construct (Anderman et al., 2011), the present study is a response to Piechurska-Kuciel's (2017) call for further systematic research on teacher support in EFL learning. More specifically, this study is an attempt to develop and validate a comprehensive, domain-specific scale, known as Foreign Language Teacher Support Scale (FLTSS), for measuring perceived teacher support among EFL learners. The following research questions (RQs) motivated the present study: RQ1: What is the factor structure of the FLTSS? RQ2: Is the four-factor structure hypothesized for the FLTSS confirmed? RQ3: How valid and reliable is the FLTSS?

Questionnaire development
Following Dörnyei's (2010) steps for questionnaire development, an initial item pool was prepared by borrowing questions and using qualitative, exploratory data. More specifically, we first constructed an item pool by borrowing and adapting items from the existing teacher support scales (e.g., Patrick et al., 2007;Wong et al., 2018). All items were translated to Persian and back-translated to English by translation experts for guaranteeing the translation quality. In the next stage, semi-structured interviews were conducted with EFL learners to ascertain their views about different types of support they received from their teachers. Data saturation was accomplished when 18 students were interviewed. The interview data were analyzed following Dörnyei's (2007) approach. In doing so, the data were first transcribed and meticulously read several times in the pre-coding stage. Then, they were coded and recoded many times to extract "higher-order pattern codes" from "descriptive and low-inference" codes. It should be noted that since the theoretical framework of the study was based on social support model, only themes relating to social model of teacher support were retained while others were excluded. Furthermore, following Lynch (2003), we asked two applied linguistics and two educational psychology experts familiar with analysis of interview data to crosscheck the coding and extraction of the themes. In this stage, 4 items were added to the item pool. All items were based on five-point Likert type scale, where a higher score indicated a higher level of perceived teacher support. Next, clarity, content-relatedness, and format of the questionnaire items were checked and some minor modifications were made. Finally, the constructed questionnaire was submitted to a panel of 8 experts to assess the content validity of the items and total scale. Content validity index (CVI), as the most widely used method for checking content validity (Polit et al., 2007), was calculated for all items. In this stage, 8 items with CVI values lower than .7 were removed. The remaining items (n=25) had CVI values in the range of .75 to 1, which indicates the good content validity of the FLTSS.

Participants and procedure
The participants were Iranian EFL learners in Tehran (Iran) who were selected through convenience sampling from 12 language institutes. A total of 1150 questionnaires were administered among the EFL learners, and 1074 questionnaires were returned, which led to a response rate of 93.3%. It is noteworthy that 22 questionnaires with missing values on more than five items were excluded. Therefore, 1052 questionnaires completed by 530 male students (50.38%; mean age=15.55) and 522 female students (49.61%; mean age=15.78) were retained. In the first section of the questionnaire, the EFL learners first provided their demographic information (age and gender) and responded to the FLTSS items. The participants were given sufficient time to answer the questionnaire items.
To recruit the sample, several language institutes were first contacted and the aims of the study were explained for their managers. The managers granting their permission were further contacted to organize the data collection procedure. The questionnaire was administered in Persian, the EFL learners' native language, to ensure their maximum understanding. The anonymity and confidentiality of the collected data were guaranteed for the participants. In addition, learners' informed consent was obtained and they knew that their participation was completely voluntary. The exclusion criteria were incomplete questionnaires, unwillingness to continue the study, and refusal to give informed consent.

Data analysis
The data analysis was performed in three phases: constructing the item pool and descriptive analyses, exploratory factor analyses and item analysis, and confirmatory factor analyses. The data were examined for missing data, and the results indicated that the percentage of missing data per each variable was .1 to 1.2%. The results of Little's test indicated that the data were missing completely at random (MCAR) (χ 2 (755) = 723.054, P = .793). Thus, multiple imputation using the EM algorithm was performed to replace the isolated missing values in the data set. Then, the univariate normality and multivariate normality were investigated by standardized scores and Mahalanobis Distance, respectively, which led to removal of no cases. Then, EFA was run to extract the underlying Page 6 of 16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 factors and exclude the items with factor loadings less than .4. Finally, CFA was performed to verify the internal, external, convergent, and discriminant validity of the scale.

Results
Exploratory factor analysis (EFA) and item analysis EFA using principal axis factoring (PAF) with Promax rotation was performed to determine the separate, underlying factors in the prepared 28-item questionnaire. Common factor methods such as PAF are preferred over Principal Component Analysis (PCA) to evaluate the underlying factor structure of psychological and educational measures (Fabrigar et al., 1999;Floyd & Widaman, 1995). Given the fact that the decision to choose the appropriate rotation method is mostly dependent on the theory/model of the study (Meyers et al., 2005), Promax rotation method was applied in this study. Kaiser-Meyer-Olkin measure of sampling adequacy (KMO=.938 > .8) and Bartlett's test of sphericity (chi-square = 13682.055, df = 378, p<.001) indicated that the assumptions underlying PAF were fulfilled. PAF with Promax rotation after 6 iterations resulted in four factors with eigenvalues greater than 1, accounting for 50.55% of the total variance. Following Raubenheimer's (2004) recommendation, three items with factor loadings less than 0.4 were removed. Thus, another round of factor analysis was needed (see Qin, 2003). No item was removed in the next EFA round, and the results of KMO = .938 > .8 and Bartlett's test of sphericity (chi-square = 12,776.532, df = 300, p<.001) were significant. After six iterations, four factors with eigenvalues greater than 1 (7.63 for factor 1, 2.54 for factor 2, 1.81 for factor 3, and 1.53 for factor 4) were extracted, which explained 54.12% of the total variance. To sum up, the EFA results indicated that all the four extracted factors were clearly identifiable. The retained 25 items as well as their mean, standard deviation, skewness, and kurtosis are shown Table 1.

Confirmatory factor analysis (CFA)
CFA was performed in Amos to evaluate the factor structure of the FLTSS, which showed good model fit.  (Hu & Bentler, 1999). As for RMSEA, values below .08 are considered as acceptable (Bentler, 2007).

Internal validity
CFA was performed to examine the internal validity of the FLTSS. The construct validity of two models, namely, one-factor and four-factor models, was evaluated competitively. Model A (one-factor model) supposed that the whole scale was based on only one factor while Model B (four-factor model) distinguished between the latent factors such as emotional, instrumental, appraisal, and informational teacher support. Based on Table 2, model A did not have good fit to the data. However, the hypothesized model B had a good fit with the empirical data, and all factor loadings were larger than .66. Confirmatory factor analyses corroborated the four-factor structure by Page 7 of 16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 distinguishing between different types of teacher support (i.e., emotional, instrumental, appraisal, and informational).
In addition, to further examine the internal validity, inter-correlations between the four types of teacher support were calculated. Students perceiving more support in any of the four sub-scales reported higher levels of support in other sub-scales. To conclude, the inter-correlations among the four sub-scales confirmed the internal validity of the FLTSS.

External validity
Gender differences were calculated as a measure of external validity. Multiple analysis of variance (MANOVA) was performed to investigate the mean level differences between female and male students in different sub-scales. Interestingly, the only significant   16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 gender difference was related to emotional perceived teacher support as female EFL learners (M=19.73, SD=3.54) reported more emotional teacher support than male EFL learners (M=18.64, SD=3.75, F (1,1050) =23.66, p<.001). It should be noted that there were no significant differences between the two genders in terms of other types of teacher support.

Convergent validity
The convergent validity of the FLTSS was examined to verify the validity of each subscale. Based on Table 3, the model in each subscale has a good fit with the data, which shows that the data were suitable for convergent validity analysis.

Discriminant validity
Discriminant validity was examined by two common methods including Fornell and Larker's criterion and heterotrait-monotrait (HTMT) ratio using SmartPLS software (version 3.2.9). In the first method, i.e., the Fornel and Larcker's criterion, when the square root of the average variance extracted (AVE) value of each component is higher than the correlation coefficient between the variables, the discriminant coefficient between the variables would be strong (Hair et al., 2010). As can be seen in Table 4, the Page 9 of 16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 AVE square roots for each construct are higher than the correlation between the constructs (off-diagonal), which indicates the discriminant validity of each construct. Furthermore, HTMT based on multitrait-multimethod (MTMM) matrix was employed to check the discriminant validity. HTMT values lower than .85 demonstrate discriminant validity (Kline, 2011). In addition, the bootstrap procedure was run with confidence interval to examine whether it includes one. Confidence intervals including the value one indicate the lack of discriminant validity while those not including the value one show distinctiveness of the constructs. Based on Table 4, since all HTMT ratios are lower than .85 and the confidence intervals (shown in parentheses) do not contain the value one, the FLTSS has discriminant validity.

Reliability
The Cronbach alpha coefficients of different sub-scales ranged from .83 to .91. Furthermore, part-whole corrected item-total correlations for all items were higher than .5 (see Table 3), which is considered very good (Bortz & Döring, 2006). Therefore, all sub-scales of the FLTSS have sufficient reliability.

Discussion
Teachers play an undeniably crucial role in language teaching programs by providing their learners with essential support, which can promote their grit , L2 willingness to communicate (MacIntyre et al., 2001;Wei & Xu, 2022), motivation (Bi, 2015;Piechurska-Kuciel, 2013), engagement (Sadoughi & Hejazi, 2021, 2022, positive achievement emotions Sadoughi & Hejazi, 2021), learning experience , and foreign language achievement (Piechurska-Kuciel, 2013) and reduce their anxiety (Jin et al., 2017). However, no domainspecific scale is available to measure the types and amounts of support they can provide for learners. Therefore, in response to the necessity for further systematic research on teacher support in learning English as a foreign language, the present study aimed to address the research gap relating to the need for a domain-specific scale to measure different types of perceived teacher support. To this end, a psychometrically valid and reliable measure called FLTSS was developed based on some generic measures of teacher support, semi-structured interviews with EFL learners, and experts' judgements and went through rigorous exploratory and confirmatory factor analyses.
The FLTSS developed in this study can measure four types of perceived teacher support, namely, emotional, instrumental, appraisal, and informational. Emotional support refers to learners' perception of their teachers when they are warm, encouraging, Page 10 of 16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 approachable, and caring (Malecki & Demary, 2002). This type of support, which concerns empathy, love, belonging, and trust, is assessed by 5 items in this scale. When learners encounter challenges such as difficulty in doing their class/homework and feel disappointed, EFL teachers can emotionally support them by understanding their feelings, being friendly with them, caring for their progress in learning English, and carefully listening to their concerns about learning English. Recent research indicates that emotional teacher support can play a crucial role in promoting adaptive learning outcomes such as L2 willingness to communicate (Wei & Xu, 2022), L2 grit , engagement (Sadoughi & Hejazi, 2021, 2022, and motivated learning behaviour . The present study also showed that instrumental support, which is related to providing learners with resources such as time or skills, is another separate dimension of teacher support (7 items). This type of support is reflected by practical help (Suldo et al., 2009) and tangible support provided for learners when they are working on a difficult task or exercise (Semmer et al., 2008). Teachers can provide instrumental support for students by offering them sufficient help for learning English, fostering pair and group work in class, giving them opportunities to express their opinions, allocating adequate time for them, and making the classroom atmosphere and learning process fun. Past research have shown that instrumental support is positively related to intrinsic motivation (Federici & Skaalvik, 2014), positive emotions Sadoughi & Hejazi, 2021), and subjective well-being (Suldo et al., 2009) and negatively associated with anxiety (Federici & Skaalvik, 2014).
Another dimension of teacher support in FLTSS is appraisal support (6 items), which is related to teachers' evaluative feedbacks and/or instructions for improving students' performance (Malecki & Demary, 2002). This type of support helps learners monitor their learning process and regulate their efforts by interpreting teachers' feedbacks (Nicol & Macfarlane Dick, 2006). The evaluative nature of teachers' comments and suggestions about students' present and target levels of understanding and performance helps learners improve their learning process and outcomes. The key role of feedback for enhancing foreign language performance is highlighted in interaction hypothesis (Long, 1996). EFL teachers can provide their learners with appraisal support by giving them constructive feedback on their weaknesses, strengths, performance, and progress as well as talking with them about factors relating to their success or failure in learning English.
Finally, informational support, which refers to teachers' provision of information, guidance, or advice on a particular content area, is measured by 7 items of the FLTSS. EFL teachers can provide learners with this type of support through different ways such as introducing extra resources and materials (Zare-ee & Hejazi, 2018), offering good examples and clear explanations, and providing them with useful guidelines for improving their English. More specifically, this type of support is considered as a good scaffolding through which teachers can guide learners during the course of their studies to adopt better strategies for promoting their language learning process.
The results of previous studies on gender differences in terms of perceived teacher support are inconclusive. For example, while some studies (e.g., Bokhorst et al., 2010) showed that female learners could perceive higher levels of teacher support, others (e.g., Thompson & Austin, 2010) found that male learners could receive more Page 11 of 16 Sadoughi and Hejazi Language Testing in Asia (2022) 12:30 encouragement and eye contact from their teachers and thus perceive higher levels of teacher support. In the present study, the MANOVA results for gender differences as a measure of external validity showed that female learners perceived significantly higher levels of emotional teacher support. This could be justified by considering the fact that girls can make stronger emotional investments in their relationships with teachers due to traditional sex-role norms and perceive higher levels of teacher support (Piechurska-Kuciel, 2013). It should be noted that the measurement model suggested in this study had good model fit with the empirical data collected from both female and male EFL learners.

Conclusion
The FLTSS developed and validated in this study could be administered as a valid and reliable instrument to assess EFL learners' perceived emotional, instrumental, appraisal, and informational teacher support, which can help better accommodate their need for different types of support by providing useful guidelines for EFL teachers to develop effective classroom interventions. In this way, teachers can help EFL learners reduce their frustration in learning English and enhance their learning experience, willingness to communicate, positive emotions, grit, and motivation, engagement, and performance. Additionally, given the fact that EFL teachers play an undeniably important role in learning and teaching situations and that teacher support is an important antecedent of many key outcomes in education and educational psychology (see Introduction and Literature review), researchers may use this scale as a research tool to measure EFL learners' perceived teacher support and explore its effects on other variables as well as its possible relationship with them.

Limitations and suggestions for further research
The present study had some limitations. The participants were selected through convenience sampling. The FLTSS was built based on adaptation of items from some generic teacher support measures and results of semi-structured interviews. Future studies can use stimulated recall and think-aloud protocol to further consider the four types of perceived teacher support investigated in this study. The scale developed in the current study was based on social support model (Tardy, 1985). Therefore, it is suggested that future studies focus on teacher support from other perspectives such as self-determination theory (SDT) based on which teachers can provide students with different types of support such as autonomy and emotional support. In addition, given the undeniable role of gender in educational research (Pahlke & Goble, 2015), future studies can take into account not only students' gender but also teachers' gender to check the invariance of the FLTSS developed in this study. Since the FLTSS was developed and validated among language learners in institutes in Iran as an EFL context with its own educational, cultural, socioeconomic, and political features, future studies can explore the psychometric properties of the scale in high schools and university settings in other EFL or English as a Second Language (ESL) contexts. Future studies can measure perceived teacher support across different


The Role of Midwives and Information Media in Knowledge, Attitude, and Behavior of Postpartum Mothers about COVID-19 Health Protocol

Transmission of COVID-19 in pregnant women and postpartum mothers is potentially high risk. Postpartum mothers implement health protocols influenced by predisposing, enabling, and need factors. This study aimed to analyze the role of midwives and information media in postpartum mothers' knowledge, attitude, and behavior regarding the COVID-19 health protocol. This research was conducted in RSUD Kabupaten Kediri, in August–October 2021. The study used a correlational analytic method. The sample was postpartum mothers using a simple random sampling technique where 100 respondents included in inclusion criteria were given a questionnaire to complete. This study employed Kendall's tau correlation test to analyze the data. The result showed that the role of midwives is mainly in the good category (66%) while the media information is also in the good category (58%). Most of the respondents had sufficient knowledge (37%), the attitude of respondents in implementing the COVID-19 health protocol had a positive category (52%), and the behavior of the respondents mostly had a good category (54%). This study concludes that there is a significant correlation between the role of midwives to knowledge (p=0.009), attitudes (p=0.003), and behavior (p=0.000) of postpartum mothers. There is a significant correlation between information media and knowledge (p=0.042) and behavior (p=0.012) of postpartum mothers. However, there is no significant correlation between information media and the attitude (p=0.756) of postpartum mothers regarding the COVID-19 health protocol.

Introduction
Some people infected by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) will have mild to moderate respiratory illness before recovering . The virus can aggravate the disease of the elderly and have health problems such as diabetes, chronic respiratory disease, cardiovascular disease, and cancer. Since March 11, 2020, it has declared COVID-19 a global pandemic with more than 500 million cases in 114 countries, and more than 6 million people have died. Since March 14, 2020, COVID-19 has become a national disaster in Indonesia. 1 As of April 2022, the Ministry of Health reported 6 million confirmed cases of COVID-19 with 150 thousand deaths. 2 Transmission of COVID-19 that occurs in pregnant women and postpartum mothers is considered high risk. It means that pregnant women and postpartum women have physiological and physical changes that can increase their susceptibility to infection. The research in America stated that pregnant women infected by COVID-19 among as many as 55 people and 46 neonates. 3 In Indonesia, there were 18 pregnancies infected with COVID-19, all of them were infected in the third trimester, and the clinical findings were similar to those of non-pregnant adults. This condition needs more attention from health workers. 4 Based on these limited data and several examples of cases in handling coronavirus, knowledge about COVID-19 infection in pregnancy and fetus is still limited. There are no specific recommendations for handling pregnant women with COVID-19. Therefore, it is necessary to prevent and control COVID-19 for health workers to stay healthy, safe, and productive and for patients to get services according to standards. 2,4,5 Thus, patients get information about the principles of preventing COVID-19 in pregnant women, postpartum mothers, and newborns. The success of pandemic control in hospitals because it has an Infection Prevention and Control Committee, and all health care facilities have it for screening COVID-19. 6 Postpartum mothers can carry out health protocols influenced by predisposing factors (sources of information, knowledge, and attitudes), enabling factors (role of health workers and availability of services, and need factors (perception of health and condition of mother's health). 7 Information influences one's knowledge. Even though someone has low education but gets good information through online and offline media, it will increase one's knowledge. 8 Health workers (midwives) have a role as perpetrators who are obliged to carry out their duties according to their positions. 9 Other research found that there was an effect of the role of health workers, sources of information, and husband's support on postpartum mothers' adherence to breast care with p<0.05. 10 Based on a preliminary study conducted in Teratai Room, RSUD Kabupaten Kediri, some postpartum mothers do not wear masks before being reminded by the midwife on duty. Therefore, this study aimed to analyze the role of midwives and information media on postpartum mothers' knowledge, attitudes, and behaviour about COVID-19 health protocols.

Methods
This research is a correlational analytic study with a cross-sectional approach. The study conducted in Teratai Room, RSUD Kabupaten Kediri, in August-October 2021.
The sampling technique used was simple random sampling. The sample is 100 respondents, with the inclusion criteria: postpartum mothers willing to be respondents in Teratai Room, RSUD Kabupaten Kediri. The exclusion criteria: mothers who experienced post-abortion suffered from gynecological and oncological diseases. Independent variables are the role of midwives and information media with an ordinal scale. Dependent variables are knowledge, attitudes, and behavior of postpartum mothers with an ordinal scale, and the instrument used is a questionnaire. The Research Ethics Committee of Universitas Kadiri approved the study with approval number 005/19/VII/KEP/UNIK/2021. Analysis of the data using Kendall's tau correlation test.

Results
The frequency distribution of respondents based on characteristics (midwife's role, information media, knowledge, attitudes, and behavior of postpartum mothers) can be seen in Table 1. The midwife's role is mainly in the good category, reaching 66%. Media information has a good category, that is 58%. Most of the respondents had sufficient knowledge, namely 37%, the attitude of respondents in implementing the COVID-19 health protocol had a positive category reaching 52%, and the behavior of the respondents mostly had a sufficient category which is 54%.
Based on Table 2, it can be seen that there is a significant correlation between the role of the midwife with knowledge (p=0.009), attitude (p=0.003), and behavior (p=0.000). There is a significant correlation between the information media with knowledge (p=0.042) and behavior (p=0.012), while the information media on attitudes has no significant correlation (p=0.756).

Discussion
Based on the research results, 100 postpartum mothers have a reasonably good knowledge (41-60%) of (68%). It showed that postpartum mothers know quite well about the COVID-19 health protocol. It allows the reception of information well, and the memory function works well. By receiving good information, it is hoped that postpartum mothers can maintain this information to be applied, especially information about COVID-19 health protocol.
The analysis test used Kendall's tau analysis, namely the role of the midwives in the knowledge, attitudes, and behavior of postpartum mothers. There is a significant correlation between the role of midwives and the knowledge of postpartum mothers about the COVID-19 health protocol with a positive (good) correlation. It occurred because the respondent correctly received information or socialization about the COVID-19 health protocol, especially on the role of health workers in the success of exclusive breastfeeding. Success can be achieved because respondents have good knowledge obtained from counseling activities carried out by health workers, especially midwives. 11 The results showed that the midwife's role was mainly in the good category, which was 66%. The correlation between the midwife's role and knowledge about COVID-19 health protocols, which were mostly good, could be influenced by the midwife's role regarding COVID-19 health protocols. The results showed the good midwife role regarding the COVID-19 health protocol was affected by the knowledge of postpartum mothers, mainly in the good category with 66%. In comparison, the role of midwives who had sufficient knowledge of the COVID-19 health protocol for postpartum mothers understanding in the adequate category was 18%.
Most of the knowledge that is quite good is obtained through communication, information, and education given by the midwives, which is obtained after giving birth. According to the professional standards of midwives, they are required to be able to provide care for postpartum and breastfeeding mothers. One of the roles of midwives in the puerperium is to provide health education and counseling to clients. 12 The role of the midwives and the attitude of the postpartum mother have a positive correlation. The better the midwife's role, the more positive the respondent's attitude regarding COVID-19 health protocol (52 respondents). Health workers can have a positive influence by demonstrating this attitude to mothers and their families so that respondents are successful in providing exclusive breastfeeding. 11 The results showed that the attitude of postpartum mothers about the COVID-19 health protocol was mainly positive, with 52%. The correlation between the role of midwives and attitudes about the COVID-19 health protocol can be influenced by the role of the midwife. The results showed that the role of a good midwife regarding the COVID-19 health protocol affects the attitude of postpartum mothers, mostly positive 53.03%. In comparison, the role of midwives who have sufficient knowledge about the COVID-19 health protocol caused postpartum mothers to have a negative attitude of 66.66%.
Attitudes are influenced by personal experiences, culture, and other people who are considered important. Midwives play a significant role in providing midwifery care, including during the puerperium and breastfeeding. Information, education, and communication (IEC) and counseling provided by midwives about the COVID-19 health protocol will be able to increase the knowledge of postpartum mothers. A good understanding of preventing COVID-19 will cause a person to have a positive attitude towards it. A person's attitude begins with the stimulus he gets. The stimulus can be in the form of counseling about the COVID-19 health protocol. Midwife's role as an educator so that from that knowledge, she will respond positively or negatively. 13 Based on the results of Kendall's tau statistical test on the role of midwives on the behavior of postpartum mothers regarding the COVID-19 health protocol, a p value of 0.000 was obtained. There is a correlation between the part of health workers and the conduct of mothers in exclusive breastfeeding. Health workers and health providers are obliged to initiate early breastfeeding for newborns to their mothers for 1 (one) hour. 15 That the role of health workers, particularly midwives, is a major part of efforts to prevent and control the(COVID-19. The study results showed that the behavior of postpartum mothers regarding the COVID-19 health protocol was mainly quite good (54%). It showed that a good level of the midwife's role would be followed by the behavior of postpartum mothers, who are pretty good. 16 The midwife's role has also been stated in the principle of preventing health protocols during the COVID-19 pandemic for pregnant women, postpartum mothers, and newborns in the community. The active role of midwives as one the health workers at the forefront is expected to remain selfless in providing maternal and child health care and must be more careful and alert to the "high risk" of exposure to COVID-19 transmission. Several measures to prevent COVID-19 infection during pregnancy, childbirth, and breastfeeding, among others, include universal precaution, by constantly washing hands with soap for 20 seconds or using hand sanitizer, using personal protective equipment, maintaining body condition by diligently exercising and getting enough rest, eating with balanced nutrition and practicing ethics cough and sneeze so that mothers can maintain and improve their health. 17 The role of midwives through IEC and health education about the COVID-19 health protocol is needed as a psychological boost in growing selfconfidence and encouraging behavior. In addition, the midwife's role regarding the COVID-19 health protocol will increase the knowledge of postpartum mothers so that a positive attitude and good behavior will be formed.
There was a correlation between information media on knowledge, attitudes, and behavior and prevention of COVID-19 transmission in the community. The results showed a significant correlation between information media and ability (p=0.006, p<0.05) r=0.308, which indicated a correlation between information media on knowledge, attitudes, and behavior and prevention of COVID-19 transmission in the community. 18 That there is an influence of the role of health workers, sources of information, and husband's support on postpartum mothers' adherence to breast care with a p<0.05. 10 A study stated there is a correlation between sources of information and knowledge of adolescents about sexually transmitted diseases at GBKP Christian Private High School Berastagi. 19 In the COVID-19 pandemic, the information media has become the most sought-after place for the public to collect information to increase knowledge and behavior. [20][21][22] The results showed that the information media about the COVID-19 health protocol was mainly good, 58%. Based on the anamnesis results for postpartum mothers in Teratai Room, RSUD Kabupaten Kediri, most good information media were obtained from cellphone information media, television, newspapers and websites, and magazines.
The results showed that not all media information about the COVID-19 health protocol was not good at 34.48%. Knowledge can be influenced by characteristics which include education, age, gender, intelligence, and socioeconomics. With good education and intelligence, supported by good information media. 23 The correlation between information media and postpartum mothers' attitudes showed that the statistical test through Kendall's tau obtained p=0.756 (p>0.05) means there is no significant correlation between information media on postpartum mothers' attitudes toward the COVID-19 health protocol. A study showed no significant correlation between information and attitudes in students at the Public High School 9 Manado. 24 A person's attitude towards objects has different intensities or levels because attitudes are influenced by internal factors, namely personal experience, cultural influences, family, mass media, educational institutions and religion, and emotional factors. 13 However, other factors that could influence the attitude of postpartum mothers in implementing the COVID-19 health protocol were not investigated in this study, for example, education and socio-culture, which could have been known more clearly. Differences in the results of this study from previous studies may occur because there are several aspects not examined in this study, namely factors that can influence attitudes, such as personal experience factors, cultural influences, and emotional factors. 23 The attitude is a readiness or willingness to act and not a particular motive. It will be easier to receive information to have a better attitude than someone with lower education. 25 The analysis results showed a significant correlation between information media and postpartum mothers' behavior in implementing the COVID-19 health protocol, p=0.012 (p<0.05).
There is a correlation between information media and knowledge, attitudes, and preventive measures for the transmission of COVID-19 in the community in Pelangi Gadang village, Ranah Pesisir subdistrict, Pesisir Selatan regency. 18 The results showed that there was a significant correlation between information media and COVID-19 prevention and transmission measures (p=0.028, p<0.05), r=0.241, which indicated that there was a correlation between information media and knowledge, attitudes, and preventive measures for COVID-19 transmission in a society with sufficient closeness. 18 The information media is a factor that influences the behavior of preventing COVID-19. 26 The results showed that the information media was primarily good about the COVID-19 health protocol affecting the behavior of postpartum mothers on good criteria by 50%. Information media has an effect related to changes in attitudes, feelings, and communication behavior. 27 Individuals who obtain information can determine how to react and make decisions when facing problems. 28 The information they get will influence the community to act and prevent the transmission of COVID-19. The use of media can motivate positive behavior from its users. 29 The media are broadly human, material, or events that build conditions to make a person acquire knowledge, skills, or attitudes to COVID-19 prevention. 30 There is a correlation between digital media and COVID-19 prevention behavior. This study explains empirical evidence that information related to COVID-19 in various digital media can trigger behavior to prevent the transmission of COVID-19 directly or indirectly. 20 The results showed a significant correlation between information media (p=0.002, p<0.05) and adolescent sexual behavior. 31 Information media cannot be left behind to participate in conveying important information to the general public, especially postpartum mothers. Media Information is one of the factors that influence knowledge and behavior. So what is seen can affect the behavior of postpartum mothers and tend to be imitated in implementing the COVID-19 health protocol. 27 Electronic information media influences because the information media in the form of electronics is audio and visual. Submission of information with audiovisual makes users understand more quickly and often immediately interested in doing the same thing.
The results showed that not all good information media affected the behavior of postpartum mothers at 3.4%. Not all communities comply with these provisions. Only 10% of the community has good COVID-19 health protocol habits: washing hands, wearing masks, maintaining distance, and complying with the COVID-19 protocol correctly. 26 This indicates that there may be other factors that influence the behavior of postpartum mothers regarding the COVID-19 protocol. Predisposing factors that influence behavior include belief, belief, education, motivation, perception, and knowledge. 23

Conclusions
This study concludes that there is a significant correlation between the role of midwives to the knowledge, attitudes, and behavior of postpartum mothers. In addition, there is a significant correlation between information media and the knowledge and behavior of postpartum mothers. However, there is no significant correlation between information media and the attitude of postpartum mothers regarding the COVID-19 health protocol.

Conflict of Interest
There was not a conflict of interest in this article.