Variable that influence achievement of indonesian students in the program international student assessment (PISA) 2015 using structural equation modelling (SEM)

The Program for International Student Assessment (PISA) is an examination system that evaluates the education system every 3 years to 15-year-old students randomly selected. Indonesia ranking was generally still below the average of the Organisation for Economic Cooperation and Development (OECD). In addition, the results of the PISA study are not only interpreted as ranking, but also important to carry out follow-up based on diagnoses produced from the PISA diagnostic survey to determine the weaknesses and strengths of Indonesian students and evaluate the education system in Indonesia. It is important to consider the use of the right and simple model to see the relationship between the diagnostic results of the performance of students’ PISA achievements in 2015. The purpose of this study was to select variables that significantly influence PISA achievement in 2015 using the SEM regularization model using The Least Absolute Shrinkage and Selection Operator (LASSO) method. The results of the analysis show that the variables that significantly effects students’ PISA results are the students’ insight into the environmental, the students’ perceptions of science truth, and the students’ participation in The International Standard Classification of Education level 0 (ISCED 0).


Introduction
Programme for International Student Assessment (PISA) is a test system that evaluates the education system from 72 countries around the world. The specific objectives of PISA in 2015 not only assess what students know about science, but also what they can do with what they know, and how they can creatively apply their scientific knowledge [1]. PISA is held by the Organization for Economic Cooperation and Development (OECD) every 3 years for 15-year students randomly selected. Indonesian students participated in the PISA since 2000. Based on the PISA student ranking results, Indonesia is at the bottom, above Brazil, Peru, Lebanon, Tunisia, Kosovo, Algeria and the Dominican Republic. The 2015 PISA result for Indonesia are ranked 65th grade Mathematics, 66th grade Reading literacy, 64th grade science.
In 2015 Indonesia was claimed to have experienced a significant increase in achievement which brought Indonesia 6 ranked upwards compared to the previous year, but this achievement was generally still below the average of the OECD. In addition, the results of the PISA study are not only interpreted as rankings, but also important to carry out follow-up based on diagnoses produced from The diagnostic survey assessed in PISA contains many latent variables that measure students' background and characteristics, so it is important to consider the use of appropriate and simple models to see the relationship between student background and characteristics of the 2015 PISA performance. Least Absolute Shrinkage and Selection Operator(LASSO) method has been popularly used to obtain a simpler model if there are many variables, especially if n << p [4]. However, this method cannot be applied to data containing latent variables. The estimation methods of possible penalties on structural equation models (SEM) had been developed to handle it, namely Regularized SEM [5]. This method minimizes certain parameters in SEM so that the model obtained is easier to understand and simpler [6]. This method is expected to provide a description of the relationship between students 'backgrounds and characteristics of students' PISA performance simpler so that they can be interpreted and are more efficient in terms of evaluating education in Indonesia. The purpose of this study is to select variables, both exogenous latent variables and observed variables that influence the performance of 2015 PISA achievement students using Regularized SEM.

Data
The data in this study are the results of the 2015 PISA diagnostic survey, especially participants of 15year-old Indonesian students with a sample of 6513 students. This data is obtained from http://www.oecd.org/pisa/data/. The missing observations in this study were issued, so the amount of data was reduced to 4.522. The response variable observed in this study is the result of PISA achievement, namely the value of mathematics, science, and reading. Furthermore, the explanatory variable is the student's background and the characteristics of students especially in the field of science (Table 1). Students' perceptions of science truth

Model and Analysis
In Table 1, it can be seen that the independent variables in this study consist of two observed variables and nine latent variables, so that the linear model used in describing the relationship between the response variables with independent variables is SEM. In this study it is expected that the model obtained is a model that is easier to understand and simpler. SEM Regularization is one method that can be used to achieve this goal. The modelused in this study is SEM, i.e. which is a structural model, where η 3x1 is a vector of endogenous latent variables, ξ 9x1 is a vector of exogenous latent variables, ζ 3x1 is a structural error matrix, B 3x3 is a coefficient matrix for endogenous latent variables, Γ 3x9 coefficient matrix for exogenous latent variables.
SEM generally implements the use of a RAM matrix that requires three matrices to specify SEM as a whole, namely filters (F px(p+l) ), asymmetric (A), and Symmetric (S). The F matrix contains one that corresponds to the manifest and zero variables except that p is the number of manifest variables and l is the number of latent variables. A matrix contains a loading factor which is a regression between manifest variables with latent variables. The S matrix is a square matrix that contains a variety or uniform for each manifest variable relationship with a latent variable. The three matrices are used to calculate the uniform matrix derived from the model, i.e.
The matrix is then entered into the loss function, for example maximum likelihood (ML).
where C is the observed covariance matrix, and p is the expected number of parameters. LASSO is one of the general methods of regularization in regression which suppresses nonessential variables to zero [4]. The LASSO estimation defined by equation (3) uses l1-norm to press βj to 0 with λ as the selection parameter.
SEM regularization (regSEM) generally implements the use of a RAM matrix which has the main benefit of only one matrix needed to capture direct influence as a natural parameter for regularization. RAM makes formulations simpler than having to add penalties to more than one matrix. SEM Regularization is built by adding elements to suppress certain model parameters, namely: with λ is the regularization parameter with λ ≥ 0. When λ is zero, the results obtained are the same as MLE, and when λ is very large, all of the marginalized parameters will be reduced to zero. P (·) is a function of the sum of one or more values of the model parameter matrix. Form P (·) in LASSO (ǁ.ǁ1), which terminates the absolute number of parameters. RegSEM specifies which parameters are penalized in the SEM model in general. Penetrating parameters in the regSEM in RAM matrix notation, only terminates the A or S matrix (not both) in the model. This is because penalizing parameters of one matrix means that it will also suppress other matrix values that are compatible.

Stages of Analysis
The data analyzed using Software R.3.5.2 uses the Package "lavaan" and "regsem" with the steps of the research, which first changes the category variable into a numerical variable, namely the dummy variable and Method Of Successive Interval (MSI) [9], this is important in linear modeling. Then the second step standardizes each data first before being analyzed so that the LASSO method can work properly. The standardization of this data is done so that the data has mean 0 and variance 1 [10]. The third step in this research is to model Y with X using SEM Regularization using selected lambda values with CV regularization SEM results. The choice of lambda is important to get a more concise model. The final step is the interpretation of results, which are the variables that have an important influence on student PISA achievement.

Choosing of Selection Parameters (λ)
The choice of the value of λ is very important in obtaining a model of sparsity, which functions to suppress non-essential coefficients to zero [11]. The general technique that is usually used to select the value of λ is cross-validation (CV) based on the Bayesian Information Criteria (BIC) and Root Mean Square Errorof Approximation (RMSEA) values. Previously, parameter estimation was done on the SEM model using the ML method, then the estimation results were used as the initial model in SEM regularization through the CV method to choose the right λ value. Figure 1(a) shows that the lambda value that produced the smallest BIC value is zero. This meant that all variables are not selected, so CV result needed to be analysis further to get the best lambda value used in modelling. Based on this, the lambda value used to model the data is λ = 0.14 and λ = 0.18. Of the three SEM Regularization models with different λ analyzed further about the goodness of the model not only from BIC, but also from RMSEA. Table 2 is a comparison of the RMSEA and BIC values for the SEM model and SEM Regularization with different λ. The results in Table 2were consistent with the results in Figure 2, namely the ML method or λ = 0 produced the smallest RMSEA and BIC values, but not too significantly different for RMSEA, as well as BIC.

The SEM Model and The SEM Regularization Model
The SEM regularization model with the LASSO method produced a more concise model than the SEM model. Penalty LASSO in the SEM regularization model was only given to the regression coefficient, namely the coefficient on the structural model in the SEM model. This can be seen in Figure 2, which is the estimated result of the coefficient parameters of the SEM, RSEM.14, and RSEM.18 models. Note that the SEM Model included all the variables in the model, because there was no penalty given to the regression coefficient, so the non-essential variables remain in the model. Therefore, it needed a method that can produce a simpler and more concise model, namely a model that only included important variables or which had a significant effect. The expected results using RSEM.14, there were 5 variables which wereshrinked to 0 with 1 negative variable, while RSEM.18 as many as 6 variables wereshrinked to zero. Note also that the selected variables in the SEM regularization model were indeed variables with insignificant effects in the SEM model. Therefore RSEM.18 was a SEM regularization model with λ = 0.18 having a BIC value (i) which had the largest difference with BIC thereafter (i + 1) and produced a more concise model than the regularization model with λ = 0.14, then the selection of explanatory variables which affects the achievement of Indonesian students' PISA in 2015 is the SEM regularization model with λ = 0.18. The results show in Figure 2.  Table 3 was estimated results of parameters using the SEM and SEM regularization method. Based on the results in Table 3, the SEM regularization method with λ = 0.18 produced a model consisting of variables that have a considerable effect if returned to the SEM model, so the variables that significantly affect the results of student PISA were student's insight into the environmental issues, student's perceptions of science truth, and the student's participation in ISCED 0. Students' insight into the environment had the greatest effect, which was equal to 0.338. This meant that the better the student's insight into environmental issues, the higher the PISA achievement obtained. While the students' perceptions of the truth of science had an effect of 0.195, which was increasingly agreeing to the truth of science, hence increasing the PISA achievement obtained. The participation of students for one year in ISCED 0 tended to give a higher score of PISA results of 0.110 times than those who did not attend ISCED 0. Meanwhile, the participation of students for more than one year in ISCED 0 tended to give PISA results higher 0.217 times than not following ISCED 0.

Conclusion
The SEM regularization method provided predictive results that are simpler and easier to understand than the predicted SEM model. This was because the model produced only consists of the variables that had the greatest influence or the most important. SEM regularization using the LASSO method with λ = 0.18 was quite good in selecting variables that have very little effect. In addition, the resulting model was also simpler and easier to interpret. The variables that significantly influence the results of student PISA are student's insight into the environmental issues, student's perceptions of science truth, and the student's participation in ISCED 0. In this study only included variables that only use a few background variables and student characteristics, there were many other variables had not been included which can affect student achievement of PISA. This could be used as the basis of further research.