A novel academic performance estimation model using two stage feature selection

Received Nov 2, 2019 Revised Feb 15, 2020 Accepted Mar 5, 2020 Educational data mining has gained tremendous interest from researchers across the globe. Using data mining techniques in the field of education several significant findings have been made. Accurate academic performance estimation has been a challenging task due to the variety of students’ attributes involved. In this study we have developed a novel framework to estimate the academic performance of students. Our proposed model outperformed existing models of students’ academic performance determination and gives a new direction to academic performance estimation. The proposed model can help not only to reduce the number of academic failures but also help to comprehend the factors contributing to a students’ academic performance (poor, average or outstanding). Some of the techniques used were conversion of categorical attributes into dummy variables, instance segregation, classification using an optimised and improved differential evolutionary algorithm.


INTRODUCTION
Educational Data Mining is an emergent field of research that uses data mining in the field of education. Determination of academic performance has gained the attention of several researchers in the recent times using machine learning techniques [1]. Machine learning algorithms enable educationist to predict the students' results before the examination was conducted. These results could be used to make pedagogical interventions to prevent failures of the academically underperforming students. Many researchers have used data mining techniques in the domain of education [2][3][4][5][6][7][8][9]. Most researchers have used classification algorithms to train the classification /prediction model using a dataset consisting of students' information. Some of the classification algorithms used was neural network, decision trees, support vector machines and naive bayes classifier [10,11]. Experiments initially were conducted using these popular classification techniques. The accuracy of the classification model highly depends upon the quality of the data used as input. Data pre-processing techniques like handling of missing values, noisy data and balancing the minority class instances were used before the implementation of classification technique to improve the accuracy of the classification models [12,13]. Feature selection has been used to identify the relevant attributes in student datasets and others [14][15][16].
The data collected from the students through questionnaire is often incomplete and requires preprocessing techniques to be implemented. The student dataset is generally unbalanced. A dataset is unbalanced if the class labels of data instances are not uniformly distributed. In such datasets it is found that most instances belong to one class often called the majority class and a few instances of the dataset belong to the minority Others used attributes like internal grades and Cumulative grade point Average (CGPA)). Demographic features like parental background, health condition, were found to have an impact on academic performance [17]. Psychometric tests have been conducted by researchers to determine academic performance. Amongst several classification algorithm, Radial Bias Function (RBF) network, Decision tree, Naive Bayes classifier and Support Vector machines (SVM) have gained significant popularity. RBF has been used for classification in several domains [18]. Support Vector machines (SVM) has been used by researchers in classification problems and has been found to outperform other classifiers [19]. Naive bayes classifier has gained its popularity due to its high accuracy despite its simplicity [20]. The attributes of the dataset have are often tested using feature selection algorithms. Feature selection is the process of selecting a subset of attributes/ features from a set of features [21]. The outcome of feature selection algorithm is a sub-set of features using which the classification model can accomplish the task of generating highly accurate results. The findings from literature survey were as following: a) The student dataset mostly consisted of several categorical attributes. Categorical data attributes need to be handled properly before it can be used as input for classification. The attributes like gender, mother's qualification, internet browsing pattern are categorical data. The categorical data of any dataset holds valuable information. The categorical data cannot be treated as numeric data for classification algorithms and need to be handled. b) All models for academic performance determination are based on classification. Some models of academic performance determination use feature selection techniques to find the significant features from the student dataset. c) No research has been conducted before to identify attributes specific of each class of students (based on their academic performance: poor, average and outstanding). For example, identification of specific attributes/ characteristics of the academically underperforming students has not been tried by researchers. Hence our problem statement was to identify the characteristics of academically weaker students, the characteristics of academically average students and the characteristics of academically outstanding students using an optimised classification algorithm. Further, the intention was to compare these characteristics and address the issues attributing poor and average academic performances. The proposed solution consisted of comparision and implementation of existing EDM models. Further a novel framework for academic performance estimation is proposed. Our proposed novel model not only outperforms existing EDM models but also helps to identify specific attributes of students who are academically weak, average or outstanding. The paper is organised as follows: Section 2 describes the experimental setup. Section 3 discusses the result obtained and its implication. Section 4 concludes the paper.

RESEARCH METHOD 2.1. Data collection
The primary dataset consisted of students' data from KIIT University and Biju Patnaik University. Before considering data collection from students, a systematic analysis of the features that should be used in the research was done. For this purpose, we have taken the opinion of researchers from literature survey, faculties, parents and placement cell. The following attributes were considered as the relevant and necessary features required for the estimation of academic performance.  These attributes were collected from students pursuing engineering from two educational intuitions: School of computer engineering, KIIT University and Silicon Institute of Technology in Bhubaneswar, India. Data was collected in form of questionnaire. Prior permission was taken from the concerned authorities.

Data pre-processing
The data collected from students could not be directly used as input for the classification. Pre-processing the data improves the effectiveness of the classification algorithms. There were missing and incorrect values in some of the data instances. Missing and invalid data entries were replaced by the mode of the attribute values. Since most attributes in the data set are categorical data hence special techniques were used to handle the categorical data. For example, the attribute 'father's occupation', could have the values: service, business and others. In order to use these values as a part of classification model, these values should be mapped into numbers. One of the frequently used methods for this would be mapping each of the unique value of the attributes to numbers say 1, 2, 3. The number of values required will depend upon the number of levels of an attribute for example number of levels is three for attribute occupation. However, converting values into numbers may not be correct procedure. For example, if attribute value service is marked as one, business as two and others as three then it implies that business is less than service and service is less than business. Working with these assumptions would give inaccurate results. Hence a search for other approaches with more accurate results was done. Dummy variable is one of the techniques. Dummy variable represents one level of categorical variable. Presence of a level is represented by 1 and absence is represented by 0. For example, instances 1, 2, 3 and 4 are shown in the table with the attribute gender and their respective class are shown in Table 1. For every level present, one dummy variable will be created. Table 2 shows dummy variables male and female created. The same principle was adopted for all categorical attribute levels were converted into dummy variables. This further can be used for classification algorithms. The dataset was unbalanced and 90% of the instances belonged to the majority class. To restore balance synthetic samples were generated using SMOTE algorithm [22].

RBF as a classifier
Though several classifiers were used, RBF was mostly used in our proposed work as it outperformed other classification models. RBF network is a supervised learning algorithm used to build classification models. It has been increasingly used by researchers as a popular artificial neural network in the fields of machine learning. RBF is a three-layered feed forward network as shown in Figure 1. It is a three-layered feed forward network consisting of input layer, a hidden layer and output layer. The hidden layer implements the activation function. The training process of RBF network is faster than MLP. It allows selecting suitable parameters for centre and spread of hidden units and can provide high accuracy rate comparable with classification methods like SVM.

Optimised RBF with novel and improved DE
Differential Evolution is heuristic search algorithm [23] similar to the genetic algorithm which maintains a population of possible solution. There are different variants of the differential evolution. However, the authors have used a scheme based on the DE/ rand/1/bin scheme [24]. It also maintains a pool of chromosomes each encoding a possible solution. An iterative process of mutation, recombination (crossover) and selection of chromosomes continues until the desired fitness of the chromosomes is achieved. Displaying the iterative process of mutation, recombination and selection as shown in Figure 2. A randomly generated population of size n is initialized and evaluated using a fitness function f iteratively. The mutation process creates a donor vector which is capable of encoding the solution using randomly selected members for the population. Recombination then creates a trial vector by combining two donor vectors. Selection is the process of determining whether the newly generated trial vector can form a part of the population using fitness function. The donor vector is obtained using the following equation: Where k, l and m are mutually distinct and randomly selected indices. The trial vector is created by the following process of recombination as explained in (4): (4) Where i is a random number generated, is the crossover constant. If the child instance created or the trial vector has higher fitness function than the parent will be replaced. Then selection process helps to select the best individuals as per (5).

(5)
In the novel and improved Differential Evolution algorithm, we have modified the mutation process and altered the process of creating donor vector as in (6).
( ) (6) where, r1, r2, r3 and r4 are randomly selected four indices. is the best valued individual among x r1 , x r2 , x r3 and x r4 . Also x r1 > x r2 x r2 > x r3 and x r3 > x r4 The reason behind modifying the donor vector is that an improved donor vector would contribute to better individuals in the population. It will further positively impact the recombination process and selection process.

Feature selection
Feature selection is the process of identification and removal of redundant attributes. These attributes do not do not contribute to the classification process [25][26][27]. The reason behind feature selection is not only to reduce computational complexity but also makes the classification model simpler and accurate. There are two types of feature selection method: filter and wrapper-based method. Filter based classification model is a process assigns a weight to each attribute [28]. The weight depends upon the statistical characteristics of the dataset. Feature selection also helps to analyse and interpret results. Feature selection has been done using filter-based feature selection method. There are several filter-based feature selection algorithms [29,30]. The Relief Based Feature Selection, Entropy Based Feature Selection and Correlation based feature selection was used. All these algorithms have been used by several researchers for their accuracy and simplicity. Using these algorithms each of the attributes of the student dataset was investigated. Since relief, correlation and entropy are filter based feature selection methods hence they return a ranking of the features. These rankings helped to identify the prominent attributes responsible for determination of academic results. Figure 3 demonstrates our process model.

1615
After pre-processing the data and implementing feature selection, our proposed model uses a novel differential evolution algorithm to optimize the RBFN as the classification algorithm. Further instances of each class category is segregated and second stage feature selection is implemented to understand and analyse the patterns in students' academic performance.

RESULT ANALYSIS AND DISCUSSION
This section presents the results of the experiments of the proposed work as well as compares the experimental techniques used by other researchers with our model. Many works have been done to determine academic performance. Experiments conducted to determine academic performance using data mining can be outlined into the following three types: a. Classification based b. Classification with data pre-processing c. Classification on pre-processed data with selected features The authors have used these methods on their data and then have compared these approaches with their proposed method. Accuracy and f-measure have been used to evaluate the effectiveness of the experiments.

Fmeasure = (7)
Similarly Figure 4, 5, 6 displays the f-measure of classification model of type I, II and III experiments respectively. It was observed that RBF Neural Network, optimised with differential evolution, resulted in better accuracy and f measure value. They also used a novel and modified differential evolution. Therefore, the authors implemented their proposed work with this model. In their approach as discussed in Section 2, all categorical attributes were converted into dummy variables. The RBF model using dummy variables gave an accuracy of 84% which is a significant improvement over type II experiments. Our model consisted of incorporating dummy variables, usage of feature selection algorithms to remove redundant attributes. Further the RBFN classification model was optimized using differential evolutionary algorithm. We also modified the differential evolutionary algorithm as discussed in the previous section. It has been used to find the optimum centre and spread for the RBFN network. Figure 7 shows the accuracy obtained from each model.  The experimental results show that a subset features consisting of 60% of original features provide the best results for the classification model. An investigation on the ranking of the attributes returned by the feature selection algorithms was done. This helped to identify the attributes which highly influence academic performance. The ranking of the attributes by the feature selection algorithms are shown in Table 3. Two stage feature selection has been used in our study. In the first stage experiments were conducted with 30%, 60%, 90% and 100 % of features.
The accuracy of the resultant classification model was measured and plotted as shown in figure. It was found that the classification accuracy was highest with 60 % of the input attributes. Hence 60% of the features were used for the final classification model. The second stage of feature selection was used on individual categories/ classes. The experimental data consisted of three classes of students' academic performance: the outstanding students, the average performing students and the underperforming students. Applying filter-based feature selection in the second stage on individual classes helped us to identify the attributes of individual categories of students as shown in table and take appropriate measure. This was achieved by segregating the data instances of three categories of data. Further a ranking of attributes was done using the filter-based feature selection algorithm. The results obtained are shown in Table 4. These significant results obtained helped us to device different approaches for students based on their academic performance. It was found that the outstanding students were self motivated, had high attendance percentage and were engaged in regular study. Most of the average students were found to have some internet addiction. They spent a lot of time with friends and mostly studied only before exams. These students accepted that a more sincerity on their part could certainly improve their grades. Once these students are identified using the classification model, they can be motivated to study regularly and to reduce the time spent on internet. The underperforming students showed a variety of attributes as a key factor to their academic performance. Lack of interest in academics, internet addiction, financial distress and family problems and very low attendance were some of the issues. Students with academic negligence were recommended remedial classes. Students with family problems and indifference were suggested counselling sessions. It was observed that internet usage has significantly increased among the student community resulting in deterioration of academic performance. Instances of internet usage were accepted by students within classrooms. This needs to be taken care by educational institutes and parents before internet addiction turns into an epidemic. Low cost of data and smartphone availability has even fuelled this addiction. Internet usage should be reduced by motivation and promoting participation in extracurricular activities Previously all research work done so far for academic performance determination have followed a unified strategy of implementing classification and feature selection techniques on the entire dataset. The features selected in the process were used to predict results of different categories of students. This method cannot work efficiently as it treats all students equally. However, our novel framefork suggests first seggregation of instaces of the academically weaker students, average students, the outstanding students and then application of feature selection on these segregated groups. This segregation of attributes helps to take different pedagogical interventions for different students.

CONCLUSION
In this paper the authors have developed a novel model to identify the academically weaker students, the average students and the outstanding students. A comparative study and analysis of the existing EDM models was done on our data set. In the proposed approach the authors have converted categorical attributes into dummy variables. Apart from use of classification model, our model proposed a two-staged feature selection technique. In the first stage of feature selection, filter-based feature selection methods were used to analyze the student attributes that affect their academic performance. In the second stage of feature selection, segregated instances of each class were used separately to identify the dominant attributes specific to that class. The findings were important and helped to comprehend academic performance pattern of each class category of students. The RBFN network model used for classification was optimized by a novel and improved differential evolution technique. Our proposed framework of student academic performance estimation could not only help to reduce the number of failures in university examination but also improve the academic performance of the average students.