An exploratory PLS study of academic success / failure

Strategies to combat academic failure in higher education rely on the identification and the knowledge of the causes associated with it. In order to identify these causes, a questionnaire survey was implemented in one public Higher School of Technologies and Engineering in Portugal. Student-related aspects were those that exhibited the highest impact on academic success/failure. Teachers and institution/curriculum exhibited an indirect impact through mediation by student-related aspects. Other variables were identified with a negative residual impact on academic success, such as: being displaced from the habitual residence, disliking to study, and being a male. With a slightly more substantive impact, emerged the sleep time on the eve of the tests.


Resumo
Um estudo exploratório PLS do sucesso/insucesso académico As estratégias para combater o insucesso académico no ensino superior dependem da identificação e do conhecimento das causas a ele associadas. Para identificar essas causas, uma pesquisa por questionário foi implementada numa Escola Superior de Tecnologias e Engenharia. Aspetos relacionados com os estudantes foram os que apresentaram maior impacto no sucesso/insucesso académico. Professores e instituição/currículo apresentaram um impacto indireto através da mediação por aspetos relacionados com os estudantes. Outras variáveis foram identificadas com um impacto residual negativo no sucesso Introduction Academic failure in higher education translates into a scholar drop-out and high retention rates and into an extended time to complete university courses. It is necessary to understand the involved dynamics in adaptation, retention, and scholar drop-out, identifying causes of failure so that appropriate preventive measures can be introduced and implemented.
Strategies to combat academic failure in higher education come from the knowledge of associated causes, as well as identification of the importance for each of these causes and eventual interactions between them.
In order to study the leading causes of academic success/failure, it becomes necessary to define metrics to measure it. Different definitions can lead to different results and make it impossible to compare studies carried out on the subject.
Academic success/failure can be measured by academic results as it was done by Monteiro et al. (2005) and Coutinho (2007). Monteiro et al. (2005) analyzed methods of study from 1st-year students at the moment of the entrance at the University of Minho, using the Inventory of Attitudes and Habitual Behaviors of Study -Iache (TAVARES, 2004), which is a questionnaire to measure study methods, consisting of cognitive, motivational and behavioral dimensions. The academic performance of students was measured through the ratio of curricular units performed by students According to Tavares (2000), three domains establish academic success/ failure: academic domain (school performance, study skills, evaluation, and curriculum), social-relational domain (way of being at the university, student role adjustment, insertion on the university context, offered arrival opportunities, satisfaction with relational and social environments) and biopsychological domain (physical and psychological health, quality of life and well-being, satisfaction with life in general).

FLUXO CONTÍNUO
From a study with students from the University of Minho, Taveira (2000) identified some associated factors with academic failure and grouped them into two main sets: individual and contextual factors. Individual factors cover demographic, academic, personality, and role variables. Contextual factors cover the variables concerning university context, family context, and context of peers.
According to Almeida and Araújo (2015), academic success/failure can be measured by school classifications, a weighted average of ratings; the ratio between the number of approved curricular units and number provided in the curriculum; permanence and completion rates, number of years to complete. They also identify associated factors with academic success/failure. These factors may be associated with the student, teachers, university course, and institution. They identify critical moments for academic failure and dropout. Intervention measures are mentioned.  Tinto (1993) states that the two main reasons for college student attrition are related to low academic and social integration. Academic integration encompasses academic journey, students' perception of their progress, and belief in the personal commitment of teachers to teach and help students. Social integration covers self-esteem, relationships with colleagues, and teachers. Bento (2008) points out to adaptation difficulties during the first year such as distance or indifference from teachers, work rate, academic traditions, family withdrawal, demanding assessment and complexities of curricular units, teaching methods and evaluation system, large classes, lots of material to learn, and too long classes. Sibanda et al. (2015) identify several factors that were considered by students as having a significant influence on academic success, such as regular study, presence at classes, hard work, commitment, and dedication.

FLUXO CONTÍNUO
In the study performed by Fraser and Killen (2003), the authors identify, at the University of Pretoria, which factors teachers and students perceive as having a more critical influence on academic success/failure of students at the university. They found that many students pointed to external sources to explain academic failure.
As stated above, different causes were identified in various studies to explain the academic success/failure of students. The present study aims to explore the role of some of the reasons that may contribute to academic success/failure in the first semester of the 2017/2018 academic year, for students who were enrolled in a public Higher School of Technologies and Engineering in Portugal. Only data and success/failure related to this semester were considered. The factors used in this approach to academic success/failure focused on four factors: factors associated with the teacher, the student, the institution, and the curriculum, factors that can be summarized by empirical studies performed by other authors.

Materials and Methods
This paper considers school success/failure from the perspective of academic achievement in the course attended. Therefore, a student's failure is when he or she has not been able to reach the approval goals within the predetermined time.
However, school success/failure, in a broader sense, cannot be restricted to this aspect alone. Drop out, and the reasons for it, are not addressed in this study. Based on the identified variables in the literature associated with academic success/failure, and in order to answer the objectives proposed by this study, a questionnaire survey was built (Appendix A). A pre-test was performed with some students and teachers, and suggestions were incorporated in the final questionnaire, which was placed on a digital platform to collect students' answers. They were encouraged to respond, having been explained the importance of it. They were also informed that participation was voluntary and that anonymity would always be guaranteed. The questionnaire was available for a period of two months, from February 27 to April 27, 2018, with 205 responses collected. All students were considered regardless of the academic year.

FLUXO CONTÍNUO
For the present study, two variables were constructed to measure academic success/failure.
Since university courses of this Higher School have five curricular units in the first semester, only those students who were enrolled in that semester to at least five curricular units were considered, avoiding cases such as when a student was enrolled on a single curricular unit, which would more readily correspond to a success rate of 100%, if he or she got approval to that curricular unit. For students enrolled in more than five curricular units that got approval in at least five, the success rate was considered to be of 100%, since they would have a work overloaded, compared to others, to achieve a real 100% of approved curricular units. Thus, the first constructed variable was calculated as the ratio between the approved and planned curricular units, which would always be five. All students with more than five approvals were considered to have achieved success, in this variable, of 100%.
The second variable concerns the scores, which was computed as the sum of the five (or less, if the student had obtained approval in less than five curricular units) best scores obtained.
Hence, the two variables constructed were the ratio of the number of completed curricular units by the number of planned curricular units for the first semester of the academic year 2017/2018 (corresponding to five), and another by the sum of the five (or less, if the student had obtained approval in less than five curricular units) best scores obtained.
In order to detect situations that could influence the results of the statistical procedures to be performed, all answers obtained were initially reviewed. Three cases were eliminated owing to inconsistencies in the responses. As only students who were enrolled in the semester in at least five curricular units were considered, all students with fewer than five curricular units were eliminated from the database, which corresponded to 34 students.

FLUXO CONTÍNUO
Scores were only considered if their number agreed with the number of curricular units performed, except when the student was approved in more than five curricular units and provided at least five scores. In this case, the sum of the five best scores was computed from the available ones. This way, 17 additional cases were eliminated from the dataset. Hence, the database reduced to 151 students, from which four were master students, which were also excluded. Finally, the database remained with 147 undergraduate students.
The items of the questionnaire measuring the factors associated with the teacher, the student, the institution, and the curriculum, were first submitted to exploratory factor analysis, and three factors were identified.
To explore the possible causal effect of the different variables and of the three factors that arose from the exploratory factor analysis, on the academic success/ failure, a partial least squares (PLS-SEM) model was built.

Results
The final database included 84.4% male and 15.6% female students. The majority of the students on the sample (72.1%) were not displaced from their habitual area of residence. About 28.6% of the students in the sample considered that their financial situation was insufficient or very inadequate, and 25.2% reported suffering from a chronic health problem.

Factor Analysis
The items of the questionnaire measured on a Likert scale with seven levels were first submitted to exploratory factorial analysis. For the present sample, parallel analysis and Velicer's minimum average partial pointed to the existence of three factors. Through principal component analysis followed by a varimax rotation, three factors were identified: one associated with teachers, one associated with students, and a third associated with the institution and the curriculum. It was not possible to separate the institution, and the curriculum in two different factors since the analysis pointed to one single factor. The factors that emerged from the exploratory factorial analysis with the items measuring each of them are described in Table 1.

Structural Equation Modeling
PLS-SEM (WOLD, 1980;HAIR et al., 2014) was used to analyze the relationships between the variables. It consists of two distinct submodels, the measurement model and the structural model. The later establishes the relationships between latent variables aiming to minimize the residues of fundamental equations. The model applies the ordinary least squares method to each of its equations. PLS-SEM is thus suitable for predictive or exploratory analyses, and when the sample is small (HSU et al., 2006). The model can work well with variables measured in ordinal scale and dummy variables. The distribution of the estimators is unknown; therefore, it is necessary to use methods such as bootstrap to evaluate their significance. The measurement and structural submodels were analyzed.
The model considered is formed by two central endogenous variables already described. In essence, these are the rate of curricular units performed, and the total score obtained, metrics used to quantify academic success/failure. The model considers that the rate of curricular units completed with success impacts the overall score and that there is an indirect effect on the two central endogenous variables, by the factors "Teachers" and "Institution/curriculum", through mediation by the "Students" factor. Besides the two primary endogenous variables, all other variables, except for latent variable "Students", are exclusively exogenous.
The software used to analyze the PLS-SEM model was Smart-PLS 2.0 (RINGLE et al., 2005). Estimates for the relationships between the variables were obtained, that is, the values and importance of the impact of different variables on the endogenous variables, in particular, the two central variables that measure academic success/failure.
The tested model (after elimination of some others exogenous variables due to being simultaneously nonsignificant and having very low impact coefficients, so that their inclusion could not be justified), and the results obtained by the PLS-SEM analysis, loadings of the measurement model, impact factors and determination coefficients of the structural model, are depicted in Figure 1 (the abbreviations are described in appendix B).

Measurement Model
The loadings in the measurement model were not too low; therefore, none of the items were eliminated. Also, a model with good convergent validity and good discrimi-   "Teachers" factor. The AVE value should be at least 0.5 to support good convergent validity. Although one of the values was less than 0.5, the criterion of Fornell-Lacker was verified (FORNELL, LARCKER, 1981). This criterion stipulates that the square root of the AVE for each of the factors of the model should be superior to all correlations between that factor and all the others. It serves to evaluate the convergent and discriminant validity of the model.

Structural Model
The model explains 26.7% of the variable "Rate of disciplines performed", and 84.6% of the variable 'Total score obtained' in the five best disciplines.
The SmartPLS software standardizes all the variables before estimating the impact factors. What this represents is that if an impact factor takes the value x, for each variation unit of the exogenous variable standard deviation, on average, the change in the endogenous variable is of x standard deviations.
The significance of the model was estimated using bootstrap resampling, in which 5000 subsamples were randomly taken from the original sample with replacement. A minimum of 5000 samples was used (HAIR et al., 2014) so that the empirical t statistic follows approximately a normal distribution. The statistical significant impact coefficients, obtained by bootstrap, are depicted in Table 2. Table 2. Statistical significant impact coefficients.
The non-significant impact coefficients displayed simultaneously small effects, so they were not considered for the identification of the variables that explain the success/failure of the student. The mean scores for each factor and variables involved in the significant associations with the inclusion of the variable "Hours dedicated weekly, on average, to class attendance", are shown in Table 3. These values indicate that the three factors, "Students", "Teachers", and "Institution/curriculum", were positively evaluated by students, especially the "Students" factor. It is also observed that the students, on average, like to study and do not have a lack of time to prepare appropriately. The students spent weekly, on average, only 3.6 hours to class attendance, a deficient value. the "Institution/curriculum" on the academic success/failure of the student. "Teachers" seem to have only a residual direct influence on student achievement. However, these two factors ("Teachers" and "Institution/curriculum") had a significant impact, at a significance level of 10%, on the "Students" factor, which leads to the conclusion that their main action is perhaps via mediation through the "Students" factor and not by direct effect.
Other variables not usually referred to in the literature, were recognized as being explanatory variables of academic success/failure. Therefore, variables such as disliking to study, being displaced from the permanent residence, and being a male were identified with a negative residual impact on academic success. With a slightly higher substantive impact, the sleep time on the eve of tests appears as a reason for academic success/failure.
The number of hours dedicated weekly, on average, to class attendance appears as a marginally significant variable for academic success/failure. The sample showed that the students spent weekly, on average, only 3.6 hours to class attendance, which is a deficient value and can negatively impact academic success.
Finally, the lack of time for students to adequately prepare arises as a cause for academic success/failure. In this case, the result is unexpected, since the model indicates that success increases, on average, with the lack of time. However, the variable is just the perception that students have about the time they need to prepare and not the actual time they have. The best students, those who generally achieve success, may perceive a higher lack of time than the less successful students.
For this Higher School, the three factors "Students", "Teachers", and "Institution/curriculum" were positively evaluated by the students, and on average, they like to study and do not have a lack of time to prepare appropriately.
Although students in the first year have more adaptation issues to the university, in the present study the variable academic year revealed to be not significant and with a minimal impact in the model constructed.
Initially, a more substantial response rate, close to the population, was the goal. Several efforts have been made to achieve this goal. However, the obtained res- Teachers are not consistent between what they say and what they do. 1 2 3 4 5 6 7 Teachers displease me. 1 2 3 4 5 6 7 Teachers do not treat all students equally. 1 2 3 4 5 6 7 Teachers are not very comprehensive. 1 2 3 4 5 6 7 The questions in the exams do not capture the essence of the subject. 1 2 3 4 5 6 7 Teachers do not clearly state the subjects. 1 2 3 4 5 6 7 Teachers do not take my level of understanding into account. 1 2 3 4 5 6 7 I'm not interested in the subjects taught. 1 2 3 4 5 6 7 I cannot concentrate on the classroom. 1 2 3 4 5 6 7 I do not like to study. 1 2 3 4 5 6 7 I cannot compete with my colleagues.
1 2 3 4 5 6 7 I do not feel that I am guaranteed to get a job at the end of the course. 1 2 3 4 5 6 7 I'm worried about my future.
1 2 3 4 5 6 7 The evaluation criteria of the curricular units are not adequate.
1 2 3 4 5 6 7 My knowledge is insufficient for the course I am attending.