Quantitative reasoning and written communication. Competencies that evidence scientific thinking in university graduates

The purpose of this manuscript was to characterize and analyse both the differences and similarities of the results between the different careers in a public university in eastern Colombia, in the state tests Saber Pro 2017, in the modules quantitative reasoning and written communication, skills needed to evidence the level of scientific thinking, very useful for professionals in engineering and science, especially for professionals in mathematics and physics. In this research, a univariate descriptive analysis was carried out, followed by a principal component analysis. Engineering programs tend to perform better in quantitative reasoning and those of social sciences and education in written communication. These results indicate that graduates of engineering and science programs have a level of scientific thinking more related to the exact and natural sciences.


Introduction
The "Ministerio de Educación Nacional", Colombia through the "Instituto Colombiano para el Fomento de la Educación Superior (ICFES)" in its effort to evaluate the quality of education annually conducts state tests at different levels of the education system (primary-secondary and academic). The Saber Pro test is part of a set of instruments to evaluate the quality of the public educational service and exercise its inspection and surveillance [1]. The publication of Law 1324 and Decree 3963 of 2009, established as a mandatory requirement the presentation of this test to obtain the title of the undergraduate level. Generic competences are evaluated in five areas: critical reading, quantitative reasoning, written communication, English and citizenship competencies. This study focuses on the generic competencies of quantitative reasoning and written communication.
One of the problems related to the reports on the Saber Pro exam presented by ICFES is that its results are understood and used by a minority. Hence the little impact that these reports generate to promote change, because the bulk of the population, not understanding what they present, is left without the opportunity to reflect, and support the improvement of education. The importance of the present study consists of promoting the statistical value of the results, where, in addition to the presentation of the graphs and tables, the concepts and what the data implies within the study are clarified.
The present work focuses on making a characterization, through a univariate and multivariate statistical approach, of all the academic programs of the Universidad Francisco de Paula Santander (UFPS), Colombia, based on the average scores obtained in each of the areas evaluated in the Saber Pro 2017 Test. This allows providing a tool as a source of institutional information about the strengths and weaknesses of the careers, to be considered by the university in the improvement plans of the quality of the academic programs.
On the other hand, principal component analysis (PCA) is a statistical technique for reducing the size of the data. It allows explaining most of the total variability of the set of quantitative variables of interest with the least number of components. In addition, it is a geometric method of descriptive character whose purpose is to discover the underlying structure in the data [2]. The PCA converts the original set of variables into a smaller set of variables (principal components) that are linear combinations of the former. In addition, the principal components are characterized by their unrelatedness. One of its objectives is to facilitate the interpretation contained in the data [3]. In accordance with the above, the objective of the study was to characterize the results in the 2017 Saber Pro test of the academic programs of the public university of Colombia.

Theoretical framework
The ICFES makes available to universities and researchers' information on the results of the Saber Pro test broken down into different levels, so that studies are conducted whose results provide the sustenance to take actions that improve the quality of education. This test evaluates a set of generic competences to all students without distinction of their area of knowledge through five modules [4]. With the information that ICFES has been providing for years, a variety of studies have been conducted about evaluation of education in Colombia.
On the other hand, Perez et al. [5], carried out a study in students of recent admission to the Universidad de Santander, Colombia, for the period A-2016, on which they analyzed the results obtained in the tests Saber 11. They made a characterization of those students using factorial multivariate methods. They found that the students with the highest risk in mathematics and natural sciences are from strata and low family income, with family nuclei between 4 and 6 siblings, they do not have their own home and come from outside of Santander.
Osma et al. [6], conducted a study with the results of the Saber Pro 2010 test in students of last semesters of civil engineering at the national level, in order to determine the factors that influence the scores obtained. From a correspondence analysis they concluded that the above-average scores are associated with students of the male gender who study in public institutions with parents who have university studies and live in social strata 4, 5 or 6.
Bahamón and Reyes [7], led a study based on the scores reported by the ICFES in 2012 as well as the data reported by the psychology program of a university about the intellectual capacity, sociodemographic and academic characteristics of 68 students who were in the ninth and tenth semesters of the undergraduate. This research obtained as results that students with better performance have specific sociodemographic characteristics: high intellectual capacity, good academic performance, professional interests related to career areas and adequate study habits and techniques.
Gómez [8] carried out a documentary review where an analysis was made about the current focus on educational quality, in which he identified ten problems and proposed new conceptual alternatives. Sánchez [9] carried out a descriptive analysis of the results of the Saber Pro tests at the Universidad Pedagógica y Tecnológica de Colombia during the period 2012-2015, where he highlights that programs such as medicine, law, civil engineering, are those that register higher average scores in the generic competences.

Method
This study has the average scores obtained by the students of the UFPS, of the branches San José de Cúcuta and Ocaña, Norte de Santander, Colombia, evaluated in the year 2017, which constitutes the study of the target population. To describe the performance of the programs in each area evaluated, univariate and multivariate statistical exploratory methods were applied. A principal components analysis was carried out with the purpose of characterizing and analysing differences and similarities among the careers, by means of the average scores obtained in the different areas. The statistical packages SPAD 5.6 and SPSS 24 were used.

Results and discussion
In this section the results of the statistical analyzes are presented. Table 1 lists all the academic programs of the UFPS by faculty and indicates the corresponding label that was used in the analyzes.  Table 2 and in the box diagrams of Figure 1, descriptive indicators of the overall results of the UFPS can be observed. It is appreciated that quantitative reasoning and written communication are the areas with the highest average scores, followed by critical reading and citizenship competencies, while the English area has the lowest average. In the quantitative reasoning test, the average scores of the programs reflected greater variability, although the other areas evaluated also show considerable variability. This means that there are appreciable differences between the scores of the programs within the same area. You can also see that in the quantitative reasoning test the highest average score was obtained (184), while the lowest was written communication (123). In Figure 1, the average scores obtained by some programs that move away from the rest of the careers draws attention. in the area of written communication, the averages of greater magnitude and away from the rest of the programs were the following careers attached to the faculty of education, arts and humanities, San José de Cúcuta headquarters: basic education (art), humanities and social sciences and law. while the averages considerably lower were in: mathematics-computing (faculty of education, arts and humanities, San José de Cúcuta headquarters) and zootechnics (faculty of agricultural sciences and Environment, Ocaña headquarters). In the critical reading test, the law program (faculty of education, arts and humanities, San José de Cúcuta campus) stands out as the highest average score.
When comparing the data according to the headquarters where the program operates (Figure 2), it is generally appreciated that the medians of each type of test are greater in San José de Cúcuta, Colombia. At the Ocaña campus, a comparatively very low value of its median stands out in the English area, however, the systems engineering program obtained the highest average score and very distant from the rest of the careers (similar to the high scores obtained by programs of the San José de Cúcuta headquarters).  In relation to the comparison by faculty, it is observed in Figure 3, that in a general way, the programs assigned to the faculties of engineering and health sciences obtained better averages than the programs of the rest of the faculties. The faculty of health sciences has only the nursing program ascribed; therefore, no dispersion is observed in the respective box diagrams? the faculty of business sciences shows comparatively low median values. The faculty of education, arts and humanities, is the one that presents the greatest variability for each of the tests.

Principal component analysis
A standard PCA was carried out, although the scores obtained in the Saber Pro test are expressed in the same units of measure, it is desired to compare the programs of the UFPS giving the same importance to all the areas evaluated. If the non-normalized analysis were carried out, it would imply giving more importance to the areas with the greatest variability [10]. As active variables, the averages in each of the tests were considered and as supplementary variables both the venues and the faculties were used.
According to Table 3, the first two main components capture 84.6% of the total variability [11]. While the first three components accumulate 94%. How many components to retain is a critical question in this type of analysis [12]. One of the rules used is to select those whose eigenvalue is greater than 1 (in standardized analyzes). In this case, that would entail retaining the first two components.  Figure 4 shows the biplot of the first two components. In blue, the programs (individuals) are represented, in green the active variables while in pink and dark red the supplementary modalities. This first factorial level explains 84.6% of the total variability of the data. The quality of representation of the areas evaluated (variables) in that plane is as follows (see Table 3): written communication 90%, quantitative reasoning 89%, critical reading 88%, citizenship competencies 82% and English 75%. All the tests are very well represented in the plane conformed by the first two main components.
The first major component orders the academic programs according to their global average in the five tests. In this way, programs that are located to the right of the factor 1 are the ones that obtained the best averages and to the left the programs with lower averages, while near the center of gravity of the point cloud are the programs with similar averages to the global average of all the UFPS careers. Thus, it is observed in this biplot that among the programs that stand out with the best averages in all the tests are EAH-DerC, EAH-CSoC, ENG-IsiC, EAH-ArqC, ENG-IInC, CS-EnfC, CAA-IAmC, ENG-ICiO, ENG-ISiO. The programs with lower averages are: CAA-ZooO, EAH-InfC, CAA-IPeC, CE-AEmO, CE-CPuO, CE-AEMC, among others.
When projecting the supplementary modalities of the faculty variable on the factor 1, it can be observed that the faculties whose programs present the best averages in the Saber Pro Test are the faculty of health sciences and the faculty of engineering, while the faculty of business sciences and the faculty of agrarian and environmental sciences have attached programs with averages lower than the overall of the UFPS. The faculty of education, arts and humanities, is located very close to the global average of the university, however, product of the dispersion that was previously identified in the exploratory analysis of the data (see Figure 3), some of their careers are between those with the best average in the UFPS and others among the lowest in all the areas evaluated. It can also be seen in Figure 4, that the san José de Cúcuta headquarters tends to present better averages in its academic programs than those of the university's headquarters in Ocaña, Colombia. However, the civil engineering programs (ENG-ICiO) and systems engineering (ENG-ISiO) of the Ocaña headquarters are among the UFPS programs with the best performance in the Saber Pro test.
Regarding the second main component, Figure 4 shows the opposition between the tests of quantitative reasoning and written communication. In this way, towards the top of factor 2 are located

Conclusions
The characterization of the results achieved by the academic programs of the UFPS in the Saber Pro test, allow a comparison to be made internally. Globally, greater strengths were found in the areas of mathematical reasoning and written communication, while the greatest weakness in English. The careers of the faculty of engineering stand out among the best averages of the UFPS in all the areas of the Saber Pro test, excelling in quantitative reasoning. All the programs of the faculty of business sciences presented averages below the global average of the university in all the areas, also the careers of the faculty of agricultural and environmental sciences except for the environmental engineering program. The faculty of education, arts and humanities has affiliated programs whose performance was among the best in the Saber Pro test, however, some of their careers had a performance similar to the general average of the university and others with results among the poorest.
There was a tendency for the programs of the faculty of engineering to show greater strength towards quantitative reasoning than towards written communication and, in contrast, there was also a tendency for programs attached to the faculty of education, art and humanities to show greater strengths in written communication than in mathematical reasoning. It is recommended to conduct studies with the information of the characteristics of each student and the result in all the areas evaluated. This will allow obtaining a more detailed diagnosis. It also seems very interesting to compare the results obtained by each student in the state test for admission to higher education with those of the Saber Pro test. This would make it feasible to carry out analyses to study the predictive ability of the entrance test in the subsequent performance of the student in the university.