Design and validation of a Scale of Attitudes towards Academic Feedback (SAAF)

: Feedback is a type of formative evaluation linked to student learning and academic achievement. However, it is not known how students perceive this process since there are no instruments to measure attitudes towards feedback. The objective of this work was to develop and validate an attitude scale towards academic feedback through an Exploratory Factor Analysis (EFA), using the principal components method and an internal consistency analysis using Cronbach's Alpha. The sample consisted of 274 students from 2015, 2016, and 2017 cohorts of pedagogy careers from five Chilean universities. The results showed the consolidation of the instrument and its final conformation in four components (Negative experience towards feedback, Effectiveness of feedback, Feedback as an opportunity, and Resistance towards feedback) with a total of 15 items, which allow measuring the students' attitude towards this written formative evaluation.


Introduction
Professors working in the university context inevitably give opinions about student performance, offer suggestions, ask questions, give orders, and so on. Offering feedback on the performance of students seeks to strengthen their training processes. However, despite the frequency with which this process is carried out, few studies focus on how feedback is provided, what are the most efficient ways to do so, what effects and implications it has for students, and, therefore, what type of learner is promoted. As confirmed by Smith [1] and Martinez Rizo [2], teachers rarely receive training on how to provide feedback on student performance in their jobs. Given this situation, it is necessary to investigate how students perceive teacher feedback to know if it is effective during the training development of future professionals.
Considering the above, the measurement of university students' attitudes has been a topic of research interest developed over several decades in different disciplines, including educational assessment [3][4][5]. However, attitudes towards specific evaluation processes such as feedback have been very little researched, and therefore, there are no adequate instruments to measure different factors of attitudes towards feedback reliably and validly. In this study, an attitude scale towards academic feedback was developed and validated, containing four dimensions: Negative experience towards feedback, Effectiveness of feedback, Feedback as an opportunity, and Resistance towards feedback, which constitute the first validated instrument that exclusively measures the attitude towards academic feedback.
When referring to formative evaluation, Sadler [6] identifies different criteria for making judgments about the quality of students' responses to a task, where feedback is a key element. According to this consideration, feedback informs the student and the teacher about a task developed over time, allowing changes to be made to achieve improvements by contributing to the evaluation [7,8]. When feedback allows for the evaluation of what students have learned, it promotes metacognition, autonomy, and self-regulation in learning. In this way, it helps the student to better understand the goal in a given task, the state of their achievements concerning that goal, and ways to bridge the gap between their current state and the desired state [9].
These characteristics of a new training environment allow the development of a climate for the expression of ideas, decision making, and alternative evaluations Gilmore [10]. Associating relationships of trust with student feedback and attitude would allow for individual or group performance that enhances learning at all educational levels because students expect their achievements to be replicable in their professional performance and allow them to achieve greater success thanks to a positive reception of the interventions made by the teachers who support them in their academic work.
Assessment is not a one-sided process, nor is it alien to the expectations and emotions of students. Some authors consider that evaluation can condition what and how a student learns, which would imply that leaving aside what the student expects, could harm the end of a process that seeks to support and generate substantial changes in the integration of knowledge [11][12][13][14][15]. What is relevant then is to understand that intervening in the actions of students through the corrective processes of their academic work must consider, on the part of the teacher, a series of strategies that allow young people to understand why they are being questioned.
In the area of teacher training or Initial Teacher Training (ITT), as it is known in Chile, the above is of substantial value since it is relevant to understand and comprehend how future teachers deal with and effectively recognize the feedback provided by teachers throughout their training process since up to now we only know the effect of feedback on learning, but we do not know what students think about it. Although instruments exist to measure the concept of attitude, these are directed at the predisposition of students toward a particular discipline. This is the case of the Questionnaire to assess the attitudes of students in Obligatory Secondary Education (OSE) towards mathematics [16]; the Scale of Attitudes of University Students towards Academic Tutoring [17] and Attitudes towards the inclusion of students with disabilities in physical education questionnaire (AISDPE) [18], among others.
In the area of feedback, studies show the application of instruments that consider feedback as one of their dimensions or items within the general dimensions or categories and not as the central objective of the instrument. Thus, the Learning Assessment Scale (LES) [19] has the central objective of determining assessment practice in university teaching but from a learning-oriented assessment approach. For its part, the instrument created for the Evaluation of Teacher Performance based on competence training sought to evaluate such performance from the perspective of students [20]. In this case, only within the dimension called Didactics, two items can be seen that are directly related to the concept of feedback: indicator 15, called Specific feedback to each student on evaluation, and indicator 22, which refers to providing continuous feedback to participants as a form of formative evaluation during the sessions.
Concerning students' reactions to feedback, studies have shown that students prefer clear and understandable comments that are focused on the final objective of the writing and that indicates how to improve writing in terms of what needs to be done [21]. Ferris [22] in his research on the teaching of English as a second language mentions the positive behavior of students about comments, classified by the author as marginal and final. Its main characteristic is that the former tend to be questioning and shorter, while the latter is more general and positive related to the grammatical dimension of the text.
In this sense, Duijnhouwer [23] in an experimental study on feedback and motivation postulated that students would improve the quality of their texts if they received written comments that showed them directly how to improve the text. The results also revealed the existence of factors external to written comments that influence the reception of teachers' interventions, for example, students with low self-esteem would better receive negative comments and vice versa.
According to the above, the instruments mentioned that today are found in the specialized literature cannot efficiently measure student behavior in all situations and academic contexts. Most of the instruments that measure attitudes do not consider the association of this concept with formative assessment, specifically, with written feedback.
Considering how relevant it is to integrate attitudinal and formative aspects in the process of academic feedback, this article reports on the creation and validation of a Scale of Attitudes towards Academic Feedback (SAAF) in students in Initial Teacher Training (ITT). With this, it is hoped to advance in the knowledge of both the feedback as a type of formative evaluation and the causes that could explain its effectiveness in contexts.

Participants
The sample included 274 students of Pedagogy (82 men, 191 women, and 1 subject without gender definition). These students belong to different majors, such as Primary School Education Pedagogy, Pre-School Education Pedagogy, Secondary School Language and Communication Pedagogy, Pedagogy in Physical Education, Pedagogy in History and Geography, Bachelor of Education/Philosophy Pedagogy, and Secondary School Pedagogy in English, belonging to cohorts 2015, 2016 and 2017.
The students come from five public and private universities belonging to the Council of Rectors of Chilean Universities (CRUCH in Spanish). The number of participants for this face of the study responds to a little more than 10 subjects per item, considering this application as the first version of the Scale. The size of this sample is within the recommended range for a factorial analysis, since several authors have indicated that between 5 and 20 participants per item are advisable [24][25][26][27]. Furthermore, authors such as Kass and Tinsley [28] also mention that the parameters of the test tend to be stable regardless of the participant/item relationship.

Instrument
The Scale of Attitude Towards Academic Feedback (SAAF) seeks to show how students deal with feedback in the form of written comments on academic papers. The dimensions considered in the first version of the creation of the instrument were based on theoretical aspects studied by Brinko [29], specifically, criteria that the author directs towards the What, How, Where, When and To Whom feedback is given.
1. Disposition, associated with how an individual faces feedback or the state of mind before a given task, after having received written comments; 2. Knowledge, understood as what is known about feedback or recognition of this phenomenon in the assessment process, its identification; 3. Belief, understood as the preconceptions about feedback, what each individual identifies independently of the experience or knowledge he or she has of the term, and finally, 4. Timing, associated with the moment in which the feedback occurs, concerning the delivery of written comments to previous or subsequent works.
The design of the first version of the instrument was made up of 25 items whose answers were obtained based on a Likert scale of 7 options, where 1 is Totally Disagree and 7 is Totally Agree.

Procedure
The application of the scale was focused on the sample of students in Pedagogy careers from five higher education campuses mentioned above, to analyze the behavior of student responses and the final shape of the Scale. The response time did not exceed 20 minutes. To safeguard ethical aspects, all participants agreed to respond, after signing an informed consent form that gave all the characteristics of the procedure to the participating students, making it clear that the data would be treated confidentially.
After piloting the Scale, an Exploratory Factor Analysis (EFA) was performed, using the principal components method, which explains the total variance among the variables. This technique is the most frequently applied in studies related to the development and validation of tests when there are no previous hypotheses about the possible dimensions of a construct. Prince [30] suggests that EFA is a method that can be used to compare factors and factor loads in cultural settings. Welkenhuysen-Gybels and Van de Vijver [31] consider it a simple practice to assess construct equivalence. Its use has also been considered in research to assess the construct validity of scales that have been translated and/or adapted to different languages for application in specific contexts [32][33].
This analysis made it possible to define the number of items and dimensions that allowed the identification of the students' attitude after the experience of having received feedback through written comments. Finally, an internal consistency analysis was considered using Cronbach's Alpha, a method used to verify the reliability of tests, scales, or quizzes, which are made up of a set of items or reagents that are expected to measure the same attribute or content field. The main advantage of this method is that it requires only one administration of the test, coinciding with the application mentioned above.

Results
The data obtained with the application of the instrument were analyzed with the use of the statistical software SPSS v.23. The validation of the construct was carried out through an Exploratory Factor Analysis (EFA), to verify the internal structure of the scale allowing the identification and grouping of items that are strongly correlated to each other [34]. Once the factors that make up the construct were obtained, reliability was obtained using Cronbach's Alpha.
The first stage of this validation consisted of applying a normality test with which it was possible to identify a group of items whose behavior was outside the normality of the remaining instrument. All these items (2, 3, 12, and 21) presented indices of kurtosis and asymmetry higher than 3.25, far above the ideal of values that should fluctuate between -1,000 and 1,000 [35]. Therefore, it was decided to eliminate these 4 items and repeat the analysis to meet acceptance criteria and ensure the best possible sample behavior, before EFA.
Subsequently, the main component analysis method was carried out with varimax rotation. The Kaiser-Meyer-Olkin test (KMO) (0.809) and Bartlett test indices (p < 0.000) indicated that the model was adequate, so we proceeded to extract factors, which could explain 59.70% of the variance, showing that the decision of the previous extraction did not compromise a significant weight.
Although the values of this first model presented figures close to what was expected, it was decided to eliminate items 18 and 25, as they had factor loads lower than 0.45, which is relatively low. Then a second EFA was carried out, which presented positive values, since it was possible to increase the percentage of variance to 60.35% and the reliability, through Cronbach's Alpha had an overall value of 0.77. However, when investigating the value by dimension, two of these showed relatively low values (0.57 and -0.33). Due to this result, it was decided to restructure the analysis of main components by eliminating one of the items (8) of dimension 3 with a medium value (0.57) to try to force the regrouping of items in a smaller number of dimensions.
The new EFA managed to show greater clarity of content with a greater correlation in the 4 dimensions, allowing a large number of indicators of little theoretical significance to be replaced by a smaller number of significant conceptual variables [36]. This final analysis showed a KMO index of 0.81, explaining the variance of 63%, with a Cronbach's Alpha, for the final scale with a value of 0.77 for a total of 15 items. The items finally eliminated were 2, 3, 7, 8, 12, 14, 18, 19, 21 and 25, while those that were part of the final scale were items 1,4,5,6,9,10,11,13,15,16,17,20,22,23,24. See Table 1.  (15,9,16) with a reliability index of 0.812; the same number of items for dimension 3 called Feedback as an opportunity (5, 1, 20) whose Cronbach's Alpha was 0.709 and finally, dimension 4 called Resistance to feedback (10,11,23) with a reliability value of 0.706. See Table 2.

Discussion
The main objective of this study was to design and validate a scale to find out the attitude of students towards written feedback from their teachers during the development of academic work. The creation of this instrument was carried out because currently, no scale can measure students' attitudes by associating the process of academic feedback with this response.
The results obtained show that the instrument created presents good levels of construct validity and reliability for each dimension (factor 1: 0.779; factor 2: 0.812; factor 3: 0.709 and factor 4: 0.709), and for the instrument in general (0.77). Optimal indexes for the internal structure of the scale [34], achieving a correlation and several significant variables in the 4 dimensions.
Given the appropriate adjustment indices of the instrument, its application can be guaranteed to determine how the attitude towards the written feedback process is in initial teacher training. The EFA showed that the four dimensions of the Scale of Attitude Towards Academic Feedback (SAAF), in their final form, measure what is expected and can be applied in diverse and general educational contexts.
Among the most relevant changes when comparing the initial version of the instrument and the scale finally validated was the need to rename the four final dimensions. Dimension 1, called Disposition, which is thought of as how the student faces the feedback or the state of mind that the student manifests when faced with a certain task, after having received written comments from a teacher, was renamed Negative Experience towards Feedback, grouping together a total of 6 items whose main characteristic is their reference to a negative predisposition as a result of the students' experience in the process.
The second dimension called Knowledge, understood as what the student knows about feedback or recognizes from this phenomenon in the assessment process and its identification, was named Effectiveness of feedback, grouping 3 items whose reference refers to the recognition of the contribution that the feedback process has in the development of training as a student, whatever the area of training.
The third dimension was originally named Belief, being understood as the preconceptions about Feedback, what each student would identify from the process independent of experience or knowledge of the term. Finally, this factor was defined as Feedback as an opportunity, also with 3 items grouped and focused on how the student recognizes the contribution of the feedback in his/her formation.
As the fourth and last dimension, the EFA allowed regrouping 3 items by changing the name from Temporality, recognizing it as the moment when feedback occurs concerning submissions of written comments to previous or subsequent works, to the new dimension renamed as Resistance to feedback that focuses on students' frustration during the process according to the way their teachers perform this task.

Conclusions
The validated Scale of Attitude Towards Academic Feedback (SAAF) can be applied in broad and generic educational contexts. This can be stated because the dimensions generated allow: a) identifying potential negative experiences that students have lived in their formative process. These experiences could generate relevant changes in the attitude when receiving new feedback; b) obtaining data related to positive expectations in the feedback allowing students to recognize the contribution of this process in their academic formation; c) knowing if the students understand the process carried out by their teachers and their concrete contribution to the performance of a given task, and finally, d) understanding if the students resist the process (i.e. they participate in it but do not recognize their potential to improve in their academic performance).
The use of this instrument enables teachers who use the feedback technique through written comments to gather relevant data to understand why their comments or suggestions are accepted, rejected or ignored by their students. This is particularly relevant when teachers spend a lot of time providing feedback, trying to motivate their students and contributing to the writing process, for example, by pointing out important aspects of the text quality and giving suggestions about the content [23].
Research on this kind of topic is an important contribution to improve teaching performance in general, providing information to teachers to enhance their practice with formative assessment. As mentioned by Orrell [37], teachers are mainly practice-oriented, which allows them to generate a grade that reflects grammatical, conceptual, and spelling errors, which translates into a minimum commitment to the construction of ideas by students.
The application of this instrument in different universities in Chile and other Spanish-speaking countries is highly recommended since it specifically associates the level of relationship between the attitude of students and the reception of feedback through written comments. In addition to considering the potential effectiveness of such a training process in achieving and enhancing student performance in a given academic activity, it should be mentioned that its application in contexts other than the field of initial teacher training necessarily requires a new validation process. Even so, this instrument represents an important reference for the development of any scale that seeks to measure students' attitudes towards academic feedback.
Limitations: Future studies should include an adaptation of the scale to the virtual environment due to the migration of teaching and learning that occurred as a result of the advent of COVID-19. The