A literature review on the affective factors that in ﬂ uence data-driven decision-making

Data-driven decision-making (DDDM) has been playing an increasing role in contemporary teaching, since it includes systematic collection, analysis, and application of data to improve students ’ educational performance. However, little is known about the affective factors that in ﬂ uence this data-based practice. Thus, the purpose of this study was to systematically examine previous research on the affective factors that in ﬂ uence DDDM based on the following criteria: (1) the level of DDDM usage; (2) the emphasis of affective factors; and (3) the nature of the interventions and their effects on teachers. According to the ﬁ ndings, this literature review showed how little DDDM-related affective factors have been researched, even though the knowledge of DDDM can help expand its application in the educational ﬁ eld. For example, although the most widely used tool is the Data-Driven Decision-Making Ef ﬁ cacy and Anxiety Inventory (3D-MEA), which has shown promising results in terms of measuring the ef ﬁ cacy and development of data literacy, other affective components have yet to be tested, due to their novelty in the ﬁ eld. The implication of the ﬁ ndings is that obtaining more information about DDDM and its affective elements can help reduce teachers ’ anxiety toward this approach and ultimately enhance the overall educational process.


INTRODUCTION/BACKGROUND
Data-driven decision-making (DDDM) has been playing an increasing role in contemporary teaching, since it includes conscious, systematic data collection, analysis, and application to enhance students' performance and support educational matters (Marsh, Pane, & Hamilton, 2006). For example, DDDM-related tools are used to measure students' diagnostic, formative, and summative knowledge in national competency tests. A similar system is the academically certified eDia (Csap o & Moln ar, 2019). This Hungarian diagnostic support tool provides immediate feedback to teachers about their students' performance, which can be used to enhance teaching, learning, and pedagogical planning. Teachers can also collect the academic results from each subject and compare them with previous ones in order to form their lesson plans accordingly (B us, 2015;Means, Padilla, & Gallagher, 2010). Besides such measuring systems, examinations also have a significant effect on teaching and learning processes. However, although examinations are a natural part of the pedagogical process (Vigh, 2007), DDDM can help create a different approach to measurement data.
In order to implement this in accordance with DDDM, it is important to provide valid and quality data for teachers (Ronka, Geier, & Marciniak, 2010;Sz ell, 2015). It also crucial that teachers have a positive attitude toward data usage and measuring programs, since any anxiety regarding these factors ultimately affect their classroom activities (Firestone et al., 2002;T oth, 2011. However, our knowledge about the affective factors that influence DDDM is scarce. Meanwhile, the application of DDDM in classroom practices is influenced by the different characteristics of teachers. According to Dunn, Airola, Lo, and Garrison (2013), teachers' impressions about their effectiveness and anxiety can be the main factors that strengthen or weaken their dedication toward DDDM.
Provided that teachers perform their tasks in accordance with this approach, it is assumed that they see themselves as efficient in the usage and analysis of measurement information during their work. Conversely, possible reasons behind the lack of systematic usage of measurement information can include: having anxiety when working with data and statistical analyses (Samuel, 2008); feeling inefficient at work or possessing limited training and experience in data analysis, usage, and collection in the given field (Dunn et al., 2013); and working in a nonsupportive institutional environment (Ronka et al., 2010).

PURPOSE
The purpose of this study was to examine previous research on the affective factors that influence DDDM. The main focus was on the affective aspects influencing DDDM (RQ1) in the literature and the measurement tools used to inspect such aspects (RQ2). It is hoped that the findings of this literature review on DDDM-related affective factors can contribute to the understanding and application of DDDM and be used to determine future research directions.

METHODS
In the first phase, DDDM-related books and peer-reviewed articles were collected through electronic databases (e.g., ERIC, Science Direct, and Google Scholar) and filtered based on the condition that they approached the topic empirically. The temporal scope of the literature review was between 2010 and 2020. The following keywords were used: teacher beliefs on DDDM, affective factors of DDDM, engagement to DDDM. In the second phase, the condition was the linkage to practical applications and affective factors. Overall, 25 publications were selected and categorized into two groups: the first group, which focused on the affective elements of DDDM; and the second group, which focused on the empirical analyses of teachers' data literacy and intervention-related affective elements. Moreover, the criteria in this study were as follows: (1) the level of DDDM usage (N 5 8); (2) the emphasis of affective factors (N 5 7); and (3) the nature of the interventions and their effects on teachers (N 5 10).

RESULTS
Based on the systematic review of the DDDM-related literature, a significant aspect was the distinction between teachers' effectiveness and their perceived level of efficacy. In this regard, effectiveness refers to the self-reflective, future-oriented supposition in which individuals possess the required knowledge and ability to successfully perform certain tasks. However, in terms of teachers, it applies to the influential factor on students' educational performance (Bandura, 1997). As for teachers' sense of efficacy, it is possible to obtain a clearer picture about how they evaluate their own DDDM-related efficacy. For example, provided that teachers' perceived sense of efficacy is high, students are positively affected in terms of the results (Dunn et al., 2013). Thus, the higher the DDDM-related sense of efficacy, the more that teachers effectively use DDDM to enhance their teaching and foster their students' knowledge. The effectiveness of DDDM can also be increased through DDDM-related interventions, as shown in previous research on their effects on teachers and teacher trainees (Green, Schmitt-Wilson, & Versland, 2016;Reeves & Chiang, 2018). In the next chapter, I would highlight some researches in systematic order, by presenting some great examples and the most important methods of measuring affective factors of DDDM.

DDDM-related efficacy and anxiety
According to the results, one of the heavily focused areas was teachers' self-perceptions of their data usage and efficiency. In this regard, DDDM-related sense of efficacy refers to the dedication and actions necessary for carrying out successful interventions (Airola & Dunn, 2011). Such efficacy also influences teachers' aims, perseverance, and motivation. According to previous studies, if a teacher believes that he/she will be successful in the implementation of a task, then the task itself might trigger positive emotional reactions, make them more open to try new practices (Straub, 2009), and be more resilient when tackling other task-related problems (Bruce, Esmonde, Ross, Dooki, & Beatty, 2010). In general, the starting point of these efficacy-related studies was teachers' self-perceptions of data usage.
It is also important to note that one of the influencing factors regarding the sense of efficacy was anxiety (Bandura, 1997). In this case, the higher the level of anxiety, the less effective the teacher feels. Meanwhile, the expectation that teachers are required to understand and apply DDDM can also raise their stress level, which, in turn, can increase their feeling of incompetence and negatively affect their application of this approach. However, there are limited resources about the affective aspects of DDDM, even though DDDM-related efficacy and anxiety have a significant influence on its application.
As for teachers' sense of efficacy and anxiety, Dunn et al. (2013) aimed to optimize such aspects by creating the well-established Data-Driven Decision-Making Efficacy and Anxiety Inventory (3D-MEA). Before this complex measurement tool was introduced, questionnaires were used as the basis for confirmatory factor analyses (Walker, Reeves, & Smith, 2018) or mapping DDDM-related opinions from teachers and trainees (Green et al., 2016;Price, 2018;Reeves & Chiang, 2017. The overall purpose of Dunn et al.'s (2013) study was to validate the 3D-MEA. In order to implement it, a sample of elementary and high school teachers (n 5 1,728) were asked to complete a questionnaire, the results of which underwent explorative and confirmatory factor analyses. The study confirmed the validity of the questionnaire in which four DDDM efficacy subscales and one anxiety subscale were identified. In this case, the four subscales included data access and detection, technology use, understanding data, and applying data, while the feeling of anxiety itself was one subscale. However, their study did not focus on subject-related differences. The given questionnaire was also used in the study by Price (2018), which focused on the correlation between data-informed school leadership (DISL) and DDDM-related efficacy and anxiety among a sample of teachers (n 5 300). The two main measuring tools in this study were the DISL questionnaire and the 3D-MEA. In particular, the DISL questionnaire included data-related goal setting, development of evidence-based decision-making among teachers, educational programs, and the establishment of a data-rich culture. According to the results, high DISL is related with a high level of DDDM-related efficacy, while there was no significant correlation with DDDMrelated anxiety. Consequently, while the head of institutions tend to consider their work to be data-driven, teachers feel more effective in the field of DDDM.

The effect of DDDM-based interventions on individuals
Based on the literature review in the present study, DDDM-based interventions proved to be a well-researched field. The teachers in the United States who did not participate in special training had limited knowledge about the elements of data literacy (e.g., statistical knowledge, different data systems and their usage, etc.) and relied less on the implementation of DDDM (Means, Chen, DeBarger, & Padilla, 2011;Reeves & Chiang, 2018;Walker et al., 2018). Moreover, there was no strong correlation between their data use and their educational decisions (Dunn et al., 2013).
In related research, DDDM interventions were carried out in different levels of educational systems such as elementary school teachers (Reeves & Chiang, 2018) and teacher trainees (Reeves & Honig, 2015). They also focused on teachers' knowledge of data usage and their abilities and practices, rather than institutions and their learning support. In the present study, it was possible to identify the common conditions of DDDM-based interventions, including their application and effects on individuals as well as their characteristics (e.g., capacity, data properties, and the effects of leadership and organizational culture). Overall, the purpose of the interventions was to develop teachers' DDDM-related knowledge and abilities, and to increase their feeling of satisfaction. In the following, DDDM-related abilities and knowledge are used to determine data literacy (Gummer & Mandinach, 2015).
It is also possible to get an inside view of teachers' perceptions of DDDM and determine how such perceptions change after interventions are performed. In this regard, interventions can enable teachers to gain important knowledge from data that can be used to support certain educational decisions (e.g., differentiation). Nevertheless, the majority of the intervention-focused studies were only based on self-assessment questionnaires and post-training measurement tools (e.g., students' performance), instead of classroom observations and other correlations (Marsh, 2012). For example, some researchers introduced a DDDM system to the participants, performed data collection, analyses, and application training, and administered a satisfaction survey.
In other studies, different types of trainings were found, including one-to two-day trainings (e.g., Green et al., 2016;Reeves & Chiang, 2017), online trainings (e.g., Reeves & Chiang, 2018), and one-year trainings (Van Geel, Keuning, Visscher, & Fox, 2017). For instance, Green et al. (2016) organized data literacy training seminars (n 5 16; 3 3 0.5 days) in order to improve teachers' analysis skills and cooperative practices. University summer camps were also organized (n 5 30) to enhance teachers' data collection skills and statistical knowledge, after which the 3D-MEA and the 3-2-1 formative measuring tool were used. This tool is an activity which encourages participants to reflect on their learnings, giving examples and recognize the unresolved areas. According to the results, the participants eventually recognized the value of the data, especially from the aspect of decision-making in teaching. In the research by Reeves and Chiang (2017), 58 teacher trainees completed a one-day intervention to analyze external data and support their teaching activities based on such data. In this case, the 3D-MEA and a questionnaire regarding the effectiveness of the intervention were used. Overall, the results showed positive changes in the following 3D-MEA subscales: data access and identification, interpretation, and application.
In a related study by Reeves and Chiang (2018), an online data literacy prevention program and changes in the application of teaching materials were monitored. In the latter, there was no significant added value, but a positive change was detected in the DDDM-related sense of efficacy and anxiety among the teacher trainees (n 5 99) and practicing teachers (n 5 25). Moreover, the teacher trainees' attitudes became more positive toward pedagogical evaluations. In this case, the online intervention contained three modules in which the participants received the following information: the aim and the characteristics of the program; the data usage in individual, group, and class levels; and the data usage and grading in class and school levels. Then, they were asked to identify and determine the weak/strong points, the extent of development, and the application in education on the given level. Overall, a total of 7.5 hours was spent in the training. As for the measuring tools, they included the 3D-MEA and a questionnaire about data usage habits, attitudes toward pedagogical evaluations, and the efficiency of the intervention.
In other research, Reeves and Honig (2015) conducted a six-hour intervention among a sample of teacher trainees (n 5 64), after which the data analysis, interpretation, classroom evaluation, and decision-making were measured. The attitude toward data literacy was also a subject of focus. Regarding the measurement tools, they included the Conceptions of Assessment III and Survey of Educator Data Use, which focused on the participants' attitudes and beliefs. Based on the results, increased data literacy and a positive attitude toward data usage was found. In Van Geel et al. (2017) two-year study, they inspected a large sample of teachers (1,182 teachers from 83 schools) and their data literacy. In this case, the measuring tool contained general data literacy-related items and system-specific questions in order to interpret the results. Their study not only found a significant change in these two areas, but it also narrowed the gap between the data literacy of teachers with college and university degrees.
The scope of inspection can be extended to not only practicing teachers' and teacher trainees' intervention programs (Ebbeler, Poortman, Schildkmap, & Pieters, 2016), but also to headmasters and external consultants (Begin, 2018), by using the Data Team Procedure. This procedure is an eight-step, scientific-based process that leads team members (e.g., teachers, headmasters, data experts, etc.) to identify the problems in educational development and make improvements in students' performance. Other problems can include early school leaving, poor academic results in a subject, and decreasing efficiency of an institution. Moreover, this procedure provides data users with the ability to move easily between the various steps of DDDM. According to previous studies, since it builds on the inner capacities of schools, it can be an efficient way to deal with reoccurring problems (Bolhuis, 2017;Schildkamp et al., 2018).
In sum, both short-and long-term intervention programs were primarily used in the literature to focus on DDDM-related efficacy and anxiety as well as the development of data literacy. The 3D-MEA and satisfaction surveys were also used to interpret the training-related efficacy and determine the level of data usage satisfaction.

CONCLUSION
The present study examined the affective aspects influencing DDDM (RQ1) in the literature and the measurement tools used to inspect such aspects (RQ2). For this purpose, the main criteria included: (1) the level of DDDM usage; (2) the emphasis of affective factors; and (3) the nature of the interventions and their effects on teachers. By highlighting the most important researches and methods this literature review also showed how little DDDM-related affective factors have been researched, even though the knowledge of DDDM can help expand its application in the educational field. Self-perception of sense of teacher's work as one of the most influencing factor has been revealed. For example, the most widely used tool to track teachers' DDDM-related sense of efficacy and data literacy was the 3D-MEA. In fact, the teachers with a higher level of efficacy were more willing to use this data-based practice (Dunn et al., 2013). The second affective factor was attitude toward data teachers' own data literacy. This important DDDM-related approach was at interventions. The third one was used to determine their intervention-related satisfaction after pre-and post-evaluations. However, other affective components have yet to be tested, due to their novelty in the field. Thus, obtaining more information about DDDM and its affective elements can help reduce teachers' anxiety toward this approach and ultimately enhance the overall educational process. Moreover, along with measuring the sense of efficacy and anxiety, it would be advisable to conduct studies on teachers' systematic thinking and even motivation toward DDDM application, professional development, and information processes. Mapping these affective elements can also provide a better understanding of the conditions, possibilities, and limits of DDDM application.