Eﬀects of Scrum methodology on students’ critical scientific literacy: the case of Green Chemistry

Secondary science education plays a key role in students’ process to become scientifically literate citizens. However, teaching students to acquire the necessary skills and knowledge to deal with complex societal issues is challenging. This paper reports about a study in which Scrum – a methodology to manage complex projects – was implemented in secondary chemistry classrooms to increase students’ conceptual understanding as well as their critical scientific literacy. A quasi-experimental design was used with 198 Grade 11 students from eight different classes. The experimental condition (99 students, 4 classes, 25 groups with 3 or 4 students, 2 teachers) used Scrum methodology during a context-based course on Green Chemistry. The comparison condition (99 students, 4 classes, 29 groups of 3 or 4 students, 3 teachers) completed the same module about Green Chemistry, without using Scrum methodology. At the end of the course students formulated a written advice on the greenest synthesis of adipic acid. A pre-test on prior knowledge of Green Chemistry principles and a post-test on conceptual understanding of the chemistry concepts involved were administered. In addition, the Standard Observed Learning Outcomes taxonomy (SOLO) was used to analyse the quality of the written advices as a measure for students’ critical scientific literacy. Students from the experimental condition outperformed their peers from the comparison condition in their conceptual understanding. Moreover, the quality of the advices of students from the experimental condition were rated higher than the quality of advices of students in the comparison condition. These findings are discussed and connected to Scrum methodology as teaching approach to scaffold both students’ conceptual understanding and its potential to promote the development of their critical scientific literacy.


Introduction
An important goal of secondary science education is to promote students' competences to become scientifically literate citizens. Secondary education enables students to engage with sciencerelated dilemmas that play a role in their personal lives (Eilks and Rauch, 2012). This is of the highest importance as society faces a broad variety of challenges (UN, 2015), including ecological issues such as climate change and chemicalisation (Ekberg, 2007;Sjöström et al., 2016). Inevitably, students, need to learn how to discuss dilemmas related to science, industry and the environment. Therefore, there is a clear need for students to acquire substantial science knowledge to understand underlying concepts. Moreover, students should develop appropriate skills to deal with complex issues, which enables them to participate in societal processes of democratic decision making . In short: the reality that society evolves towards greater complexity requires students to develop their scientific literacy. However, a challenging question is how to educate students to become scientifically literate citizens. Therefore, there is a need for teaching strategies that scaffold students on their way to become scientifically literate citizens.
This research explored to what extent the implementation of Scrum, a methodology intended to monitor and manage complex projects, can contribute to enhance students' scientific literacy.

Scientific literacy and science education
The term 'scientific literacy' has been used extensively in the educational literature (Holbrook and Rannikmae, 2009). Although there are many, often overlapping, definitions, it seems that there is consensus that this term includes students' ability to understand, use and apply scientific knowledge (Norris and Phillips, 2003). In addition, scientific literacy entails the ability to recognize topical real-world questions and, furthermore, the skill to draw evidence-based conclusions in order to generate well-informed decisions required for participation in democratic societies (Yacoubian, 2018). Based on this concept of scientific literacy, Roberts (2011) elaborated two viewpoints of science education. Vision I focuses on scientific theories, its underlying concepts and the scientific method. Typical examples are found in teacher-directed, traditional teaching approaches in which students develop understanding of concepts in a rather isolated way, without regard to how these concepts might be transferred to other contexts (Aikenhead, 2007). Within Vision I, contexts are used to illustrate concepts. In contrast, teaching based on Vision II starts with a meaningful, real-life context, and devotes particular attention to its societal aspects. Within these authentic practices, concepts are introduced on a 'need-to-know' basis (Pilot and Bulte, 2006). Moreover, students work collaboratively with classmates and are encouraged to explain course materials to other students. In addition, students receive timely and frequent feedback intended to enhance their learning process. Furthermore, they are invited to transfer the concepts learned and competencies acquired to new contexts. They are challenged to recognise and solve real-world problems by using relevant information sources. In sum, students work in small groups, deploy versatile communication skills to become independent and lifelong learners (Overton and Randles, 2015) as well as scientifically-literate citizens (Bennett, 2017, 23). Learning within Vision II can be characterised as student-centred. Its characteristics can be found in several teaching strategies, including problem-based learning (Barrows and Tamblyn, 1980), project-based learning (Krajcik and Shin, 2014) and context-based approaches (Pilot et al., 2016).
Worldwide, the major trend in secondary chemistry education reform is a shift from Vision I to Vision II (Pilot et al., 2016;Sevian et al., 2018). However, recently, a more elaborated form of Vision II is proposed, called Vision III (Sjöström and Eilks, 2018). It emphasizes that students need to develop a criticalreflexive attitude towards the context and concepts presented to them. Vision III is value-based and intended to take into account the complexity of life, of society and their mutual interactions (Sjöström, 2015) and is similar to critical scientific literacy (Sjöström and Eilks, 2018). Vision III aims at strengthening students' learning beyond content (Vision I) and contexts (Vision II). Within Vision II students learn about a specific context and its underlying concepts, whereas within Vision III students develop a critical-reflexive attitude that goes beyond the context and concepts and that helps them to make wellinformed, data-based, and value-based decisions. Vision III is intended to stimulate students to take responsibility for their personal lives and to participate in society. Although there is some overlap between Vision II and Vision III with regard to learning objectives and learning strategies (Table 1), it seems reasonable to assume that measurement of students' learning progress within these learning environments requires additional, innovative assessment strategies. In contrast to a Vision I learning environment, in which teachers often use standard, straightforward, summative assessments, with multiple choice questions or questions with well-defined answers, within Vision II and Vision III special attention is paid to feedback and reflection. Therefore, formative assessments might play an important role as an appropriate assessment tool, although summative assessments are not without relevance and often used (Orpwood, 2007).
Clearly, due to the multifacetedness of socio-scientific issues, assessing students' critical scientific literacy is rather complex, as students might propose different solutions for the real-world issue. Assessing and subsequently quantifying the multiplicity of students' solutions, is challenging (Romine et al., 2017). Romine et al. (2017) showed that a broad variety of assessment tools, including tools to measure informal reasoning, argumentation, and reflective decision making, have been used to measure students' scientific literacy. However, it seems reasonable to expect that a Vision III learning environment might benefit from additional, innovative teaching and assessment strategies to measure students' critical scientific literacy.
The key-characteristics of Vision I, II and III with regard to learning goals, learning process and assessment have been brought together in Table 1. The characteristics are adapted from Pilot et al. (2016, 228-229) and Sjöström et al. (2018) and complemented.
Enhancing students' critical scientific literacy (Vision III). The selection of an appropriate context or task is a crucial first step to enhance students' critical scientific literacy. A suitable context meets with the following criteria: (1) it consists of a reallife question, (2) students experience the context as relevant, (3) it promotes discussion, and, preferably, (4) it has a controversial character in society (Marks and Eilks, 2009;Stolz et al., 2013). Such contexts force students to think about potential (dis)advantages, and invite them to propose solutions based on values and scientific data (Eilks and Hofstein, 2014, 10). There are numerous examples of suitable contexts available, including socio-scientific issues on climate change (Flener-Lovitt, 2014), and genetic modification (Lederman et al., 2014). Furthermore, the field of green chemistry provides a broad variety of socioscientific issues, that can be used in an educational context: e.g. on biofuels (Mamlok-Naaman et al., 2015), and biopolymers . Green chemistry takes into account the effects of chemicals on people and planet and focuses on sustainability. It intends to stimulate people to reduce or eliminate the use and generation of substances which are harmful to human health and the environment by investigating alternative processes to synthesize chemicals (Anastas and Warner, 1998;Zuin and Mammino, 2015). Green chemistry takes into account the entire life cycle of a substance, including its design, use and disposal. Green chemistry education aims to (1) provide information how chemical reactions could be designed to be more eco-friendly; (2) deepen students' knowledge of underlying chemistry concepts; and (3) teach students to develop their scientific literacy and corresponding skills (Zuin and Mammino, 2015, vii). Clearly, green chemistry education goes beyond the objectives of Vision I and meets with the criteria of Vision II. In addition, it offers the ingredients that might contribute to Vision III education in which students acquire skills to act as responsible future citizens. Furthermore, if the socio-scientific issue is embedded in a classroom environment that scaffolds students' learning, the development of their scientific literacy will be strengthened (Presley et al., 2013). A socio-scientific issue, for example derived from the field of green chemistry, is by its nature multifaceted: conceptual, contextual and societal aspects are strongly intertwined. Students deploy both cognitive and metacognitive skills to address its complexity. In such student-centred learning environments students might perceive difficulties in connecting concepts, context and social aspects, which, in turn, might slow down their learning. Constable et al. (2019) showed that systems thinking approaches might assist students in ameliorating the challenges associated with students connecting all these aspects. However, given the fact that, in general, such learning environments provide less guidance to students, they need scaffolds to apply concepts and to connect the context to their personal lives (Broman et al., 2018). Thus, teachers might use teaching strategies that support students to recognise and understand key concepts present in a socio-scientific issue. Scaffolds help students to manage and monitor their learning process, support mutual collaboration and are intended to provide tools to stimulate students to reflect critically on their learning process as well as on their own role as future citizens.
There are many examples of teaching strategies intended to scaffold students' learning in such a way. Marks and Eilks (2009) describe a lesson series with authentic media, using newspaper articles to introduce the context and to prompt questions. In addition, they used role-play activities, in which students adopted the role of journalist to produce a news item, and panel discussions, with students in the role of chemist, engineer or environmental protection activist. Students learned chemistry with a combination of practical lab work, cooperative learning techniques and conceptual learning. Marks and Eilks (2009) found that both teachers and students appreciated the approach. Teachers and students characterised the teaching strategy as motivating, intense and relevant for their personal lives. Table 1 Characteristics of Vision I, II and III. Based on and adapted from Pilot et al. (2016, 228-229) and Sjöström et al. (2018)  Challenge to reflect on relevance and opportunities for transfer Challenge to reflect on relevance, opportunities for transfer and connection with personal life and society Assessment Focus on summative assessment Both formative and summative assessment Summative, formative and additional, alternative assessments Furthermore, Marks and Eilks (2009) suggest that the use of socio-scientific issues might induce changes in students' attitude towards chemistry in general and improves their communication skills. In addition, Barraza and Ruiz-Mallén (2017) explored and investigated the 4D-approach, a teaching strategy based on dialogue, divergent thinking, discussion and debate, intended to enforce higher cognitive skills such as critical thinking. Barraza and Ruiz-Mallén (2017) showed that the implementation of the 4D-approach in a classroom enhances students' ability to deal with controversial socio-scientific issues. Moreover, they report that the 4D-approach scaffolds students to make well-informed and balanced decisions as future citizens in a democratic society.
However, despite these examples, enhancing students' scientific literacy remains a complex endeavour. Although the criteria for selecting appropriate teaching contexts to improve students' critical scientific literacy are clear (Stolz et al., 2013), it remains challenging to implement these approaches in the classroom. In addition, Sevian et al. (2018) emphasized the need for additional studies which focus on how students' progress can be monitored and what teaching strategies might scaffold students' learning process. We argue that these rather complex learning environments might benefit from the implementation of Scrum methodology.

Scrum methodology
Scrum methodology was introduced in the mid-1990s to manage complex projects in the field of software development (Schwaber and Sutherland, 2017). Basic tenets of Scrum methodology are transparency, inspection and adaptability. The framework consists of ceremonies, artefacts and roles that contribute to visualization of progress of the project, provide information on the quality of intermediate products and help employees to adjust their work to customers' desires. The characteristics of Scrum methodology have been transferred to educational contexts, including writing courses (Pope-Ruark, 2015), courses on software engineering (Mahnic, 2010) and context-based approaches in secondary chemistry education (Vogelzang et al., 2019;. Scrum methodology is an iterative process and evokes feedback moments systematically. The teacher, in their role as product owner, introduces a social-scientific issue, e.g. on a green chemistry topic. Students are divided into groups of approximately four persons. All students commit themselves to deploy their skills (e.g. writing skills, planning skills) to the team explicitly. The product owner provides each team with a product backlog, which consists of assignments, exercises and practical work, necessary to formulate an answer to the realworld issue. Every group has the autonomy to plan their own work. Their planning is visualised on the Scrum board, which basically consists of three columns, 'to do', 'doing' and 'done'. Students write tasks on Post-Its and stick them on the Scrum board. Every lesson starts with a stand-up ceremony, in which students discuss what they will do during the lesson. When a task is completed, the accompanying Post-It is positioned in the column 'done'. The lessons of approximately two weeks form a 'sprint'. A sprint concludes with a review ceremony in which the students receive feedback on their learning progress. A review ceremony might have different forms, including a check on the quality of an intermediate product, a panel discussion or a formative assessment on concepts covered in the sprint. The review ceremony sheds light on misconceptions in an early stage and offers opportunities for students to reconsider specific concepts and for the teacher to adjust their teaching to the specific needs of their students.
A review ceremony is followed by a retrospective ceremony, in which students discuss the quality of their learning process. They discuss their mutual collaboration and formulate one point of improvement for the next sprint. After two or three sprints, students release their final product, for example, an answer to the real-world question associated with the socioscientific issue, a written advice or a final product. Depending on school policy, a summative assessment might be used to finish the entire project. Especially the review ceremony and the retrospective ceremony might scaffold students by evoking critical reflection on conceptual, contextual as well as societal aspects of the socio-scientific issue in a natural and systematic way. Together, these ceremonies intend to stimulate both a reflective attitude and an ethical awareness concerning the consequences of the solution proposed. In addition, social interactions of students, in the form of discussions and mutual feedback might shape and influence their critical scientific literacy.

This study
This study was conducted in order to conceive, implement, and assess the influence of the use of Scrum methodology in a context-based approach on Green Chemistry.
The following research questions (RQ) guided this study: RQ1. What are the effects of Scrum methodology on students' understanding of chemistry concepts involved in the Green Chemistry module?
RQ2. What are the effects of the implementation of Scrum methodology on the development of students' critical-reflexive scientific literacy?

Method
Context of the study A module on Green Chemistry was implemented in secondary context-based chemistry classrooms (grade 11) in The Netherlands (Jansen-Ligthelm et al., 2010). The goal of the module was that students formulate an advice in which they argue what the greenest synthesis is to produce adipic acid, a precursor for nylon polymers and a preservative (E355) for food. The module started with exercises and assignments to deepen students' knowledge of the twelve principles of Green Chemistry (Table 2) and concepts concerning reaction enthalpy and block diagrams.
The assignments and exercises were embedded in illustrative contexts, suggesting that this part of the module typically fits within Vision I, where focus is mainly on concepts. In this stage of the course students became aware of how chemical reactions can be designed eco-friendlier.
The second part of the module was designed in line with the characteristics of context-based approaches (Vision II). In this part, full focus was on the real-world question of the greenest synthesis of adipic acid. Answering the real-world question required that students used concepts connected to this context on a 'need-to-know' basis. Students were challenged to apply and transfer the twelve principles of Green Chemistry, to a new situation; i.e. to the case of adipic acid. They received two different routes to synthesize adipic acid; (1) a multistep oxidation starting from cyclohexanol and nitric acid; (2) a single step oxidation of cyclohexene with hydrogen peroxide (Fig. 1). For both routes they applied the twelve principles of Green Chemistry, that is, they calculated atom efficiencies, Environmental-factors, theoretical yields and reaction enthalpies and searched for information on toxicity for the chemicals involved (Bodner, 2015).
Subsequently students were invited to interpret their outcomes and reflect critically on the societal consequences of the synthesis routes (Vision III). They were asked to formulate a written advice in which they balance both routes and made an informed decision on the route they preferred. The assignment was: ''Within a radius of 10 kilometres of your school a new chemical plant will be built to produce adipic acid. There are two routes to produce this chemical and you will receive information on both. As junior professionals, you will provide the block council near your school with a substantiated, scientific advice which route is preferable. Feel free to add any information or arguments to underpin your final decision.'' Implementing Scrum methodology. A pre-test/post-test control condition design was used to explore the effects of the use of Scrum methodology on students' understanding of chemistry concepts and the development of students' criticalreflexive scientific literacy. In both the experimental condition and the comparison condition there was no difference with Table 2 Green Chemistry principles (Bodner, 2015) 1.
Design less hazardous chemical synthesis 4.
Design safer chemicals and products 5.
Use safer solvents and reaction conditions 6.
Increase energy efficiency 7.
Design chemicals and products to degrade after use 11.
Analyse in real time to prevent pollution 12.
Minimize the potential for accidents Fig. 1 Two synthesis routes for adipic acid.

Chemistry Education Research and Practice Paper
Open Access Article. Published on 02 May 2020. Downloaded on 11/19/2020 2:29:37 AM. This article is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported Licence.

View Article Online
regard to learning objectives (rationale, affective and (meta)cognitive, Table 1). The objective of the module was to evoke critical reflection on the impact Green Chemistry can have on the environment students live in. In both the experimental and comparison condition, students prepared a written advice. Both conditions did differ with regard to how students' learning process was organised and with regard to intermediate assessments during the lessons (Table 3). Students participating in the experimental condition used the Scrum methodology framework to plan their work, and to monitor their progress on a scrum board. Their teachers reported that they executed the Scrum ceremonies as intended. During the stand-up and retrospective, according to their teachers, the students reflected on the quality of their learning process and discussed how they could improve their learning approaches. Students of the comparison condition did not receive specific scaffolds to manage their planning and monitor their progress. Just as the students of the experimental condition, they worked in groups. However, they were free to use their own strategies to plan and monitor their progress.
Another salient difference between the experimental and comparison condition is the intermediate assessment during the review ceremony. Students of the experimental condition worked in iterative sprints with a review at the end of them, in which they checked their understanding of Green Chemistry principles. The reviews had the form of formative assessments. Students made them individually and discussed their answers with team mates as well as the teacher. On request, the teacher provided additional explanations.
An overview of the differences between teaching approaches of the experimental and comparison condition is provided in Table 3. Table 3 Learning process and assessment of critical-reflexive chemistry education connected to teaching strategies used in the experimental and comparison condition Vision III: critical-reflexive chemistry education (see also right column of Table 1)

Scrum methodology (experimental condition) Comparison condition
Learning process Rationale Learning by doing. Students plan and monitor their progress systematically, using Scrum board and stand-up ceremonies.
Students are free to plan their work as they prefer.

Situation
Critical-reflection on socio-scientific issues.
Ultimate objective is to evoke critical reflection Ultimate objective is to evoke critical reflection Social setting Participating in teams, taking up relevant roles to search for appropriate and valuebased answers.
Teams are based on qualities. Students promise to deploy their personal qualities to their team.
Teams are formed by students. No external regulation. Teams are often based on personal friendships.

Control
Shared control, reflecting on the role of (chemistry) concepts in socio-scientific issues.
Shared control, students have the lead, the teacher in their role as productowner is near. Ceremonies are used to guide and monitor students' learning.
Shared control, students have the lead. The teacher is available on request and has a facilitating and stimulating role. No specific procedures were used. Cognitive Ideas are shared, welcomed and weighed by students and teacher.
Ideas are shared etc. However, not systematically evoked by Scrum methodology.
Ideas are shared etc., although not systematically evoked by the teaching strategy. Testing and reflecting of concepts on realistic contexts and tasks.
Review ceremony evokes testing and reflecting on intermediate products and concepts, explicitly.
No systematic reviews. Reflecting on learning process and progress is stimulated by the teacher and takes place on-the-fly. Leading to knowledge with proven value in various contexts.
Not specifically induced by a ceremony of Scrum methodology.
Not specifically induced by the teaching strategy. Learning to use knowledge in personal life and socio-scientific issues.
Not specifically induced by a ceremony of Scrum methodology.
Not specifically induced by the teaching strategy. Affective Valuing relevance for and critical reflection on knowledge contributing to responsible citizenship.
Not systematically. However, ceremonies such as review and retrospective might support the socio-scientific issue to evoke critical reflection.
Not systematically. The socio-scientific issue might evoke critical reflection.
Metacognitive Continuous challenge to improve on reflecting, defining problems, planning, steering both individual and collaborative learning.
Stand-up, review and retrospective challenge students to plan, reflect on and monitor their individual and collaborative learning.
Although the socio-scientific issue is intended to enforce reflection on problems and challenges students to plan and monitor their progress, systematic and planned reflection does not take place. Closing Challenge to reflect on relevance, opportunities for transfer and connection with personal life and society.
Scrum ceremonies might scaffold the socio-scientific issue and induce reflection on relevance, and promote transfer and connection with personal life and society.
No systematic scaffolding of the socioscientific issue.

Assessment
Focus on alternative assessments (although formative and summative assessment can play a role).
Formative assessments at the end of each sprint to get insight in conceptual development as well as quality of intermediate products.
In the final stage of the module students produce a written advice.
No use of formative assessments, or other assessments to check conceptual development or quality of intermediate products.
In the final stage of the module students produce a written advice.

Participants
The module described above was implemented in eight classes, taught by five teachers with at least ten years of teaching experience. Of these classes two teachers (both male, four classes) used Scrum methodology as teaching strategy, forming the experimental condition. The comparison condition was formed by three teachers (two females, four classes), who implemented the module with their regular teaching style (see Table 3). The research was carried out following the guidelines for research ethics and integrity of Leiden University. All students and their teachers were informed about the aim of the study, which was to gather information on how they learn the principles of Green Chemistry and to improve classroom teaching. They were told that their participation was voluntary and that they had the opportunity to opt out at any stage of the study. Students received information that their answers were anonymised and therefore could not influence their grades. Students and teachers were informed that with participation they provided their consent to use their responses for research purposes. In total 198 students, distributed over 54 groups of three or four persons, participated. In the experimental condition 25 groups (99 students, 44 females) participated and the comparison condition consisted of 29 groups (99 students, 56 females). The participating teachers worked at different school from all over The Netherlands. They were familiar with both teachercentred learning environments and context-based, studentcentred learning environments. They responded to an email invitation, written by the first author, and distributed by teacher trainers of several teacher education institutions. They voluntarily choose whether they participated in the experimental or in the comparison condition. Teachers participating in the experimental condition participated in a professional development program (five sessions of four hours) over a period of nine months in which they studied the ceremonies, roles and artefacts of Scrum methodology, shared and discussed their experiences during the implementation of the framework in their chemistry lessons.

Data and instruments
During the study three types of data were gathered: (1) a test on students' previous knowledge of Green Chemistry principles; (2) a group task on the application of these principles to answer RQ1, and (3) a joint group advice on the greenest synthesis of adipic acid. These advices were used to answer RQ2.
Students' previous knowledge. At the start of the module, all students individually completed twelve open questions about the twelve principles of Green Chemistry. For instance, students were asked to define the Green Chemistry term 'E-factor' and to calculate the atom-efficiency as well as the energy-enthalpy of different reactions. Maximum scores on the 12 open questions varied from 1 to 3 (Appendix A). Overall maximum score was 23. Scores of individual group members were combined to an average group score and converted in a percentage (23 points = 100%). The average scores of the experimental and the comparison condition did not differ significantly (F(1, 52) = 0.093, p = 0.762), suggesting that students in these conditions are comparable with regard to their prior knowledge. The answers of 15 students (12.6%, 300 items) were checked by a second rater, and resulted in an inter reliability score of Cohen's kappa k = 0.925 (p o 0.001), 95% CI (0.980, 0.987), suggesting a good reliability between the raters. Descriptive statistics are presented in Table 6.
Students' understanding of chemistry concepts involved in the Green Chemistry module. All student groups were asked to apply the twelve principles of Green Chemistry to the two different synthesis routes of adipic acid. All student groups planned and monitored their work themselves and released a joint report in which they answered items about the two synthesis routes. The 40 items included a variety of calculations (atom efficiency, E-factor, reaction enthalpy), development of block diagrams, toxicity of chemicals as well as on reusability of the chemicals involved (Appendix A). The maximum scores per item varied from 1 to 3 points. The overall maximum score was 46 points. The average scores were converted in a percentage (46 points = 100%). The answers of 6 groups (11%, 240 items) were checked by a second rater, resulting in an inter reliability score, of Cohen's kappa k = 0.854 (p o 0.001), 95% CI (0.944, 0.966), and showed a good reliability between the raters. Descriptive statistics are presented in Table 6.
Students' critical science literacy. The last source of data was formed by the written advices, which were used to measure students' critical-reflexive scientific literacy. All 54 groups were required to release their final product, i.e. their written advice, in which they provided a balanced response to the greenest synthesis of adipic acid. Two characteristic excerpts of the written advices, translated from Dutch, are presented in Appendix B. The complexity of the written advices was analysed with the SOLO-taxonomy, originally developed by Biggs and Collis (1982). Their Structure of Observed Learning Outcomes (SOLO) has been used extensively in a variety of disciplines (Stewart, 2012), including mathematics and chemistry to analyse students' responses (Tomperi and Aksela, 2014). It comprises five levels of understanding: (1) pre-structural, (2) unistructural, (3) multistructural, (4) relational and (5) extended abstract. In Table 4 an overview of different SOLO-levels is presented, including sublevels, corresponding scores and examples of verbs.
In the pre-structural level, there is little evidence of learning. The student did not approach the socio-scientific issue on Green Chemistry appropriately. At the unistructural level, the student focused on one relevant aspect without making connections to other aspects. At the multi-structural level, students' writings comprised several relevant aspects without making connections between them. This level can be characterised as quantitative in nature, which means that the amount of details and the number of aspects mentioned, increased. When students used two or three Green Chemistry principles their advice was characterised as multi-structural (low) and thus rewarded with only 2 points. Groups that used all twelve principles received 4 points. These advices were characterised as multi-structural (high), containing many related aspects, however without making connections between the principles.

Chemistry Education Research and Practice Paper
Open Access Article. Published on 02 May 2020. Downloaded on 11/19/2020 2:29:37 AM. This article is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported Licence.

View Article Online
At the relational level, students treated the different aspects of the socio-scientific issue as an integrated whole. This level can be characterised as qualitative (Tomperi and Aksela, 2014). In their advices they showed that conceptual, contextual as well as societal aspects of the Green Chemistry principles are closely connected and mutually related. Advices at the relational level had a minimum score of 5 points. Scores of 6 or 7 points were assigned to advices in which students made respectively several or many connections between the Green Chemistry principles. At the extended abstract level, students are supposed to conceptualise the previous integrated whole on a higher level of abstraction. Their writings go beyond the requirements of the assessment and comprise a critical reflection on the socioscientific issue from multiple perspectives, including a personal perspective. At this level, students generalize, create and transfer ideas to new contexts. Obviously, the extended abstract level is strongly connected to students' critical scientific literacy. All advices were scored independently by two raters. Further details with regard to the SOLO-scores are provided in Table 5.
Data in the matrix reveal the instances where rater 2 (dis)agreed with the score assigned to an advice by rater 1. The scores are ordinal data; therefore, Cohen's kappa was calculated. The interrater reliability measure was found to be k = 0.773 ( p o 0.001), 95% CI (0.971, 0.990), suggesting that the scores assigned by the raters have substantial reliability. Therefore, the scores of rater 1 and rater 2 were used to calculate a mean score for all separate 54 advices. Descriptive statistics are presented in Table 7.

Data analyses
To answer the first research question analysis of covariance were carried out with the experimental and comparison condition as factor, the pre-test scores as covariate and the knowledge test scores at the group level as dependent variable. To answer the second research question analysis of variance were performed at the group level with experimental and comparison condition as factor and the quality of group advises indicated by their SOLO levels as dependent variable.

Students' understanding of chemistry concepts involved in the Green Chemistry module
Students participating in the experimental condition outperformed students of the comparison condition. Analysis of data revealed a large effect-size (see Cohen, 1988), suggesting that the implementation of Scrum methodology enhances students' understanding of Green Chemistry concepts: (F(1, 52) = 11.912, p o 0.002, Z 2 = 0.189). Pre-test scores were used as covariate. Descriptive statistics are presented in Table 6.

Students' critical science literacy
Data analysis of the SOLO-scores of both the experimental and the comparison condition revealed that the advices of all 54  Biggs et al. (1982)   groups had a score between 2 and 7 and therefore fit within the SOLO-levels, multi-structural and relational (Table 7). None of the advices was characterised as pre-structural, unistructural or extended abstract. However, 70% of the groups using Scrummethodology delivered an advice on the relational level, whereas only 41.5% of the comparison condition produced an advice on this level (Table 7). Obviously, the groups participating in the experimental condition outperformed groups of the comparison condition. ANOVA-analysis revealed a medium effect-size: (F(1, 52) = 4.427, p o 0.05, Z 2 = 0.080) ( Table 7).

Discussion
In this study, effects of the use of Scrum methodology were examined on understanding of concepts connected to Green Chemistry education and the quality of written advices concerning the greenest synthesis of adipic acid.

Impact on students' understanding of concepts
The results reveal benefits of Scrum methodology for students' conceptual understanding. It provided scaffolds for students participating in teams, using a realistic real-life context, in which they were challenged to discuss, monitor and improve their learning (see Vision II, Table 1). Inherently, the ceremonies in general, and the review in the form of a formative assessment at the end of a sprint in particular, promoted mutual feedback and enforced students to reflect on their understanding of the concepts involved. Consequently, both teacher and students might be confronted with misconceptions in an early stage, increasing the probability that teacher and students discuss these issues together. Ideas and questions were shared and welcomed. Obviously, the features of Scrum methodology contribute to a classroom climate in which students reflect on chemistry concepts, which, in turn, might explain the increased student learning achievements. The findings align with educational research on the implementation of formative assessments in the classroom, which, in general, suggest an increase in conceptual understanding and learning outcomes (Wiliam et al., 2004).

Impact on students' critical scientific literacy
In general, advices developed in Scrum classes were more elaborated than advices composed by students of the comparison condition. It seemed that the Scrum ceremonies, roles and artefacts guided students through their learning process and enforced them to discuss and reflect on the Green Chemistry principles. Moreover, approximately 70% of the student groups using Scrum methodology reached a relational level, whereas the majority of the comparison condition (455%) remained at the multi-structural level. This can be understood as a clear indication that the implementation of Scrum methodology scaffolds students in their process to become scientific literate. However, none of the 54 groups reached the extended abstract level. None of them connected the issue to other actual, societal issues or to their personal lives. This might be explained in various ways. The students were unfamiliar with this kind of written assessments and were not used to go beyond its requirements to formulate their own personal opinion. The student assignment focused on principles of Green Chemistry (E-factor, atom-efficiency etc.) and their mutual relations. It might be that an extra phrase, e.g. Describe how the principles of Green Chemistry might affect your personal life or Describe whether the Green Chemistry module influences your personal choices with regard to sustainability issues, would have increased the chance that students would have reflected critically on the principles of Green Chemistry and connect them to other societal issues or to their personal lives (Vision III); (Sjöström et al., 2018). Nevertheless, it seems that alternative assessments, such as these written advices, contribute to encourage students to think about and reflect on the impact of chemistry on society as well as their personal circumstances. Students are stimulated to explore the concepts and context, and, moreover, they are invited to add and reflect on their own ideas. In addition, the SOLO-taxonomy provides an appropriate working tool to obtain a representation of students' critical thinking (Vision III). Furthermore, the results suggest that the features of Scrum methodology scaffolds students to organise their learning process and enforce them to converge their thinking to answer the real-world question. However, the implementation of Scrum methodology is not the only factor that impacts the development of students' critical scientific literacy. Other important factors include the role of the teacher and the classroom climate (Boss and Larmer, 2018). A teacher who is able to create a classroom climate in which students work collaboratively on a shared objective, increases the opportunities to enhance students' critical scientific literacy. Although all teachers in the experimental condition and the comparison condition were experienced teachers, it is impossible to exclude a teacher effect. Admittedly, even when there is a positive classroom climate, a context-based learning  environment remains rather complex, and the results seem to suggest that Scrum methodology benefits students when they work together on a rather complex real-world issue.

Limitations and directions for future research
There are some limitations in the study that should be taken into account. First, students participating in the experimental condition were unfamiliar with Scrum methodology. As a consequence, they needed some time to become accustomed to the ceremonies, roles and artefacts. Therefore, the differences between the experimental and comparison condition might be underestimated. Secondly, the number of participating teachers, in both the experimental and comparison condition, is confined. This limits the generalizability of the results. A replication of this study with more teachers, and, in addition, with a larger sample size of students who are familiar with Scrum methodology might shed light on the generalizability of the results.

Conclusions
Findings suggest that the implementation of Scrum methodology might scaffold students' learning in at least three ways. First, it enhances students' conceptual understanding (Vision I). Secondly, its ceremonies, roles and artefacts scaffold students when they collaboratively work on rather complex real-world questions. Scrum invites to think critically, to provide feedback to each other (Vision II) and scaffolds students to apply the concepts in new contexts. Scrum methodology as framework might strengthen the shift to student-centred learning environments, because it might decrease feelings of overwhelmingness among students. Thirdly, in combination with a challenging socio-scientific issue that evokes interest and discussion, it reinforces their communication and collaboration skills and helps them to take responsibility in their socio-cultural environment, promotes participation in a democratic society, and appreciates skills and talents of other citizens (Vision III). It is worthwhile to perform additional research to explore whether the potential benefits of frameworks such as Scrum methodology and probably other agile project management frameworks work in other cultures and with other subjects.

Conflicts of interest
There are no conflicts of interest to declare.
2. Provide equations for all reaction steps of synthesis route 1 as well as for route 2 (7 points).
3. Calculate the atom economy for synthesis route 1 and synthesis route 2 (7 points).
4. Calculate the E-factor for synthesis route 1 and route 2. Take into account the reaction yields as provided in the description of the module (6 points).
5. Distinguish potential waste-products and discuss their impact on the E-factor (for both route 1 & route 2) (5 points).
6. Discuss the hazardousness of all substances involved in route 1 and route 2 (4 points).
7. Calculate the reaction enthalpy for route 1 and route 2. Use heats of formation as provided in the module (7 points).
Appendix B: two characteristic excerpts extracted from the written advices SOLO level (sub-level) Score Examples from the written advices of students Explanation Multistructural (high) 4 Finally, they wrote: We advise route 2 for several reasons. It does not use solvents, there are fewer steps, and a catalyst is used. Chemicals are less hazardous. Although that the atom efficiency is lower, when we combine our findings, we think that route 2 is eco-friendlier.
For both synthesis routes group 17 used the 12 principles. They described their opinion for all the 12 aspects. Therefore, their advice could be rated with at least 4 points. However, they made some calculations errors (e.g. atom efficiency was miscalculated). For all principles, they combined data for both routes. However, they made no connections between different principles. They did not reach the relational level. In addition, sometimes their argumentation was incomplete or wrong. Both raters awarded this advice with 4 points. Relational (high) 7 The final score of route 1 is À7, whereas route 2 scores +9. Obviously, synthesis route 2 is preferable. This route Group 3 compared the two synthesis routes by awarding points (++/+/0/À/ÀÀ) to all twelve principles.
has fewer reaction steps, uses less harmful chemicals, waste products are biodegradable and its chemistry is overall less hazardous. In addition, route 1 is more expensive. It comprises corrosive chemicals and therefore there is a need for stainless reactors. On the other hand, route 2 is still in its infancy.
A lot of research is necessary whether there are alternatives in the form of cheaper and/or reusable chemicals. Route 2 can only gain, and therefore we choose route 2: synthesis of adipic acid from cyclohexene with hydrogen peroxide.
They explained their argumentations carefully, including correct calculations and to-the-point descriptions. This group compared both synthesis routes and applied their data to new situations (e.g. corrosive chemicals require expensive stainless reactors). They critically reflected on the preferred route and suggested some alternatives. They did not connect their advice to their personal lives or other societal issues. Both raters awarded this advice with 7 points.