Re-defining lab norms via professional learning communities of physics teachers

We present a study of a first step in a large-scale intervention designed to shift lab instruction away from highly prescribed lab norms. The intervention was implemented in a national network of Professional Learning Communities of high school physics teachers (N=250) operating in a high-stakes exam setting with limited resources, and catering to diverse groups of students. A pre-intervention survey examined learning goals that the teachers value as well as practices taking place in their lab lessons. The findings revealed a gap between the importance that teachers attributed to scientific practices such as experimental design and the manifestation of these goals in the national lab exam. In addition, the survey revealed disparities between the scientific practices that teachers believe that their students engage in during the instructional lab and the practices they identify in physicists' work in research labs. The intervention consisted of a series of Restricted Inquiry Labs (RILs) designed to address the teachers' interest in change as well as the constraints imposed by the setting in which they work by modest restructuring of traditional labs such as encouraging students to reflect on the considerations underlying the experimental design. The intervention proved successful in that half of the teachers reported that they introduced the RILs. They described students' engagement when granted more autonomy in designing an experiment, as well as students' difficulties in coping with a more open-ended task. Implementation of the RILs also created challenges in class management. Despite these challenges, most teachers reported that they intended to keep using RILs. Teachers asked to re-design assessment policies to emphasize scientific inquiry. The RILs proved to be a feasible way to introduce change towards inquiryoriented labs via professional development programs, while pointing out aspects that need to be addressed to respond to teachers' concerns.


I. INTRODUCTION
Key position papers 1, 2 have called to reform the instructional science lab by moving away from the widespread contrived lab 3 towards inquiry-oriented labs that represent aspects of authentic research, and give students more responsibility for their work 4. However, powerful structural features in the education system (e.g. the need to "cover" a set syllabus, and hence to complete many experiments in a short time, shortages of equipment and assistants, immersion in the established school culture where students and instructors shape existing habits and views) drive most education systems in the direction of highly prescribed labs 5. Levers for change could take the form of curricular resources, such as those provided in the Investigative Science Learning Environment (ISLE) 6, assessment policies such as the redesign of the AP exams to emphasize discipline-specific inquiry, reasoning, and communication skills 7 or of Professional Development (PD) initiatives 8.
This study examines the use of a large-scale PD program for high-school physics teachers 1 as leverage for reform in the instructional lab that directs teachers to include more scientific reasoning underlying experimental practices. The PD operates in a test-driven setting, where the lab course must follow a national curriculum and students take a national mandatory lab exam.
Teacher educators point to the value of situating PD programs within teachers' practice 9, 10 by engaging teachers in collaborative reflection on their instructional practices and student learning. Professional Learning Communities (PLCs) have been shown to be an effective PD framework 11, 12. The PD intervention examined in this study took place in a national network of PLCs for physics teachers. It consisted of a series of Restricted Inquiry Labs (RILs) designed to accommodate the goal of giving more autonomy to students but also the severe constraints imposed by the setting in which the teachers operate (e.g., time allotted for the activities, equipment available in the schools, relevance to the curriculum and the national lab exam), by a modest restructuring of traditional labs.
The PLCs program, which has been operating since 2012, is structured as a "Fan Model" where practicing high school physics teachers serve as the Teacher-Leaders (TLs, N=24) of 11 regional PLCs of physics teachers (N=225 teachers). The TLs participate in a TL-PLC operated by the Physics Education Research (PER) group at the Weizmann Institute of Science. The PLCs meet twice a month during the school year (for a total of 60 h). The design guidelines were conceptualized within the instructional paradigm of Cognitive Apprenticeship 13, situating teacher learning in their practice (the instructional lab), providing opportunities to externalize and to reflect on the considerations underlying teachers' decision-making in designing and carrying out instruction.
This study is the initial step in a process of designing and implementing the PD intervention, and is a part of a larger study. The first and most important issue for this line of research is to determine the feasibility of the design guidelines of the RILs. To that end, the extent to which teachers are receptive to making lab instruction more "science process oriented" 6 and to giving students more autonomy, despite the structural constraints.
The research questions are as follows: 1. Learning goals for the Instructional lab: How do the goals that teachers value for the instructional lab align with the scientific practices recommended as learning outcomes in central position papers? Are they satisfied with the manifestation of these learning outcomes on the national physics lab exam? 2. Scientific practices in the Instructional lab: a. Which lab practices do teachers believe their students engage in during the instructional lab? b. Which lab practices do teachers believe experimental physicists engage in while conducting research in their labs? To what extent do the instructional lab practices align with experts' practices? 3. Enacting scientific practices in the instructional lab: To what extent did teachers implement the RILs? What challenges have they faced, and how are they planning to deal with these challenges?

A. Methods
A pre-intervention survey was administered during the first yearly meeting in all 11 regional PLCs. The teachers had a wide range of teaching experience (1-49 years, Md=7 years), the gender distribution was 37% females, 63% males, and more than half (54%) switched to physics teaching as a second career. They teach in schools serving a broad spectrum of socioeconomic backgrounds.
Part I of the survey focused on learning goals for the instructional lab, and satisfaction with their manifestation in the national physics lab exam. The teachers were given a list of 11 learning goals drawing on core national and international position papers 1, 2. For each learning goal the teachers were asked to rate its importance and their satisfaction with its manifestation in the national lab exam on a 5-point Likert scale (ranging from, 5 -very important/very satisfied; 1 -not important at all/ not satisfied at all). An open option, "other", enabled addition of learning goals that were not included in the list.

B. Results
All the lab learning goals were highly valued by the teachers (N=172, based on teachers' attendance at that PLC meeting, M>4, 0.6<SD<0.8, see Table I). The teachers were less satisfied with the manifestation of the goals on the national lab-exam (3<M<4, SD>0.8, see Table I). There were significant disparities, for all 11 goals, between the importance attributed to the goal and satisfaction from its manifestation (p<.0001, paired t -test). Teachers' background variables, including gender, teaching experience, other professional experiences (i.e., second career teachers), PLC (i.e., out of the 11 regional PLCs), did not affect the results.
The teachers' responses to the open question mostly echoed the goals in the given list or added affective goals (10%), e.g., interest and motivation, and the importance of hands-on experiences (5%) as valued learning goals. The disparity between the importance attributed to the goal and satisfaction from its manifestation in the national lab exam was the largest for goal 2 related to experimental design: "designing and improving an experiment in accordance with the difficulties encountered in execution" (t=11.1, p<.0001).
Overall, the teachers' beliefs on instructional lab learning outcomes were aligned with the recommendations of the key position papers, and they were less satisfied with the manifestation of these outcomes on the national lab exam.

A. Methods
Part II of the pre-intervention survey examined teachers' views of the scientific practices their students engage in during the instructional lab vs. the practices of experimental physicists in research labs.
This part was based on a modification of the E-CLASS survey 14, in which students are presented with 30 statements representing strategies, habits of mind, and attitudes related to central lab practices, and are asked to rate the extent of agreement from both their personal perspective and that of a hypothetical experimental physicist.
The survey was composed of 21 selected translated statements pertinent to the Israeli high-school lab context, and modified to the teachers' perspective instead of the students' perspective. The teachers were asked to rate the extent of their agreement with each statement on a Likert scale (5 -strongly agree; 1 -strongly disagree) from the perspective of an average student in their class working in a high school instructional lab, and that of hypothetical physicists in a research lab. The modified survey was content-validated by six PER experts, and then by the 24 TLs to make sure that teachers understand the statements and the associated scientific practices.

B. Results
The practice of experimental design emerged most clearly in three statements in the survey: 1) "When doing an experiment, I try to understand how the experimental setup works"; 16) "A common approach for fixing a problem with an experiment is to randomly change things until the problem goes away"; 18) "When doing an experiment, I just follow the instructions without thinking about their purpose". Statement 1 is positive in terms of the desired lab practice of experimental design, whereas statements 16 and 18 are negative. With respect to all three statements, the teachers acknowledged that there is a significant gap (p<.0001, paired t -test) between strategies and habits of mind from the perspective of students in the instructional lab and that of physicists in research labs. . 7 *** 11. Deepening students' scientific knowledge and understanding 4.5 (0.6) . 3 8 (0.9) 6 . 9 *** ***p<.0001 Fig. 1 presents the frequency distribution of teachers' extent of agreement with these three statements. Teachers strongly agreed that physicists try to understand how an experimental setup works, but students less so (Fig. 1a). The practice of fixing a problem by randomly changing things (Fig. 1b) was perceived by teachers as more typical of students and rare for physicists. They agreed that students, unlike physicists, follow instructions without thinking about their purpose (Fig. 1c). These findings indicate that the teachers clearly identified disparities between scientific practices that students engage in during the instructional lab and the practices of physicists in research labs, in particular with respect to experimental design.

A. The intervention: Restricted Inquiry Labs (RILs)
The intervention was aimed to highlight considerations underlying scientific practices in the lab, in particular experimental design. A series of Restricted Inquiry Labs (RILs) was developed and conducted in four PLC meetings. The RILs consisted of re-designed lab manuals for experiments in the lab curriculum. To accommodate structural constraints students were asked either to explicate the considerations underpinning a preset experimental design, or design limited aspects of the setup.
In one of the RILs, for example, the students were presented with five standard, ready to use, experimental systems. Small group work focused on discussions of the considerations underlying the design of each of the experimental systems. One of these experimental systems was the Tangent Galvanometer, in which the students discuss how to position the apparatus to take accurate measurements of the Earth's magnetic field and why it is preferable to adjust the current in the loop and measure the angle obtained in the compass, rather than the other way around. Another RIL involved a more substantial change in experimental design: the students were given Atwood's Machine apparatus instead of the traditional horizontal Newton's second law experimental system, and were required to design how to control the experimental variables.
The teachers as well as the TLs experienced each RIL as students and collaboratively reflected on the strengths and weaknesses they identified, and explicated the goals the RILs could promote. They were asked to try out the RILs in their lab lessons and discussed their classroom experiences together. The TLs experienced this PD process before leading their regional PLCs in a similar PD process.

B. Methods
After the intervention, a post-intervention survey was administered during the last PLC meeting, towards the end of the school year. The survey consisted of open-ended questions in which the teachers were asked whether they implemented the RILs, what other changes they made in their lab lessons, how their students coped with these changes, which challenges they faced, and how they were planning to deal with the challenges. The post-intervention survey was content-validated by six PER experts, as well as by the 24 TLs. Teachers' self-reports (N=119) were analyzed qualitatively, using a combination of "top-down" categories (e.g., extent of implementation and class management difficulties), and "bottom-up" sub-categories identified in the teachers' responses (e.g., students' engagement and students' frustration). The categorization and analysis were validated in an iterative process by the four authors, reaching an inter-coder reliability of 80%.

C. Results
Despite the tight constraints in which the teachers operate, half (51%) implemented at least one RIL in their lab lessons, or alternatively, reported that they changed lab norms to give more autonomy to their students. There was no significant difference in the implementation on background variables, except the experience in preparing students for the national lab exam (M=8.2 years for those who implemented RILs vs. M=5.0 years for those who did not, p < .05).
The section below discusses teachers (N=61) who implemented RILs. 1. Implementation: Many (39%) teachers mentioned that they gave their students more autonomy in designing experiments. A third asked students to plan how to control variables in the experimental system. A few (16%) devoted more time to a preliminary discussion with their students on the goals and the design of the experiments, and the rest (12%) referred to the analysis and interpretation of the results. Most of the teachers (87%) described the ways in which they gave more autonomy to their students by omitting some of the instructions in the traditional lab manual (64%) or by not using a lab manual at all (23%).
2. How students coped with the changes: Most of the teachers (85%) referred to students' engagement; e.g., "The students had to cope alone with the missing instructions"; "I really liked to hear their lively discussions, coping with the challenge together"; "The students were enthusiastic about being able to express their ideas". More than half of the teachers (56%) were concerned about their students' difficulties; e.g., "Some students failed to carry out the experiment, and some gave up". A third of the teachers were bothered by students' frustration when trying to cope with the changes; e.g., "The students did not understand what I wanted from them"; "The weak students felt even more abandoned than usual".
3. Class management challenges: Many teachers (41%) referred to class management challenges; e.g., "I had a hard time not knowing how the lesson would progress"; " I didn't know when and how to help the students". A third of the teachers referred to structural constraints that they faced, mainly the lack of equipment; e.g., "We have equipment only for the mandatory experiments "; "We have to be careful with the equipment, so we have to tell students exactly what to do", and lack of time; e.g., "It took much more time, so we had to continue during the next lesson and set up the experimental system again".
Despite these challenges, many teachers (44%) reported that as a result of their experiences during the year they intend to keep using RILs, and to give even more autonomy to their students in the future; e.g., "I plan to give the students less guidance next year, and to increase the 'degrees of freedom' as a function of the increase in class grades."; "I will continue to apply these ways of working in the lab, now that I have more confidence on how to go about it".
On the other hand, some teachers (18%) stated that they planned to somewhat curtail the freedom they gave students, mainly because of students' difficulties and frustration, or time constraints; e.g., "I think it is important, but when you open a lab even at a basic level and give students the opportunity to design things, it takes much more time, and we don't have that time". Others (16%) stated that they were planning various scaffolds to help students cope with the challenges; e.g., more explanations before the activity, hints during the activity, or different levels of autonomy to address students' heterogeneity. Some teachers (28%) discussed the need to change their instructional priorities in the future in order to promote lab practices and integrate more RILs.

V. DISCUSSION AND IMPLICATIONS
The physics teachers in the PLCs were receptive to making lab instruction more "science process oriented" 6 as shown in the learning goals they value and their dissatisfaction with the manifestation of these goals on the national lab exam. This finding echoes those of instructors in more open-ended contexts (i.e. upper division labs 15). The teachers acknowledged the disparities between the scientific practices that students engage in during the instructional lab and the practices of physicists in research labs, in particular with respect to experimental design. While the literature points to the difficulties in bringing about change towards inquiry-oriented labs in PD programs 16, the intervention proved successful in that half of the PLC teachers reported that they introduced the RILs in their lab lessons. They perceived the changes to be feasible even though they faced severe structural constraints 5. These findings suggest that modest restructuring of traditional labs, such as encouraging students to reflect on the considerations underlying the experimental design, makes it possible to comply with the tight constraints imposed by the setting in which the PLC participants operate. Involving teachers as students, teachers, and peer groups who collaboratively reflect on their experiences in a safe teachers' community allowed the teachers to try out new norms in their lab lessons.
The roles that teachers described having assigned to their students' spanned a wide range from limited students' engagement in a preliminary planning discussion to full autonomy in designing an experiment. However, the current research design did not include classroom observations that could clarify the extent of autonomy actually practiced. Nevertheless, the changes that so many teachers believed to have made in their instructional lab went far beyond expectations, considering the high-stakes setting in which they operate and the traditional step-by-step lab manuals they are accustomed to. Despite students' frustration and difficulties in classroom management that they faced, many asked to continue this line of PD.
The RILs intervention proved feasible, thus paving the way for a follow-up on this first step, to help teachers collaboratively construct scaffolds to meet the challenges of granting autonomy to students in terms of experimental design. However, the associated demands teachers faced, as shown in the extra time needed to enable student-driven experimental design highlights the importance of redesigning assessment policies to incorporate scientific inquiry practices.

VI. ACKNOWLEDGMENTS
We thank the TLs and the PLC teachers for their devotion to continuing professional development and for their contribution to this study.