The Role Self-Regulated Learning on Project-Based Learning to Natural Science Learning Outcome

Many factors influence the acquisition of learning outcomes, both external and internal factors. One internal factor that is thought to be very influential on the achievement of learning outcomes is Self-Regulated Learning (SRL), while external factors that are thought to contribute to the achievement of learning outcomes are the learning strategies teachers use this case, Project-Based Learning (PjBL). This research aims to assess the effect of PjBL and SRL on natural science learning outcomes. The subjects were 153 students from the Elementary Teacher Education of the Kanjuruhan University of Malang. The research design used a 2 x 2 factorial quasi-experiment. The instruments of data collection were questionnaires and tests. Validation of instruments using: (1) validity test of "expert judgment" content, (2) construct validity test using exploratory factor analysis, (3) reliability test using Cronbach Alpha with the result value is 0.941 (> 0.600). The data were analyzed using twoway ANOVA with α = 0.05. Findings: (1) There is a difference in science learning result between students who were studied using PjBL with students who were studied using regular strategy; (2) There is a difference of science learning result between the student with high and low SRL ability; 3) There is an interaction between PjBL and SRL on natural science learning outcomes. This study result is useful for improving the quality of the learning process by actively involving students. The activeness of students in learning both physically and mentally or emotionally is very important for learning participants' interests. Keywords—SRL, PjBL, natural science, learning outcome

I. INTRODUCTION
The achievement of high learning outcomes in students is always a target of every learning process. The learning outcomes, according to Reigeluth [1], consist of three dimensions: (1) The effectiveness of learning, (2) Efficiency, and (3) Learning attractiveness. Learning effectiveness is the acquisition associated with the learning objectives. Indicators of learning effectiveness include (i) accuracy of behavioral mastery, (ii) performance velocity, (iii) compliance with procedures, (iv) quality of performance, (v) quality of result, (vi) rate of transfer of learning, and (vii) retention rate. Learning efficiency refers to how much time, energy, and learning resources are used in learning. The attractiveness of learning is the interest and sincerity of students to desire to learn more. Teachers make various efforts to improve student learning outcomes, among others, by the implementation of learning methods that are considered by the objectives. Many factors influence the acquisition of learning outcomes, both external and internal factors. "SRL is a process that students use to focus their thoughts, feelings, and actions systematically to achieve goals" [2]. How the role of SRL in the learning process has been revealed by many experts and researchers, as summarized by Zumbrunn et al. [3]. SRL also helps students to create good learning habits and therefore could strengthen their skills in a study Wolters [4], enhance learning outcomes by applying new learning strategies [5] and lastly to evaluate their academic progress [6]. Furthermore, in SRL, motivation could be an important factor in students' academic outcomes [7].
PjBL has characteristics that allowed learners to be active, creative, productive, and collaborative. Through the constructive learning process, the learners can do learning activities using cognitive, affective, and psychomotor abilities and achieve optimal learning outcomes. Thus, PjBL is possible to support the acquisition of learning outcomes significantly. "In PjBL, the project is part of the teaching and learning strategy. Students explore and learn the concept and application through the project." [8].
"The students who were taught using PjBL strategy performed significantly better in terms of self-directed learning skills than did students who were taught using CT strategy" [9]. "PjBL was more effective in the positive development of the student's academic achievement levels. The students who were educated by project-based learning were more successful and had higher attitude levels towards the lesson than the students who were educated by the instruction based on student textbooks" [10]. "The most valuable from PjBL approach was to observe the production of new knowledge derived from students' own investigation and exploration" [11].
The integration between the use of PjBL method and SRL ability will be able to improve student learning outcomes. This is based on the PjBL characteristics that need independence and active involvement from learners in physically, mentally, and emotionally into their learning process.
This study aimed to determine the effect of SRL on the use of the PjBL method for learning elementary school science. The study design was a quasi 2x2 experiment with 153 prospective teacher students (four learning classes), two classes as a treatment group and 2 classes as a control group. The intake group does the placement of research subjects in the learning group. A different lecturer teaches each group. The research data collection instrument used was a test, questionnaire, and observation sheet. Research data were analyzed using a t-test and one-way ANOVA.
A. The Role of SRL to Learning Outcome "Self-regulation is a personality capacity that includes selfdirected motivation" [12]. Self-regulation plays a role in selfdirected learners to have independence in learning. A good self-regulated learner has a combination of academic learning skills and self-control that makes learning easier for them and makes them more motivated. "Students who have high ability in managing their learning can transform their mental abilities into academic skills and strategies" [13]. Many factors can influence the acquisition of a person's learning outcomes, both external and internal. One of the interesting and actual internal factors to examine is the ability of self-regulation. There is a clear goal in formal education namely: to improve and update skills, seek new knowledge, and solve existing problems in life. This is a significant factor for students to develop their abilities and be self-regulated as it has been proven to give a specific picture of the various components that affect success. Selfregulation is learning that occurs on the initiative. "Students who have the initiative demonstrate the ability to use their thoughts, feelings, strategies and behaviours that are shown to achieve goals" [7]. SRL deals with one's habit of discipline, self-control, and initiative in learning. Students whose initiative is high in learning activities will reveal active, creative, and dynamic behaviours in learning, thereby increasing their learning outcomes.
According to Friedman and Schustack "Self-regulation is the process of a person in managing the action and obtaining his own learning outcomes, determining the targets to be achieved, measuring his success, and rewarding himself for the results achieved" [14]. Pintrich, define "self-regulation as an active and constructive process, where students set their own learning goals, monitor, regulate their implementation, try to control their thoughts, enthusiasm, and behavior, guided and limited by the goals to be achieved and the context of their environment" [15]. Zimmerman views self-regulation refers to the "Process used by students to focus their thoughts, feelings, and actions systematically on achieving goals" [2]. Selfregulation requires students to have choices, for example, related to what will be done and how they will do it. "Selfregulation in learning can be seen as a mechanism to help explain differences in abilities among students and is a meaningful matter for the development of abilities" [16].
SRL also implies that a person can also avoid interference from his environment and emotional aspects that can interfere with his development. People regulate their behavior by setting personal goals and evaluating their behavior according to performance evaluation standards [12]. Bandura in Schunk mentions three stages in self-regulation [16]: (1) Selfobservation, how a person sees himself and his behaviour and continues to watch over him. (2) Assessment, as a process of comparing what is seen in themselves and their behavior with a certain standard size. (3) Self-response, the stage of giving a response to yourself in return for the assessment that has been done. Forms of self-response can range from very obvious, such as working hard or studying until late at night to implicit ones such as feelings of pride or shame. Zimmerman developed a self-regulation cycle model consisting of three stages: thinking, performance, and self-reflection [2]. At the stage of thinking organization, students plan and set their strategies to achieve goals. For example, students can make plans to complete a chapter or book. Students independently demonstrate the ability to control themselves through independent learning, focusing attention, and managing strategies in completing assignments in the performance phase. Students in this phase use background knowledge and selective strategies to understand and comprehend texts during a reading. This can also include working with others and having conversations to understand the text better. Finally, the selfreflection phase, students bring themselves back to evaluate and understand their performance. Did they complete what they planned to achieve, and what the next steps were [17]?
Research on self-regulation in learning begins with developing psychological studies into the development of selfcontrol by adults and children. Much of the research on selfregulation used to be carried out in a clinical context, where researchers taught participants to change dysfunction behaviors such as aggression, addiction, sexual deviations, personal conflicts, and behavior problems at home and school. Finally, at this time self-regulation has been developed to discuss learning and academic achievement [16]. Self-regulation involves someone's behaviour when someone regulates their behaviour to focus on achieving goals. Self-regulation also involves cognitive variables and other influential variables. In learning, self-regulation is beneficial for students to maintain their self-confidence in learning by believing that positive results will arise and maintain a positive emotional state (for example, feeling proud of the results achieved). Associated with process and self-regulation strategies applied by students have a variety in general (applied to various types of learning), and specifically (applied only to certain types of learning).
Conditioning by self-monitoring and external monitoring gives rise to higher self-efficacy, ability, and tenacity than conditioning without monitoring. The benefits of monitoring do not depend on the child's performance during the learning process. Monitoring progress, rather than evaluating it, will improve children's perceptions of their learning progress and self-confidence. Self-monitoring and, in combination with implementation, is often a key component of in effective interventions.

Advances in Social Science, Education and Humanities Research, volume 566
Some research result about SRL • Self-monitoring of the belief in achieving goals will maintain learning and increase achievement [2,18,19].
• Effective self-regulation develops self-efficacy for learning settings. Attitudes towards learning and strategies in dealing with tests have a strong influence on learning outcomes in mathematics.
• Students' SRL strategies influence 3.Scholastic performance of e-learning students more than other learning management systems [20].
• Through SRL in learning, teachers can provide students with information and opportunities and help them become strategic, motivated, and independent students.
• Pintrich believes that self-regulation activities bridge students' relationship and their environment and influence their performance [16].
• Student achievement at school shows that the triadic model of self-regulation has benefits in training students to become more effective learners

B. The Role of PjBL to Learning Outcome
Based on the concepts and findings of research results on PjBL and SRL, which both positively influence the acquisition of learning outcomes, the interaction of these two variables in learning can improve the learning outcomes. PjBL and SRL are conditions that can be built or developed within individuals through systematic education and learning efforts. Thus, the implementation of the PjBL strategy, accompanied by the efforts to optimize the learning ability to learn in learning, will have a positive effect on learners' learning achievement. In line with the concept and the findings of research findings on PjBL and SRL, which both positively influence the acquisition of learning outcomes, the interaction of the two variables in learning can increase learning outcomes. PjBL and SRL are conditions that can be built or developed within individuals through systematic education and learning efforts. The implementation of PjBL strategy which is accompanied by efforts to optimize the high SRL ability of students, will positively influence the acquisition of student learning outcomes.
According to the social cognitive theory of Bandura, people want to control events that affect their lives and see themselves as perpetrators. In the conception of consciousness as the agent of the most important thing, the self-regulation or process in the individual activates and maintains behavior, cognition, and influence, systematically oriented toward attaining goals [19]. Self-regulation requires students to have choices, for example related to what to do and how they will do it. Self-regulation in learning, can be seen as a mechanism to help explain differences in students' abilities and is a meaningful thing for the development of their abilities [16].
PjBL is a model of learning organization in the form of project activities. The project in question is complex tasks based on challenging questions or problems involving students in design, problem-solving, decision making, or in research activities. PjBL is a systematic learning strategy that involves students in learning knowledge and skills through complex inquiry processes, authentic questions, work design, and product acquisition [21]. PjBL provides students opportunities to work more independently for a long time and end up with a real product or a presentation. Through PjBL students occupy themselves with a variety of skills to open the way for metacognitive development that can last a long time. Characteristics of PJBL are: (1) Enriching and developing the ability to learn to form new knowledge. (2) Gaining the opportunity to actively participate in the learning process, conduct in-depth studies, and produce something. (3) Following the learning according to different bits of intelligence. (4) Provide opportunities for students to use their learning methods such as learning by reading, studying independently, or discussing with groups. (5) Seeing ways of learning that are different from others. This allows them to learn through the application. (6) Provide meaningful information to parents of students about changes in their learning performance.
PjBL is implemented by organizing students in a team to solve problems while producing a product. Important concepts in team learning methods are team appreciation, individual responsibility, and the same chance of success. Responsibility is focused on team members' activities in helping one another learn and ensure that each is ready to do their work.
Project-based learning strategies contribute to the development of the following skills and traits: 1) Learning skills with groups. 2) Cognitive process skills (decision making, critical thinking, and problem-solving skills). 3) Selfgovernance (planning activities, setting goals, and completing tasks). 4) Skills in managing attitude (motivation and desire to practice). 5) Self-tendency (self-directed, desire to excel). 6) Confidence (independence).
PjBL has the following characteristics: (1) A creative process for obtaining problem-solving solutions based on a specific need or problem. (2) A varying stage of natural development is needed to design solutions, for example, the accuracy of problem definition, solution requirements, and constraints; generation of alternative solutions and their evaluation by criteria; building model. (3) Requests for various skills related to different functions, such as information retrieval and retrieval; represent ideas using formal notation; build physical models. (4) Requests for collaborative work skills, for example, distribution of functions according to expertise; parallel and cooperative work. (5) There is a continuous evaluation of the product produced at each stage and the solution obtained.

II. METHODS
This study's subjects were prospective elementary school teacher candidates from Kanjuruhan University Malang who were taking elementary science courses totaling 153 people who were selected by purposive sampling technique. The research subjects were divided into four classes, namely classes A, B, C, and D. Furthermore, the four classes were drawn using lottery paper rolls to determine which classes were the experimental and control groups. The drawing process was carried out jointly with the two lecturers who taught natural science learning subjects in the odd semester of the 2015-2016 academic year. From the draw, the results of class A (44 students) and class C (40 students) were selected as the experimental group, while class B (40 students) and class D (39 students) were selected as the control group. Determination of the number of classes and students in each class as lecture participants is determined by the academic administration department based on students' order in re-registering and the course programming process at the beginning of the semester without any other subjective considerations.
There are three variables of this research: (a) one independent variable (learning strategy) consisting of two dimensions, namely PJBL and regular learning strategy; (B) one moderator variable that has two dimensions of SRL that is, high SRL capability and low SRL capability; (C) one dependent variable that is, the result of natural science learning of the student. In accordance with the research design concept of Gall et al. [22], this study was designed using a quasiexperimental design of 2 x 2 factorial. Research subjects, both experimental and control groups, were not randomly selected but using subjects grouped into in the course class as it is. The data collection instrument used is the learning result test and the SRL capability measurement questionnaire. Test the validity of the instrument with content validity test, and construct validity. The learning result test is validated by content validity test through "expert judgment," while the SRL measurement questionnaire is validated by content validity test and construct validity using exploratory factor analysis. The result data were analyzed by (1) free sample t test, and (2) Analysis of two-way variant. In addition, descriptive analysis is also conducted to obtain a description of the students' initial knowledge.

A. Student Initial Knowledge
Statistical data that illustrates the initial knowledge of students before the treatment can be seen from the results of the descriptive test and t-test, whose results are presented in Tables  1 and 2 below:   TABLE I Table 1 shows that the results of learning science in the two groups of students in the experimental and control groups are relatively the same. The mean values were 68.76 and 68.72, respectively. This shows that the initial abilities of students before treatment are the same.
Testing of differences in the initial ability of students in the experimental group and the control group before treatment using the test results in table 2. A requirement before analysis of the results of the free sample t-test is to test the variants' similarity. The variance similarity test results with Levene's test obtained F count with Equal variance assumed was 0.497 with a probability of 0.482. The probability value is> 0.05, meaning H0 is accepted, or it can be stated that the two variances are not different. Based on the results of the similarity test of the variance, the analysis of the results of the t-test was continued by using the Equal variance assumed basis (assumed both variances are the same). The free sample t-test presented in Table 2 obtained t values of 0.074 with a probability of 0.941. In a two-sided test, the probability is 0.941 / 2 = 0.470 (> 0.025) means that there is no difference in initial knowledge between the experimental group students and the control group. This can be interpreted that the mastery of elementary science by the two groups of students before treatment did not significantly differ.

B. Description of Student Learning Outcomes After
Treatment Descriptive data of student learning outcomes obtained after treatment (post-test) is presented in table 3.  Table 3 shows that when viewed from the SRL ability in the two categories, the experimental group's mean learning outcomes were higher than those of the control group. In the high SRL category, the mean value of learning outcomes achieved by the experimental group was 82.31, while the control group was 71.91. In the low SRL category, the mean value of the experimental group learning outcomes was 75.57, and the control group was 70.30. The interesting thing is that the mean score of learning outcomes in the experimental group with low SRL is 75.57, which is higher than that of the control group with a high SRL of 71.91. This can explain that the learning strategy applied to the experimental group provides higher learning outcomes than the conventional learning strategy applied to the control group. Two-way ANOVA test results, whose data are presented in Table 6 regarding the interaction between the PjBL strategy and self-regulation on learning outcomes, obtained the coefficient F is 9.388 and the significance of 0.003. Significance of 0.003 (<0.05) means that H0 is rejected, or it can be stated that there is a significant interaction effect of using the PjBL strategy with the ability to self-regulate learning outcomes. It can be interpreted that the use of PjBL strategies in students with high self-regulation abilities will provide high learning outcomes. Students' self-regulation ability will strengthen the connectivity of the use of PBP strategies with learning outcomes.

Advances in Social Science, Education and Humanities Research, volume 566
The results of the descriptive analysis presented in Table 4 can reinforce the intended explanation. The mean value of science learning outcomes of experimental group students (taught with the PjBL strategy) with high self-regulation ability was 85.32, and the mean score of students in the experimental group with low self-regulation ability was 75.57; The mean score of control group students (taught with conventional strategies) with high RD ability was 71.91, while the mean value of control group students with low RD ability was 70.30.
An interesting fact from this study results is that the average score of learning outcomes of students who have low selfregulation but learned using PjBL strategies is still higher than students who have high self-regulation but are taught using conventional strategies. This situation reinforces the notion that the PjBL strategy can increase student learning outcomes. Students with any self-regulation ability turned out to use the PjBL strategy to improve their learning outcomes. The condition is suspected because PjBL strategy has advantages in learning interactions between lecturers and students and between students and other students. For example, PjBL-based learning interactions always provide wider opportunities for students to explore their learning. Students are more flexible in finding and exploring information that can strengthen their mastery of various concepts they learn.
In contrast to students who are taught using conventional methods, students who have high self-regulation abilities, mean scores of learning outcomes are lower than students who have low self-regulation abilities who are taught with PjBL strategies. It is suspected that conventional learning does not provide opportunities for students to explore their abilities. Learners tend to be passive participants, waiting for instructions, and are always forced to accept learners' learning materials. Considering that students have often been dealing with similar learning strategies since elementary school, students tend to get bored with such monotonous learning.
The findings of research results related to the proof of the research hypothesis 1 show that there are differences in learning outcomes between students who were studied using the PjBL strategy with students who were taught using regular strategies. It can be interpreted that PjBL strategy has a plus value compared with the regular learning strategy. PjBL strategy's advantages can be learned from the concept and its application in building learning independence in the learners. Several factors are suspected to affect the improvement of student learning outcomes through project-based learning [13]. Woolfolk argues that learning is an active process of learning by not only accepting knowledge but actively choosing, practicing, caring, ignoring, reflecting, and making many other decisions to achieve its goals. According Reigeluth and Carr-Chellman [23], learning through experience is learning centered on learners, not placing learners as passive recipients, but as an active negotiator of their experience. In harmony with this thinking, PjBL strategy is expected to facilitate learners more actively, pay attention, choose, make decisions, reflect, and seek experience.
The research results related to research hypothesis 2 indicate that there are differences in learning outcomes between students who have high SRL ability with students who have low SRL capability. Students with high SRL capability can Advances in Social Science, Education and Humanities Research, volume 566 generally achieve better learning outcomes than students with low SRL capability. Students with high SRL skills tend to be more independent in learning. Students are no longer too dependent on other parties in learning and can determine for themselves about various things related to learning. Students with high SRL are thought to be better able to direct themselves to meaningful learning activities for themselves. They tend to be more able to determine what they need to do to achieve their learning goals. The condition is very different from the students whose low SRL ability is suspected of tending to do learning activities based on certain directions or directives. Students with high SRL capabilities are thought to have more positive thinking about their learning, so when they are exposed to a creative and challenging learning process they are more interested. Students with high SRL capability tend to feel more interested in learning using the PjBL method, making them more actively involved in the learning process. The situation is in line with the opinion of [13]" that learners who have high ability in managing their learning, can transform their mental abilities into academic skills and strategies". According to Zimmerman [7], students are said to have used self-regulation when the student has a strategy to activate metacognition, motivation, and behavior in their own learning process.
Findings of research results related to research hypothesis 3 indicate an interaction effect of PjBL strategy and SRL ability on student learning outcomes. The fact that is quite interesting from the results of this study is that the average score of student learning outcomes with low SRL but learning by PjBL strategy is still higher than students who have high SRL but learning by using conventional strategy. This situation reinforces the notion that the PjBL strategy can improve the achievement of student learning outcomes. Students with any level of SRL ability turn out the use of PjBL strategy can improve learning outcomes. The condition is suspected because the strategy of PjBL has advantages in the interaction of learning either between lecturers and students and between students and other students. For example, in PjBL-based learning, interaction always provides wider opportunities for students to explore their learning. Students more freely in searching and digging information that can strengthen his mastery of various concepts he learned.
Unlike conventional students, students with high selfregulation ability and mean scores of learning outcomes are lower than those who have low self-regulation ability studied with PjBL strategy. It is presumed that less conventional learning allows learners to explore their abilities. Learners tend to be passive participants, waiting for instructions, and constantly forced to accept the learner's subject matter. Given that students have often faced similar learning strategies from elementary school, they tend to be saturated with such monotonous learning.
Associated with SRL students' ability, it can be interpreted that PjBL will be more effective if taught to students who have high self-regulation skills, thus becoming a recommendation for learners to strengthen the ability of self-regulation in each learning. Thus, SRL can be said is a prerequisite for the effectiveness of the use of strategies PjBL and a perfect match in learning. Zimmerman mentions that "Self-regulation is important for learners because the main function in education is the continuous development of learning skills for the rest of his life" [7].

IV. CONCLUSIONS
The research conclusions are: (1) There are differences in the results of science learning between students who are studied using PjBL strategies with students who are taught using regular learning strategies. (2) There is a difference in science learning results between students who have high SRL capability with low SRL capability. (3) There is an interaction of the PjBL strategy and the ability of SRL to the student learning outcomes.


s2fieldsofstudy: Education Physics

text:
Evolution in student conceptual understanding of electricity and magnetism

The Conceptual Survey of Electricity and Magnetism (CSEM) is a multiple-choice survey that contains a variety of electricity and magnetism concepts at the level of introductory physics used to help inform instructors of student mastery of those concepts. Prior studies suggest that many concepts on the survey are challenging for introductory physics students and the average student scores after traditional instruction are low. The research presented here compares the performance of students in introductory, upper-level, and graduate-level physics courses on ﬁve CSEM questions to understand the cross-sectional evolution of student understanding of these concepts from the introductory to upper-level to graduate-level. We discuss ﬁve CSEM questions that remain challenging for many upper-level and graduate students.

I. INTRODUCTION AND GOAL
One goal of an introductory electricity and magnetism course for physical science and engineering majors is for students to develop a robust conceptual understanding of the underlying physics [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15]. This remains one of the goals in courses for physics majors as well as for physics graduate students as they take advanced physics courses even though their courses use complex mathematics to solve problems [16]. In these courses, students are often expected to make the mathphysics connection themselves to develop a good knowledge structure without much scaffolding support from instructors [17,18]. In particular, the focus of upper-level undergraduate and graduate courses is often on being able to solve complex quantitative problems with course assessment focusing exclusively on such problems, which students can do without a robust understanding of the underlying concepts.
In the context of electricity and magnetism (EM), students learn about the basic concepts including the Maxwell equations in integral form in the calculus-based introductory courses. However, in the upper-level undergraduate and graduate EM courses, students are taught to use differential forms of the Maxwell equations to solve complex problems often without any course assessment focusing on conceptual understanding. If students are not given grade incentives or provided support to make the appropriate math-physics connection, they may be able to solve problems in advanced courses using algorithmic approaches by pattern matching without developing a deep conceptual understanding [17,18].
Many validated conceptual multiple-choice assessments have been developed to evaluate student understanding of basic concepts covered in introductory physics [19][20][21][22][23][24][25][26]. The research reported here compares the conceptual understanding of upper-level undergraduates and first year graduate students at a large research university with that of introductory students in calculus-based physics courses using a large national data on a validated EM survey suitable for introductory physics called The Conceptual Survey of Electricity and Magnetism or CSEM [20]. Our goal here is to investigate the extent to which advanced students have mastered these concepts compared to introductory students and what difficulties persist even for advanced students.
The CSEM has been used widely to investigate the conceptual understanding of students in introductory EM courses before and after instruction in relevant concepts. It is a 32question multiple-choice survey in which EM topics range from Coulomb's law to Faraday's law. The CSEM can be used to measure the effectiveness of evidence-based curricula and pedagogies. Past research using other surveys has shown that even in upper-level undergraduate classes, students have difficulty with many introductory and advanced level concepts [27,28].
Here we use the CSEM to study the persistent difficulties of upper-level undergraduates and graduate students with EM concepts measured by the CSEM. In this paper, we discuss five questions from the CSEM that were challenging for undergraduates after instruction in upper-level EM. Our re-search questions are as follows: RQ1: How difficult are these CSEM questions for the upper-level undergraduate and graduate students compared to the introductory students based upon their performance?
RQ2: What are the common difficulties of advanced students based upon the alternative choices they commonly selected? Are they the same or different from the alternative choices selected commonly by the introductory students?

II. METHODOLOGY
We use the CSEM to compare the EM conceptual knowledge of students in calculus-based introductory physics courses before and after instruction with students in a required upper-level physics undergraduate EM course before and after traditional instruction, and with graduate students enrolled in a teaching of physics course before their first-year physics graduate EM core course. Introductory students and upper-level undergraduate students took the CSEM before (pre) and after (post) instruction in relevant concepts, whereas the graduate students only took it as a pretest. The introductory students took a calculusbased course in their first or second year. Data were collected over one semester in large-enrollment classes. Students took the CSEM both as a pre-test within the first week of classes and as post-test in the last few weeks. The upper-level EM course is required for physics majors and is usually taken by students in their second or third year. This course used Griffithâs Introduction to Electrodynamics as the textbook. Data for this course were collected over 3 different years and averaged together. Students took the CSEM both as a pre-test within the first week of classes and as post-test in the last few weeks. The graduate class consisted of first year physics graduate students in a Teaching of Physics class that prepared them for being teaching assistants. Data were collected from 4 different years and students took the CSEM as a pre-test, prior to their graduate EM instruction. All students used paper scantrons to bubble their responses for each question and had approximately 50 minutes, which was sufficient to answer all questions on CSEM. All three levels of students were from the same large research university.
The five questions from the CSEM to be discussed here are questions 13, 14, 21, 29, and 31 and they are shown below to give clarity to the underlying concepts. The answers for each question are bolded, italicized, and underlined. 13. Figure 1 shows a hollow conducting metal sphere which was initially given an evenly distributed positive (+) charge on its surface. Then a positive charge +Q was brought up near the sphere as shown. What is the direction of the electric field at the center of the sphere after the positive charge +Q is brought near the sphere? 14. Figure 2 shows an electric charge q at the center of a hollow uncharged conducting metal sphere. Outside the sphere is a second charge Q. Both charges are positive. Choose the description below that describes the net electrical forces on each charge in this situation. (c) There is no force on Q but a net force on q.
(d) There is no force on q but a net force on Q.
(e) Both charges experience a net force but they are different from each other. 21. What happens to a positive charge that is placed at rest in a uniform magnetic field? (A uniform field is one whose strength and direction are the same at all points.) (a) It moves with a constant velocity since the force has a constant magnitude.

III. RESULTS AND DISCUSSION
There were 456 students who took the introductory level pre-test (Intro-pre), 401 students who took the introductory level post-test (Intro-post), 93 students who took the upperlevel pre-test (UL Pre), 85 students who took the upper-level post-test (UL Post), and there were 87 students in the graduate level class (Grad). We present data from all upperlevel students who took the pre/posttests because using only matched data for pre/posttest yields very similar outcomes. Table 1 shows the distribution of mean percentages for answer choices for the five CSEM questions with the correct answers bolded, italicized, and underlined. Percentages for a given question were calculated from students who did not leave that question blank (since some students skipped a few questions on the survey). Question 13 is the first of two questions that investigated student understanding of electric field and force on a charge inside of a conductor. The situation involves a charge distribution on the outside of a conducting sphere and a positive charge to the right of the sphere. In order to answer the question correctly, students must recognize that the conductor will rearrange the charges on the outside surface to create zero electric field inside it in equilibrium. Table I shows that at 54% of introductory students and 62% of upper-level undergraduates after instruction chose the correct response to question 13. At the graduate level, 77% of the students provided the correct response.
Question 14 builds on question 13 and evaluated student conceptual understanding of the electric force on a charge inside of a hollow conductor. In order to answer correctly, students must realize that in equilibrium, there is no electric field due to outside charges inside of a conductor. The outside charges induce charges on the surfaces of the conductor to make the net electric field zero due to outside charges. In particular, students must realize that since the net electric field from the outside sources at the point where charge +q is located is zero, there would be no net force on +q. However, since charge +Q is outside the conductor, it does experience the electric force due to +q. Table I shows that this question was the most challenging of the five questions and less than 45% of any group of students provided the correct answer.
Contemplating student responses to questions 13 and 14, we infer that in the contexts provided, while students realized that there is no electric field inside a charged conductor, they did not realize that there would be no force on a charge inside a conductor due to the charge outside. For example, 77% of graduate students correctly answered question 13 about the electric field inside of a conductor, but only 38% of them answer question 14 correctly. The most common incorrect answer was choice (a), i.e., the net force on each charge is equal in magnitude and opposite in direction. Thus, students used Newtonâs 3rd law type reasoning to answer the question, which would have been correct if there was no conductor. It appears that even advanced students struggle to reason about how induced charges on the surface of a conductor shield the inside hollow region from charges outside.
Question 21 focuses on a charge at rest in a magnetic field. In order to answer the question correctly, students must recognize that in a uniform magnetic field, if a charge is initially at rest, it will continue to be at rest since there is no magnetic force on it. Table I shows that 17% of introductory students and 57% of upper-level students provided the correct response after instruction in relevant concepts, and 69% of the graduate students answered this question correctly before instruction in graduate EM. The most common incorrect choice among upper-level undergraduates and graduate students was (c), i.e., the particle would move in a circle with a constant speed since the force is always perpendicular to the velocity. This alternative conception shows that these students did not realize that a charged particle moves in a circle with a constant speed in a uniform magnetic field only if the initial velocity of the particle is perpendicular to the field. In other words, students must recognize that a charged particle experiences a non-zero magnetic force only when it is moving at an angle other than 0º or 180º to the magnetic field.
Question 29 shows different situations in which students must determine if a lightbulb will light up due to induced current or not. In particular, students must recognize that the lightbulb in the loop of wire will light up when a current flows through the wire (in this case, an induced current flows due to an induced emf created by the changing magnetic flux through the loop). Students should realize that the flux through the loop is a measure of the number of magnetic field lines passing through its surface area and a collapsing loop would mean a decrease in area and a change in flux. Students must also recognize that along the axis of a bar magnet, the strength of the magnetic field is strongest near the poles and weaker far away so that when the distance between the loop of wire and the bar magnet decreases, the strength of the magnetic field passing through the loop would increase and vice versa. The correct answer is choice (c). Only 34% of the introductory students and 56% of upper-level students after instruction answered the question correctly. Also, only 61% of graduate students selected the correct response. The most common incorrect choice is (b), which shows situations I and IV, leaving out II, in which the loop is collapsing. Table 1 shows that 25% of introductory, 21% of upper-level students after instruction, and 25% of graduate students did not realize that the light bulb would light up in case II in which the loop collapses. This suggests that these students did not realize that collapsing the loop in this case will lead to a decrease in flux through the loop and hence cause an induced current.
Question 31 involves the concept of motional emf associated with a metal bar moving in a uniform magnetic field and asked students to choose the correct charge distribution on the metal bar among the five choices. In order to answer the question correctly, students must recognize that since the metal bar has free electrons, the right-hand rule can be used to find the direction of the force on the electrons when they move to the right (in the moving bar shown in Fig. 4) and the magnetic field points out of the page. In particular, they must realize that the downward force would act on the positive charges, and an upward force on the electrons. Thus, the emf induced would have positive charges accumulating on the bottom and the negative charges accumulating on the top of the metal bar as shown in choice (e) in Fig. 5. Only 18% of introductory students, 58% of upper-level students after instruction, and 68% of the graduate students selected the correct response. The most common incorrect answer for graduate students is tied between options (a) and (d), with 14% of students selecting each choice at the graduate level. The incorrect choice (d) shown the opposite charge separation to the correct answer and may signify difficulty with using the right-hand rule correctly in this context. The incorrect option (a) shows an uncharged bar, which signifies that the students who selected this answer thought that a neutral metal bar moving in a uniform magnetic field would not have any charge separation. Although this situation of a motional emf produced in a metal bar moving in a uniform magnetic field is routinely discussed especially in introductory physics courses, it is possible that at least some of these students who selected option (a) thought that an emf would only be produced if this metal bar was part of a complete circuit. In other words, they may have confused the fact that in the situation in question 31 there would be an induced emf even though an induced current would require a complete circuit.

IV. SUMMARY AND FUTURE DIRECTIONS
We investigated the difficulty on five CSEM questions for the introductory and upper-level undergraduates and graduate students at a large research university in the US and compared their performance. We also investigated the common difficulties of advanced students based upon the alternative choices they commonly selected and analyzed whether they were the same or different from the alternative choices selected commonly by the introductory students. We find that many upperlevel physics undergraduates after instruction in relevant concepts in their upper-level EM course and first-year graduate students before instruction in graduate EM struggled with the underlying concepts in these questions that are designed to be at the level of EM covered in introductory courses. Furthermore, analysis of upper-level undergraduate students' performance on all CSEM questions after instruction (to be discussed elsewhere in detail) shows that if introductory students did not reach 50% correct on a question after instruction, less than two thirds of the upper-level students provided the correct response for that question after traditional instruction.
These findings make us contemplate whether the manner in which the traditional upper-level undergraduate EM course is being taught is achieving its goals or whether it is a missed learning opportunity. In particular, upper-level students may become proficient in solving complex problems algorithmically but without an effort to also develop a deep conceptual understanding of the underlying content simultaneously, they are not likely to retain much of what is taught.
We have begun to interview upper-level course instructors and shared these data with them to understand their perspectives on these issues. For example, we asked some instructors who have taught these upper-level courses what these findings mean to them and whether they would make more effort to integrate conceptual and quantitative aspects of physics both in their lectures and on their course assessments since assessment drives learning. Preliminary findings suggest that some instructors feel that they already discuss physics concepts throughout in these advanced physics courses, but they do not assess students on their conceptual understanding since they expect them to have learned the underlying concepts in their introductory courses. Interviews suggest that some instructors feel that the goal of the advanced physics courses is to help students learn to use sophisticated mathematics to do complex problems and it is the students' responsibility to reflect upon how it connects with the underlying concepts. Some acknowledged that they expect that students would recognize that it is important for them to build on their prior knowledge of relevant concepts in order to retain what they have learned in their advanced courses. They noted that while they did not have the time to contemplate how to integrate conceptual and quantitative aspects of the content in their teaching and assessment, if others had developed materials, they would be willing to try them in their classes. The persistent difficulties of advanced students such as those discussed here should be taken into account to improve their learning.


Core Competencies for Interprofessional Collaborative Practice Among Teacher Education, Health and Social Care Students in a Large Scaled Blended Learning Course

Introduction Society’s demands for better coordination of services for children are increasing. Interprofessional learning (IPL) has been suggested to achieve the triple aim of better services, better outcomes and reduced costs. The aims were to assess 1) to what extent students taking teacher education, health and social care programmes agreed that blended learning was a suitable approach in a mandatory IPL course, 2) to what extent they had learnt about the WHO’s core IPL competencies (roles and responsibilities, values and ethics, interprofessional communication, and teams and teamwork), and 3) the students’ ranking of the learning outcomes from different components of the IPL course. Methods This was a quantitative cross-sectional study. Students completed an online course evaluation after a two-day combination of online and face-to-face IPL small-group training. Findings The response rate was 25.8% (n=363). Among the students, 60.6% strongly agreed that blended learning was suitable, while 8.9% strongly disagreed. Among the respondents, 46.8%, 50.2%, 56.8% and 62.3% gained increased insight into roles and responsibilities, values and ethics, interprofessional communication, and teams and teamwork, respectively. In ascending order, students were most satisfied with the learning outcomes from the supervision (16.0%), the syllabus (28.6%), the submission assignment (42.4%), the digital learning content of Canvas (43.8%), the combination of everything (43.8%), and the IPL group discussions (78.6%). In stratified analyses, ‘teacher education and child welfare students’ were significantly more likely to gain better insight into the WHO competencies than “health and social care students”, and they were also more overall satisfied. Conclusion Students agreed that blended learning was a suitable approach, although the learning outcomes from the face-to-face discussions were markedly higher than from other course components. While the majority had learnt something about the WHO competencies, the teacher and child welfare students achieved the best learning outcomes, including new knowledge about the WHO competencies.

Introduction
There are no except number on how many children and young people who are living with a complex physical and/ or mental health condition, nor of those who are growing up as relatives to parents or siblings who are sick, dying, imprisoned, etc. As such vulnerable children and young people enter adulthood, they may encounter more health challenges than those commonly associated with the transition into adult life. [1][2][3] Educational achievement and completion of schooling are basic components of the healthy development of children and youths. 4,5 However, the school teachers need skills and expertise beyond the pedagogical to accommodate the diversity of pupils backgrounds and needs, and such expertise is currently lacking in teacher education. 4,6 Currently, students from education, health and social study programs, are mainly educated in educational trajectories which harbour different disciplinary professional identity, culture, tradition, syllabus 4 etc., all of which may act as barriers for professional collaboration and teamwork.
Interprofessional learning (IPL) is suggested to be an important pedagogical approach for achieving the triple aim of better services, better outcomes and reduced costs. [7][8][9] IPL is defined by the World Health Organization (WHO) as being present when students from two or more professions learn about, from and with each other in order to prepare them for interprofessional collaboration (IPC) and improve outcomes in the welfare services. 10,11 In response to the need for IPL, education associations from dentistry, medicine, nursing, osteopathic medicine, pharmacy, and public health formed the Interprofessional Education Collaborative (IPEC). 12 They developed 38 competencies for interprofessional practice in four domains: Values and Ethics, Roles and Responsibilities, Interprofessional Communication, and Teams and Teamwork. 12 To create a collaborative practice-ready workforce, health, social and educational professions educators must help students develop the knowledge, skills, attitudes, and behaviors relevant to the WHO core IPL competencies 7,11 well before these professionals enter childhood settings in the working field. 13,14 IPL must include interprofessional student groups working together in real-life situations, including situations that extends beyond everyday normal life, such as domestic violence, abuse, and deaths. [14][15][16][17][18][19][20][21] However, such issues and the WHO core IPL competencies are rarely addressed on the professional curricula.
One method for students to learn to function interprofessionally is through working in small IPL groups resembling IPC teams in specially prepared pre-service-training. Students may prepare themselves using the flipped classroom method ahead of participation in IPL groups. 22 Blended learning is a type of education in which students learn via electronic and online media as well as traditional face-to-face teaching. 23,24 Face-to-face interaction provides the foundation for social communication, which can be critical to online learning. 23 A case-based blended learning approach in small groups has been suggested to be a useful strategy for facilitating IPL since it integrates components of traditional face-to-face discussions with online learning. 8,23,25,26 However, IPL research is as yet mainly restricted to the field of health and social care, 13,27 and few published studies have examine outcomes related to the WHO core IPL competencies. 7 Additionally, there is a knowledge gap concerning potential differences in learning outcomes from IPL courses that include students not only from health and social care programmes, but also students from teacher educations and child welfare.
The aims of this study were to assess 1) to what extent the pre-service students taking teacher education, health and social care programmes agreed that blended learning was a suitable approach in a large-scale mandatory IPL course, 2) to what extent they had learnt about the WHO's core IPL competencies, and 3) the students' ranking of the learning outcomes from different components of the blended learning course.
(INTERACT) project 28 at Oslo Metropolitan University (OsloMet) in Norway. The IPL course was designed for bachelor students taking health, social care and teacher education study programmes. 29 The aim of INTERACT is to meet society's demand for better coordination of services relating to children and young people, involving better interaction between professionals and better cooperation between children, young people and their families, and the professionals.

Students
The first year bachelor students were enrolled in the following study programmes (n=1410): Early Childhood Education (n=250), Primary and Lower Secondary Teacher Education (n=380), Teacher Education in Art and Design (n=60), Physiotherapy (n=150), Mensendieck Physiotherapy (n=85), Nursing (n=160), Social Work (n=150), Child Welfare (n=90) and Occupational Therapy (n=85). The nursing students were based at the Kjeller campus (20 km outside Oslo city), while the others were based at the Pilestredet campus in Oslo city. The enrolled students were divided into pre-defined IPL groups each consisting of eight students representing health, social care and teacher education programmes.

Blended Small-Group Learning Course
The INTERACT project and the development of the largescale blended learning IPL course is described in detail elsewhere. 28 The required coursework included participation in a two-day seminar (working in the IPL groups only) and the submission of an IPL group assignment. The seminar days were structured as a combination of face-to-face IPL group discussions and the use of digital learning materials provided by the learning management system (LMS) Canvas. The latter included case-based learning material (produced by user organisations, employers and public authorities) and mini-lectures (produced by staff et al from the working field). The group assignment aimed to link the seminar days, coursework assignments and the syllabus, and could be submitted in the form of an academic text, a podcast or a video. The supervisors, who were recruited from among the staff, master students and professionals working in the field, visited the IPL groups during the second day of the seminars. The supervisor either approved or failed the groups' coursework and provided each group with feedback on their assignments. The digital content was available through LMS Canvas from mid-December 2019, so that the students could voluntarily prepare themselves before the IPL course. 30 The provision of the 2020 IPL course was repeated on the basis of the 2019 IPL course, and the course's development and provision are described in detail elsewhere. 22,31 Online Evaluation Survey Students were invited to complete a specially prepared online student-evaluation questionnaire after the course. In the questionnaire, most of the questions were identical to those in the survey from 2019, 22 except for additional questions about the WHO's four interprofessional core competencies; values and ethics, roles and responsibilities, interprofessional communication, and teams and teamwork and a few more questions regarding the blended learning approach. The questionnaire was provided as an internet link embedded in LMS Canvas. One reminder was sent. The responses were scored on a scale from 0 ("completely disagree") to 5 ("completely agree").

Data Analysis
The data were presented as frequencies and percentages. The share of students "strongly/completely agreeing" (score 4/5) and share of students "strongly/completely disagreeing" (score 1/0) on various items of questionnaire was presented. For the stratified analyses, the age was dichotomized to less than 25, or 25 years or older, and study programme to "teacher education and child welfare" consisting of Early Childhood Education and Care; Primary and Lower Secondary Teacher Education; Child Welfare; and Teacher Education in Art and Design, since they only target children and young people as end users, and "health and social care" consisting of Physiotherapy; Mensendieck Physiotherapy; Nursing; Social Work; and Occupational Therapy, as they target all age groups as end users. The association between the students' answers on scale 0-5 and dichotomized age and study programme was assessed by using a χ 2 -test. As post hoc analyses, the χ 2 -test was applied to assess the differences between the groups defined by age and study programmes and share of students "strongly/completely agreeing". All tests were two-sided, and results with p-values of <0.05 were considered statistically significant. No adjustment was made for multiple hypothesis testing due to the exploratory nature of the study. The statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS) v25.

Ethics
Ethical Guidelines for Research at Oslo Metropolitan University (OsloMet) were followed. 32 Further, the study was discussed with the Norwegian Centre for Research Data (NSD), 33 and they replied that it was unnecessary to report the study to them since if it was completely anonymous and no sociodemographic information beyond the participants' age and gender was collected, as was the case in the current study (NSD reference number 741649). None of the participants were under the age of 18 years. The data were collected through an anonymous web survey using "Nettskjema", 34 in line with ethical guideline. 32 The participants were provided written information about the study on beforehand in LMS Canvas. The voluntariness and anonymity of the participants were emphasized, and the participants were informed about the purpose of the study and how the data would be used. Answering the questionnaire was considered informed consent to participate. The study complies with the Declaration of Helsinki.

Findings
Students (n=363) from all of the included study programmes answered the evaluation questionnaire, with a response rate of 25.8% (Table 1). More than two-thirds (77.3%) of the respondents were under the age of 25. Fifty-four percent of the students were taking teacher education and child welfare study programmes, and 46.0% were taking health and social care study programmes.

Evaluation of Blended Small-Group Learning as a Learning Approach
In ascending order, students were most satisfied ("strongly/completely agree") with the learning outcomes from the supervision (16.0%), the syllabus (28.6%), the submission assignment (42.4%), the digital learning content of Canvas (43.8%), the combination of everything (43.8%), and the group discussions during the seminar days (78.6%) ( Table 2). The findings did not differ according to age or study programme (data not shown).
Among all students, 60.6% agreed that blended learning was a good approach for this large-scale IPL course (Table 3). In contrast, only 8.9% disagreed with this statement. Among the students, 49.1% agreed that blended learning was better than traditional plenary lectures, while 15.7% disagreed with this statement. Only 7.7% agreed while 73.3% disagreed that the learning outcomes would have been higher if the group discussions had been virtual instead of face-to-face.
The vast majority of students (83.4%) agreed that the collaboration in the IPL groups was good, and only 2.8% disagreed. Less than 10.0% disagreed that the learning objectives were clear, that there was a clear relationship between the learning objectives and assignments, that the digital content in LMS was well organised, and that the learning resources, syllabus and discussions were relevant. In contrast, 35.6% disagreed that the supervision was relevant, 27.2% disagreed that the practical organisation on campus was good and 44.9% disagreed that the information provided ahead of the IPL course was good.
In the analyses stratified by age (76.7% were below 25 years), there were no significant associations between the overall students' responses and age (data not shown). However, post hoc analyses showed, that younger students agreed to a lesser extent than older students that the syllabus was relevant (40.1% vs 52.4%, respectively, p=0.048), but to a larger extent that the learning approach worked well (63.4% vs 51.2%, respectively, p=0.046). In contrast, students attending the teacher education and child welfare study programmes showed a significant tendency towards higher degree of agreement that the content in LMS Canvas (p=0.004) was well organised, and that the learning resources (p=0.004), assignment (p=0.003), syllabus (p=0.002) and discussions (p=0.014) were relevant, when compared to the health and social care students (Table 4). However, these students were only significantly more agreeing than health and social care students that the group discussions were relevant (66.3% vs 55.7%, p=0.038). Moreover, they tended to be more satisfied with the collaboration in the IPL groups (p=0.007 for association), even though the shares of students agreeing did not differ significantly.
WHO's Core Competencies -Had the Students Gained New Academic Insight?
Among the students who responded to the questionnaire, 57.8% agreed that they had gained better academic insight into their own professional role (Table 5). In   Students below 25 years tended towards more likely to agree that the seminar days gave them better academic insight into their own future professional role (p=0.007), with 46.2% completely or strongly agreeing as compared to 25.6% among older students (p=0.001). They were also overall more likely to be satisfied with their learning of  The teacher education and child welfare students tended to report that the seminar days gave them better academic insight into their own future professional role (p<0.001, 52.6% vs 28.1 agreed (p<0.001)) and that they had learnt about IPC (p=0.014, 47.9% vs 58.7% agreed (p=0.040)) and teams (p=0.038), however, the later with no significant difference regarding the share of agreeing (59.3% vs 64.8%). No other associations between overall students' answers and education programme were found (Table 6), though the share of agreeing with Notes: a p-value for χ 2 -test for association between students' answers and study programme. "Interprofessional communication" was significantly higher among the teacher education and child welfare students (62.2% vs 50.3%, p=0.022).

Discussion
These students had positive perceptions of the blended learning approach in the large-scale IPL course, although few of them had prepared themselves ahead of the course through the digital resources available. The students agreed that the course had given them some academic insight into the WHO's core competencies. The highest learning outcome was, however, reported from the face-to-face IPL group discussions, and not from the digital learning material. Approximately threequarters of the students disagreed that virtual groups would had led to better learning outcomes than face-to-face group discussions, which is consistent with the overall high satisfaction with the face-to-face IPL group discussions. Notably, the teacher and child welfare students were overall more satisfied and reported a higher learning outcome than the health and social care students, which suggests a potential for IPL course improvement.
Although most students reported having learned something about the WHO's four interprofessional core competencies, the teacher and child welfare students were more likely to report that they had gained better academic insight into their own future professional role, IPC and teams and teamwork. In our previously published study, the majority of the IPL students had learned about their own future role. 22 In contrast, less than 20% had learned about other roles. 31 A successful IPL has been found to include all involved students' awareness of both their own and others' professional identity and professional roles. 35 While all the students will be familiar with the "teacher role" from growing up, fewer students will be familiar with the health and social services. This skewness might have affected our results, but for ethical reasons, we did not ask these young students about their prior private experience with health and social care settings. The teacher education and child welfare students are also more familiar with children and young people as a user group, which may explain why these students to a significantly higher degree agreed that the learning resources, assignment, syllabus and discussions were relevant. The health and social care students were overall less satisfied than the teacher education and child welfare students. Further analyses into the different specialities were not considered relevant in this IPL study. The limited number of respondents in several of the different specialties would also have resulted in low reduced statistical power in such analyses. In the working field, IPC can be challenging as different professions have different educational and professional cultures and varying approaches to dealing with challenges. 4 Mandatory digital preparation ahead of participation (flipped classroom) may even out some of the academic differences between the different educational groups in future courses, in addition to more active use of real-life cases relevant to all professions (for example hospitalised children). IPL has become a mandatory part of health and social care programmes (such as physiotherapy, Mensendieck Physiotherapy, nursing, social work, child welfare and occupational therapy) in Norway from the academic year 2020/ 2021. 36 The present study is supportive of the new national requirements in Norway, which make IPL/interdisciplinary teams a requirement in profession-oriented bachelor's programmes. 6,36 In Norwegian teacher education programmes, priority is given to three interdisciplinary themes: democracy and citizenship, sustainable development, and public health and well-being. 6 Learning and training in an interprofessional group pre-service is however not the same thing as functioning interprofessionally in real life settings. 7 This study suggests that the different student groups most likely were not equally prepared for later IPC, as measured by their self-reported attained WHO's four interprofessional core competencies. Our study therefore support that an early introduction of IPL in pre-service training is important, because successful IPC requires knowledge and understanding of the four WHO core competency domains, 37 which take time to develop.
In descending order, students were most satisfied with the learning outcomes from the IPL group discussions on campus, the combination of everything, the digital learning content of LMS Canvas, the submission assignment, the syllabus, and the supervision. These findings did not differ according to age or study programme, and are in line with the results from our previous study 22 and the results from other comparable studies. 38,39 Learning is an active subjective process that takes place through activity and reflection, and not through students passively receiving theoretical knowledge. 40 Our data is supportive of the opinion that when students meet synchronously, time should be used actively, not passively. 30 The present data reflect the opinions of a high volume and diversity of professional students only a few weeks ahead of the national lockdown in Norway due to the COVID-19 pandemic. 41 Although student active learning methods and digitalised learning material might have been applied to a minor extent in some of the nine included pre-service training programmes, these first-year bachelor students had limited experience of a digital transition at our university before the pandemic. 42 This IPL course using blended learning was thus a completely new learning experience for these students. The lack of prior experience with digital learning material and the flipped classroom 30 might be an explanation for why few students came prepared and informed to the IPL groups. Nevertheless, these data support blended learning as an approach in large scaled IPL studies.
The present study has some limitations, and several strengths. The cross-sectional study design does not help to determine cause and effect and cannot be used to analyse behaviour over a period of time. The response rate was low, but in line with the fact that the response rate to surveys in general is declining, which threatens the validity and generalisability of findings. 43,44 A high response rate is, however, no guarantee of sample quality. Self-selection bias may threaten internal validity, but the diversity in our sample enhances the robustness of the findings. The validated Norwegian version of ICCAS (measuring students' selfreported IPC competencies) was published after the present course was completed, 45 and was not therefore available for the present study. Major study strengths include the large heterogeneous sample size, and anonymous data collection. Students from all of the nine different study programmes included in the study contributed. Moreover, an external statistician was responsible for the statistical analysis. The present results are in line with the results from our previous study, 22 and since the provision of the present 2020 IPL course was repeated on the basis of the 2019 IPL course, this adds strength to our conclusions. To the best of our knowledge, no comparable large-scale quantitative IPL study was timed so closely to the Corona lockdown of campuses in Norway, that investigated learning outcomes and student satisfaction relevant to (post) pandemic IPL courses. Moreover, we are not aware of any other studies that evaluate aspects of digitalisation and IPL that include students from both teacher education, and health and social care study programmes.

Conclusions
In conclusion, the blended learning approach was positively evaluated, but the students' perceived learning outcomes from the IPL face-to-face group discussions were markedly higher than from other components of the blended learning course. All of the students had to some extent increased their knowledge related to the WHO's four interprofessional core competencies. However, the teacher education and child welfare students were generally more satisfied than the health and social care students, and they also reported that they had gained more insight into the WHO core competencies.

Data Sharing Statement
The data can be obtained from the corresponding author upon reasonable request.

Ethics Approval and Informed Consent
Ethical Guidelines for Research at Oslo Metropolitan University (OsloMet) were followed. 32 Permission to conduct the study was obtained from the project leader. Since the study is completely anonymous and no sociodemographic information beyond the participants' age and gender was collected, this research was exempt from approval from review and approval by an institutional review board or institutional ethics committee. Further, the study was discussed with the Norwegian Centre for Research Data (NSD), 33 and they replied that it was unnecessary to report the study to them since if it was completely anonymous and no sociodemographic information beyond the participants' age and gender was collected, as was the case in the current study (NSD reference number 741649). None of the participants were under the age of 18 years. The data were collected through an anonymous web survey using "Nettskjema", 34 in line with ethical guideline. 32 The participants were provided written information about the study on beforehand in LMS Canvas. The voluntariness and anonymity of the participants were emphasized, and the participants were informed about the purpose of the study and how the data would be used. Answering the questionnaire was considered informed consent to participate. The study complies with the Declaration of Helsinki.