EFFECT OF EDUCATIONAL TRAINING ON NUTRITION AND WEIGHT MANAGEMENT IN ELITE SPANISH GYMNASTS

The aim was to study the effect of a formative program about nutrition as well as weight management and its risks in elite gymnasts. The sample was 37 under-18 Spanish athletes from rhythmic and aesthetic gymnastics (national level). The sample was divided into a control group (n = 21) and an intervention group (n = 16). A quasi-experimental design with a pre-test and post-test was used. The dependent variable was the knowledge of nutrition, weight management, and its risks. The independent variable was the educational program. The program had three 30-minute sessions which combined talks, videos, and tasks to complete. Descriptive and inferential analyses were done. The educational program was ineffective with regard to increasing the gymnasts’ knowledge about nutrition, weight management, and its risks. The gymnasts’ knowledge of nutrition and weight management was high before beginning the intervention. Gymnasts and the coach expressed the need for individualized training adapted to each gymnast’s needs and for incorporating a follow-up that provides feedback that is applicable in the gymnast’s daily life. A pre-test and a post-test were utilized in this quasi-experimental design. The knowledge of nutrition, weight management, and its risks was the dependent variable, and the educational program was the independent variable. An ad-hoc questionnaire was utilized to measure the athletes’ knowledge. Questionnaires and interviews were used to register the coach’s and gymnasts’ perception. The program consisted of three 30-minute sessions, which combined talks, videos, and tasks to complete.

Introduction
Performance in aesthetic sports, such as rhythmic gymnastics, is evaluated by the physical appearance of the athlete's executions (Douda, Toubekis, Avloniti, Tokmakidis, 2008;Lazarevic, Petrovic, Ramnjanovic, 2012). This results in athletes and coaches placing great importance on the physical appearance and shape of the athletes (Gomez-Campos, Camargo, Arruda, Cossio Bolaños, 2013;Hergenroeder, Broun, Klish, 1997;Klentrou, Plyley, 2003;Tan, Calitri, Bloodworth, McNamee, 2016). The importance placed on the athletes' physical appearance, the search for perfection in this type of sport, and the physical and psychological changes in adolescence mean that young gymnasts are at risk for health problems related to weight management and disordered eating (Bonci #0# et al., 2008;Cupisti, D'Alessandro, Castrogiovanni, Barale, Morelli, 2000;Evans, Rich Holroyd, 2004;Halmi, 2009;Sundgot-Borgen, 1994;Sundgot-Borgen, 1996). The reference image that media and elite athletes provide to young gymnasts is a distortion of the reality in the way that they focus on peak-performance athletes. Therefore, young gymnasts have several models as a reference: from society, models stylized with the help of Photoshop, and from sport, the body that peak-performance athletes have at the pinnacle of their sport life, after years of dedication and training. Among the possible ways of reducing the risks involved in this situation is providing young athletes with information and skills through education (Coppola, Vastola, Scatigna, Fabiani, 2014;Sundgot-Borgen, 1994).
In gymnastics, although athletes' knowledge of nutrition is good, they have been found to have an energy deficit due to the intensity and duration of their training (D'Alessandro et al., 2007;Deutz, Benardot, Martin, Cody, 2000). Some studies have found that elite gymnasts have high carbohydrate intakes and low intakes of lipids, calcium, and proteins (Michopoulou, Avloniti, Kambas, Leontsini, Michalopoulou, 2011;Gomez-Campos et al., 2013;Nordin et al., 2003). Despite insufficiencies, gymnasts have perceived that they had adequate dietary habits (Jonnalagadda, Benardot and Nelson, 1993) and that they had a good dietary distribution throughout the day (Nordin, Harris, Cumming, 2003). During the competitive season, gymnasts change their dietary habits, reaching caloric levels considered inadequate (Michopoulou et al., 2011). In the bibliography, no specific studies have been found about educational training regarding nutrition or weight management. However, several studies have reported their need as well as the higher incidence of disordered eating and eating disorders in gymnastics (Anderson, Petrie, 2012;Sundgot-Borgen, Torstveit, 2004).
There are many aspects that can contribute to the development of unhealthy behaviors related to nutrition and weight management in young athletes. The different stakeholders in these sports must be actively involved in creating environments and resources that promote the athletes' wellbeing. Ideally, the coaches and parents will also be involved in this process. Further, the interventions must provide and reinforce the importance and role of nutrition, healthy reference models, and how athletes should manage their weight and the risks involved in weight management. The aim of this study was to assess the effect of a formative program about nutrition, weight management, and its risks in elite gymnasts.

Material and Methods
The sample was composed of 37 under-18 gymnasts that participated at least once in a Spanish national Championship of rhythmic gymnastics. The sample was divided into a control group (n = 21) and an intervention group (n = 16), and convenience sampling based on the training location of the gymnasts was utilized. The characteristics of the sample are described in table 1. The study was approved by the Ethics Committee of the first researcher. Parents and guardians of all participants signed an informed consent before the study began. Participation in the study was voluntary, and all participants were informed about the study. A pre-test and a post-test were utilized in this quasi-experimental design. The knowledge of nutrition, weight management, and its risks was the dependent variable, and the educational program was the independent variable. An ad-hoc questionnaire was utilized to measure the athletes' knowledge. Questionnaires and interviews were used to register the coach's and gymnasts' perception. The program consisted of three 30-minute sessions, which combined talks, videos, and tasks to complete.
Manuals about the topic were utilized to design the questionnaire and the training (Bean, 2011;Bonci et al., 2009). The questionnaire evaluated basic nutrition knowledge as well as knowledge of weight management and its risks. The questionnaire consisted of 41 true-false questions (18 about weight management and its risks and 23 about nutrition knowledge). Both the questionnaire and the training were validated by experts (content validity). Experts were asked to evaluate quantitative (on a scale from 1 to 10) and qualitative (open) questions from the questionnaire and the training with regard to: degree of understanding, degree of adequacy, and the need to reduce or include more questions or information. In accordance with Bulger and Housner (2007), questions with values <7.0 were eliminated, questions with values >7.0 and <8.0 were modified, and questions with values >8.0 were accepted or accepted with modifications. A test-retest protocol with 11 international-level gymnasts (16 ±1.16 years of age) was used to establish the reliability of the questionnaire. The reliability of each item was calculated using the Kappa Index in the SPSS software. The lowest value for any question was 0.609 (Visiedo, Frideres, Palao, 2017).
The goal of the program was to provide the gymnasts with knowledge about nutrition, weight management, and its risks. The sessions were incorporated into the gymnasts' training schedule. Each session consisted of two 15-minute blocks, the first of which was theoretical and the second of which was practical (tasks to complete). The content distribution for the three sessions included basic nutrition knowledge, weight management and eating habits, and weight management risks. One of the researchers directed the sessions. At the start of the intervention, a document with the information provided during the sessions was given to the gymnasts. The questionnaire was completed before the start and four weeks after the intervention program, and the interviews were completed four weeks after the program. Descriptive (average, standard deviation, percentages) and inferential analyses of the results were done using SPSS software. Differences between the pre-test and the post-test were calculated using T-tests for categorical questions and ANOVAs for independent variables for continuous questions. Significance was set at p < 0.05.

Results
The educational training was ineffective with regard to increasing the nutrition and weight management knowledge of the intervention group (Tables 2 and 3). Both groups had scores that were higher than 75% for nutrition knowledge as well as weight management. No significant differences were found between the intervention and control groups in the pre-test or the post-test.  Three ideas were collected about the effect of the educational intervention from the opinions and perceptions of the gymnasts and coach, which were collected from the questionnaires and the interviews: 1) retention and effect of the educational program; 2) level of engagement in the program, and 3) evaluation of the program.
1. Retention and effect of the educational program. Seventy percent of the gymnasts highlight that the educational program helped them to make some changes in their dietary habits or to remind them of information that they already knew. These ideas were expressed in the following way: It was helpful, a reminder of how to manage the meal schedule and learn more about quantity and proportions" [Gymnast #8] It will help me to pay more attention and eat better… [Gymnasts #5, #6]

I already had information about this. It was a reminder [Gymnast #1]
2. Level of engagement of the gymnast. There were differences between the engagement mentioned by the gymnasts and that mentioned by the coach. Sixty percent of the gymnasts said that they were interested in the topic and need to pay more attention to what and how much they eat. The coach commented that the gymnasts were not completely committed or engaged in the process.

I pay attention to my meals and to food proportions [Gymnasts #1-6]
It was interesting [Gymnasts #7, #10, #12] [There was] moderate engagement, but we must review the topic often. It was interesting [Coach] 3. Evaluation of the program. Eighty percent of the gymnasts liked the educational intervention. Most of the gymnasts and the coach expressed the need to incorporate more specific, individualized information that is applicable to the gymnasts and their diets in their daily life. …We should plan each athlete's diet, because we are not the same [Gymnast #8] It is necessary to combine the educational training with follow-up in their day-to-day… the information must be individualized for each one [Coach]

Discussion
The intervention, based on theoretical and practical classes and providing the gymnast with educational material, did not increase the gymnasts' knowledge of nutrition or weight management. There are possible explanations for these results. The gymnasts' knowledge was high to begin with, which is in accordance with other studies in the literature, as elite gymnasts have been found to have good knowledge about nutrition (Cupisti et al., 2000;D'Alessandro et al., 2007;Karabudak, Köksal, Ertaş, Küçükerdönmez, 2016). Another possible reason is that the intervention was not long enough and/or specific enough. Most of the gymnasts considered the intervention to be interesting but a reminder of information they already knew. The gymnasts emphasized the information regarding using a plate as a reference to monitor food proportions as well as using your hand to determine the portion sizes of specific foods. Regarding the specificity of the information, the fact that the educational training used data that were not specific to gymnasts could have resulted in having a lower impact on the gymnasts. The recommendations made by some of the gymnasts and the coach may be helpful in this regard. They mentioned the need to individualize the educational program using data from the same athletes to engage each subject more. Also, they mentioned the need for a follow-up of their diet and for incorporating feedback that is applicable in the daily life of the gymnast. This approach will allow the gymnasts to combine theoretical knowledge with their ability to apply that knowledge.
A previous study carried out with volleyball players has shown the effectiveness of the combination of educational training and a follow-up with a dietician (Valliant, Pittman Emplaincourt, Kieckhaefer Wenzel, Garner, 2012). Several aspects must be considered. The volleyball sample's knowledge was not as high at the beginning of the process. Therefore, it is possible that the educational program must be adapted and involve more individualization. If possible, the involvement of a dietician is recommended. However, oftentimes, gymnastics clubs do not have the possibility to collaborate with a dietician. Another possibility for implementing this type of educational program in this population would be the use of online educational training or educational software that allows gymnasts to automatically receive feedback from a dietician. However, it is not known whether this approach will help to increase the gymnasts' knowledge and improve their habits. Some studies have shown that gymnasts are knowledgeable about nutrition and they use this knowledge to modify their diets and to try to achieve and maintain the physical standards of the sport (Boros, 2009;Cupisti et al., 2000;Karabudak et al., 2016). More studies are needed to assess the effect of other intervention approaches in elite gymnasts.

Conclusions
The educational intervention that combines theoretical and practical classes and provides the athletes with educational material did not change the gymnasts' knowledge about nutrition or weight management. The gymnasts' knowledge of nutrition and weight management before the intervention was high, given that they scored higher than 75% on a knowledge test. Future studies are needed to determine what approach must be followed to effectively provide educational training to elite gymnasts. The gymnasts and the coach expressed the need for individualized training adapted to each gymnast's needs and for incorporating a follow-up that provides feedback that is applicable in the gymnast's daily life.


Distance Education in the Clinical Laboratory Sciences: Part of the Solution to the Professional Labor Shortage

The purpose of this paper is introducing research conducted about issues related to influences and barriers to the potential use of distance education for mitigating the clinical laboratory sciences labor shortage. Diagnostic careers such as those in the clinical laboratory sciences remain a mystery to many people because they do not have the same prominence or visibility associated with therapeutic careers. Clinical laboratory science courses often have both didactic and laboratory components. Coursework with laboratory components require additional faculty time for preparation. When health care education is offered in traditional university or college settings not affiliated with a teaching hospital or clinical setting, laboratory costs are higher due to purchasing supplies, reagents and media. Issues are further explored in a brief series of papers addressing them.  Using DE for delivering diagnostic clinical educational is a potential viable solution for addressing national diagnostic labor shortages.

INTRODUCTION
Hospitals are experiencing a shortage of qualified laboratory professionals [1][2][3][4][5][6][7][8]. This shortage includes both Medical Laboratory Scientists (MLS), and Medical Laboratory Technicians (MLT) [2,9]. Increasing numbers of graduates from clinical laboratory science programs is one solution to resolving the labor shortage. Academic rigor and quality of educational offerings are issues requiring consideration for a profession impacting patient health and its associated outcomes [10,11]. One solution to the labor shortage includes transforming educational programs from traditional to distance education (DE) for didactic program delivery. Following an exploration of the problems in pursuing diagnostic clinical careers, this review focuses on how using DE can be a practical part of the solution to resolving this area's health care labor shortages. The purpose of this short report is introducing research conducted about issues related to influences and barriers to the potential use of distance education for mitigating the clinical laboratory sciences labor shortage.

Problems
The need for MLS's and MLT's in diagnostic departments is projected to increase thirteen percent by 2026 [9]. Data indicates a labor shortage of qualified laboratory professional [1-3, 5, 7, 9]. Reasons for these labor shortages include: 1) increased retirements of laboratory professionals; 2) fewer individuals pursuing degrees in diagnostic clinical occupations; 3) closure of traditional clinical laboratory science academic training & educational programs; and 4) aging populations requiring medical care [1-3, 5, 7, 9].

Diagnostic Retirements
Many diagnostic specialists are retiring or nearing retirement age, compounding the existing labor shortage [1,3,4,5,7]. Retirement rates are projected to increase in various areas of diagnostic medical laboratories for both nonsupervisory and supervisory staff [4]. The significance of these numbers when related to direct patient care includes timeliness of sample collection, its' processing, quality assurance, and performance improvements. All these areas could significantly be impacted by labor shortages [4]. Table  1 illustrates retirements.

Career visibility & awareness: diagnostics vs. therapeutics
Diagnostic careers such as those in the clinical laboratory sciences remain a mystery to many people because they do not have the same prominence or visibility associated with therapeutic careers. This lack of awareness has in part led to clinical laboratory science program closures [5,[12][13][14]. Individuals may not consider diagnostic careers because they lack knowledge of the options available. Therapeutic careers with more direct patient contact such as nursing, pharmacy, rehabilitation, dentistry, physician assistants, nutritionists, speech pathologists, mental or behavioral health and social workers have a higher degree of public recognizability than diagnosticians [15].
Clinical laboratory scientists often have limited patient contact resulting in the general public lacking knowledge of career options available in diagnostics. Regrettably this circumstance often extends to students, parents, secondary school career counselors and teachers [13]. Support staff typically obtain and transport specimens directly to the laboratory eliminating or reducing potential contact with laboratory personnel. Patients rarely go directly to the clinical laboratory to provide samples. Unobtrusively working as part of an interprofessional team in a consultative role providing therapeutic practitioners with laboratory results for making clinical diagnoses has resulted in a decided lack of awareness of this career cluster of professions amongst students, parents, educators, and career counselors [13]. Exposing students during career exploration late in elementary or early in secondary school to the possibilities the clinical laboratory sciences offer through social media platforms, technology driven presentations, and direct interaction thru career exploration could develop publicity for diagnostic professions [5]. Unless students have a meaningful encounter, they may not be interested in pursuing a diagnostic career even with increased visibility and career exploration within public schools. Even if career exploration is even partly successful, this exposure is only one part of a potential solution for addressing the labor shortage. Counselors and educators whom know little about diagnostic careers available compared to therapeutic options cannot offer students and parents advice about the educational preparation students in middle, junior and high school could be taking in preparation for career & educational training programs [13].

Confounding factors: career advancement and salaries
Compounding the effect of a lack of awareness of diagnostic careers are negative perceptions about career advancement and salaries [12,13]. For individuals interested conceptually in direct patient contact, therapeutic care careers may provide a more attractive career pathway than one in the clinical laboratory sciences. Therapeutic careers have multiple opportunities for professional advancement and the potential for diversity in work settings [9]. Career laddering allows individuals to move from entry level to highly specialized careers, making a wider range of employment settings, specializations and geographic areas available for employment.
Individuals choosing clinical laboratory sciences as a career may not intend to stay in the field, but rather treat it as an entry point on their intended career pathway [13]. Job security, networking opportunities, the benefit of practical experience, and developing professional references may initially be why individuals pursue diagnostic careers [13]. These steps may strengthen future chances of moving toward more specialized careers and motivate diagnosticians to continue advanced education via alternative, advanced career pathways [12,13]. If individuals are unaware of career advancement possibilities this may also be a contributing factor in the diagnostic labor shortage [12,13].
Both the clinical laboratory sciences and nursing are female dominated professions [2,16]. Salaries are notably lower than therapeutic practitioners such as nursing even though both groups have similar educational requirements [9,13]. Comparisons of the median annual salary of a MLS with a bachelor's degree to a registered nurse is substantially lower especially for highest earners [2,16].
The combination of these issues may also be compounding the potential for attracting individuals to diagnostic laboratory careers.

Program Closures
Diagnostic preparatory programs have been closing.
Various factors impacting program closures include lack of career awareness, negative career perceptions, and high costs associated with offering diagnostic career education [2,3,5,7,12,13,14]. Nearly two thirds of the original diagnostic clinical laboratory science programs in the US have closed in a span of thirty years [2]. Program closures have also significantly impacted production of the number of clinical laboratory science graduates, resulting in a net loss of nearly 66% of the potential workforce [2].
Clinical laboratory science courses often have both didactic and laboratory components. Coursework with laboratory components require additional faculty time for preparation. When health care education is offered in traditional university or college settings not affiliated with a teaching hospital or clinical setting, laboratory costs are higher due to purchasing supplies, reagents and media. Diagnostic programs receiving donations from clinical affiliates, may reduce, but not eliminate, the financial liability associated with running them. If diagnostic preparatory programs are financially considered as too expensive to offer within the respective university or college they may be considered for discontinuance. This could be especially true during a time of uncertainty within higher education where projections slate half of institutions offering higher education may close within the next 10 years [17]. This circumstance further compounds the problem of reduced numbers of institutions providing education for clinical diagnostic careers, in turn adding to the labor shortage.

Aging Population
Populations in developed countries are living longer due to advancements in medical care, treatment and prevention [18]. Extended lifespans, however, do not come without potential consequences. People are living longer with multiple comorbidities including hypertension, heart disease, and diabetes [19]. Care of these patients often requires clinical laboratory services. Physicians often rely upon objective laboratory results for diagnosing and managing complex disease states requiring performance and accurate interpretation by skilled laboratorians.

RESULTS AND DISCUSSION
There are several ways the professional labor shortage in diagnostic clinical laboratory careers could be addressed. One, increasing the number of academic programs available nationally, is not necessarily viable or realistic due to cost, access and recognition of its need. A second solution, one of increasing student enrollments in existing programs, is possible and feasible, but requires a refocus by program directors about how it could be accomplished on a national scale with accrediting bodies, with educational institutions, and educators. The third solution, addressed in this review, is the most likely and practical of the three solutions proposed, using a different way to deliver diagnostic career preparation: distance education.
Some of the top factors which could be considered as solutions noted by diagnostic program directors are illustrated in Table 1.
Interestingly, directors recognize using DE as an access solution to educational programming. Those with more experience comprehend the opportunities to reach more students, enhance flexibility and autonomy, while enhancing and diversifying teaching skills and abilities. These data reflect how applying alternative delivery models to educate global workforces in diagnostic and allied health care professions impact faculty and staff doing so. Subsequent papers will illustrate other barriers and motivational factors to using alternative educational delivery systems for developing an interprofessional diagnostic and allied health workforce.

Changing Educational Paradigms in Allied Health and Diagnostic Disciplines
Limited numbers of clinical laboratory science programs are using DE modalities for offering access to didactic career preparation [20,21]. Training diagnostic practitioners for staffing urban and rural communities acute care settings by using alternative educational delivery methods is a key solution for providing education addressing occupational shortages [1][2][3][4][5][6][7][8].
Clinical laboratory science program numbers compared to therapeutic allied health programs using DE are still relatively small. Using the disruptive innovation [17] DE offers increases opportunities for students' ability for accessing diagnostic care education and careers. Academic and practical performance comparing face to face, traditionally trained diagnosticians versus those trained using DE for curriculum delivery have little to no measurable differences in performance, including grades and standardized test performance[10, 11,14,[22][23][24][25].
The clinical laboratory science profession has been slower to adopt to using DE for delivering academic education compared to other allied health professions. Nearly half of Bachelor of Science in Nursing (BSN) completion programs were offered completely online in 2016 [26]. Nearly thirty percent of Dental Hygiene (RDH) undergraduate degree completion and graduate programs are also available via online distance modalities [27]. Comparatively, there are 21 online MLS programs, 18 online MLT programs, and 25 MLS/MLT online degree completion programs [20,21]. These program numbers are small compared to the total number of diagnostic  Online programs can offer students access to educational opportunities otherwise not available due to geography, personal responsibilities, and cost [29]. Numbers of non-traditional students returning to school is increasing as individuals consider changing or pursuing different careers [30]. Using DE for career preparation can provide distinct advantages over traditional programs using face-to-face delivery, especially when utilizing asynchronous formats [31]. Some program directors and educators within the clinical laboratory science disciplines are recognizing this trend and the number of programs using DE for program delivery are slowly increasing [14]. Changing program delivery modalities requires changing educational paradigms. This concept extends to program directors, educators, and administrators throughout organizations. Administrators need to recognize the investment in health care career educational training such as the clinical laboratory sciences is higher [2] due to accreditation requirements, specialized equipment needs, student/faculty ratios and infrastructure for offering laboratory portion of program curriculum compared to liberal arts programs. Administrators and program directors need to also recognize offsetting costs for didactic and laboratory education. Using DE and asynchronous online learning for delivering the didactic portion of academic program curriculum can reduce expenses while expanding access to people seeking diagnostic career training.

CONCLUSION
Academy administration, program directors and faculty need to be willing to consider implementing DE as a viable solution to reducing professional laboratory labor shortages.
If administrators and program directors willingly embrace this option, it still falls to educators having a willingness to be adaptable in modifying educational curriculum, using DE teaching methodologies for delivering program content, and applying different means to assessing students' performance. Educational institutions are often slow to change, and an educational paradigm shift of this magnitude is typically not well received by traditionalists. Recognizing the need for change versus implementing changes for a variety of reasons can be difficult for colleges and universities. Using alternative educational delivery systems like DE are allowing people a variety of more affordable and accessible options to educational and training programs than ever before. 17


THE SPIRIT OF SPORT: EXPLORING SPORTSMANSHIP AMONG PHYSICAL EDUCATION STUDENTS

This research was conducted to explore the perception of physical education students in a Philippine school about sportsmanship. It particularly dealt with respect (1) for the opponent, (2) for rules and officials, and (3) for social conventions. This research used a quantitative descriptive method of research, with survey-questionnaire as the main data gathering tool. It was adapted from a valid and reliable instrument known as multidimensional sportspersonship orientations scale. In regard to respect for social conventions, participants perceived themselves as people who rarely congratulate the opponent, and infrequently shake hands with the opponent’s coach when they lose. Respect for the rules and the officials got good scores. Respect and concern for the opponent got low scores. They professed that they rarely do necessary actions when an opponent is unjustly penalized, hurt, or forgets his or her equipment. To achieve victory does not solely undermine character. It can be a good avenue to harness physical improvement and love for the sports. However, when the virtue of sportsmanship is discounted and forgotten, the result can be disadvantageous or even disastrous. Conclusions and recommendations were presented.

INTRODUCTION
Sports are observed as the major way to educate students about social values and life skills which will be advantageous to them, most especially, into their adult years. While there are many proper social set-ups in which student learning happens, physical education sports are matchless in that it, in general, suggests a highly controlled atmosphere where different rudiments of success and failure can be assessed by a specific and definite set of rules (Muñoz-Bullón, Sanchez-Bueno, & Vos-Saz, 2017).
The educational value of sports is well recognized throughout the Philippines, as evidenced by the abundant sponsorship and support of athletic programs around the country.
Sportsmanship is described by the National Collegiate Athletic Association (2003) as a collection of activities to be practiced by players, coaches, officials, administrators, and spectators in athletic competition. These actions are founded on universal principles such as respect, dignity, civility, integrity, and accountability. According to research, sport education is a successful teaching pedagogical model to be taken into attention by physical education teachers so as to encourage moral and ethical know-how among students (Burgueño & Medina-Casaubón, 2020).
However, there are students compelled by ego who choose to participate in order to beat their opponents. They play not for the sake of enjoyment and camaraderie, but for making someone inferior. They want to assert their dominance or superiority; they actually do that even at the expense of poor sports behaviors like breaking the rules or dishonesty.
Research also specifies that aggression, dishonesty, and other poor sports behavior happen with disturbing frequency (Kavussanu, 2006).
Many teachers generally observed negative comments were, from time to time, evident during sports and activities in physical education. There were cases also of untruthfulness in relation to results and performance. Hence, it is also perceived that the only significant matter for many was winning. These observations led to the conduct of this research. Such practices are not good and must not be tolerated by the school and teachers. Therefore, physical education is indeed a good avenue in which teachers can inspire appropriate behaviors and communicate the virtue of sportsmanship among students. Knowing such alarming perceptions among students, the school must undertake necessary actions to promote sportsmanship among them. Therefore, this research may permit to further understand and diminish elements that could hamper or interfere with the education process (Bach, 2006).

Research Questions
This research attempted to assess the perception regarding sportsmanship in a Philippine school among physical education students. Specifically, this research answered the following questions: 1. What is the level of perception of the students in terms of respect for social conventions? 2. What is the level of perception of the students in terms of respect for the rules and the officials? 3. What is the level of perception of the students in terms of respect and concern for the opponent?

Research Significance
The uniqueness of this study can be traced upon the fact that no research had been undertaken about students' perception on sportsmanship in such a Philippine school.
Understanding such paucity, this scholarly undertaking would be of great benefit to the arena of physical education and also in the further understanding the topic in line of the school context. illustrates that sportsmanship hinges on respect for the rules and the officials, respect for social conventions, and concern for the opponent and that they are interrelated with one another.

LITERATURE REVIEW
Sportsmanship refers to activities that represent a child's or adolescent's moral and character formation in physical activity contexts. Bolter and Kipp (2016). In a study of Murphy (1994), using eight-foot-tall basketball hoops or smaller and shorter soccer fields

Respect for Social Conventions
Concern for the Opponent

Respect for
The Rules and The Officials helps children to learn skills quickly and gain practical guidance. This encourages task orientation-striving for task achievement and exceeding individual efficiency objectives.
Significant studies have concluded that involvement in sports and physical education activities hold plentiful potential advantages for students. The benefits forwarded by the literature drive toward positive physical and socio-cognitive development (Kremer-Sadlik & Kim, 2007). The personal growth and life skills accomplished through these endeavors are enormously pivotal to their future success.
Physical education teachers and sport coaches may have a significant and long-lasting impact on the children and youth with whom they interact, including impacts on selfperceptions, inspiration, happiness, and sportsmanship habits (Horn, 2008). Horn (2008) contends that youths' impressions of their coach's or teacher's actions can have a clear and immediate effect on their own emotions, feelings, and behaviors.
But, sports programs and physical education activities offer only a setting for development and learning and do not deliver the actual lessons (Arthur-Banning, Wells, Baker, & Hegreness, 2009). It is the duty of the parents, teachers and coaches involved in the programs and activities to ensure that a proper environment is established and that appropriate messages are conveyed in order to encourage firm positive advancement in the character, most especially in the virtue of sportsmanship, among the students.

Participants
Respondents for this study were students (N=20) in grades 7 and 8 from a Philippine school. The sample was recruited by coordinating with the school to agree in the conduct of this study. Hence, it did not utilize a random sample technique. The researcher conveniently chose the respondents based on students' availability and accessibility.
The percentage of respondents from each grade level was as follows: (a) Grade 7, 45%; and (b) Grade 8, 55%, Of the total participants, 40% were female and 60% were male.
Participants ranged from 13 to 15 years of age. All respondents were enrolled in physical education, which was a required and mandatory subject in both private and public schools.
The descriptive research design was employed to achieve the purpose of the research. The descriptive method primarily utilized the weighted mean to find out the level of perception of students in terms of respect (1) for the opponent, (2) for rules and officials, and

Data Gathering Methods
Respondents were informed that the data collated from them would assist the researcher and physical education teachers in further understanding their perception about sportsmanship. All respondents were informed that the participation in the research had no bearing in their physical education grade and their anonymity would be valued.
The instrument administered was a quantitative survey-questionnaire. Respondents took approximately five minutes to complete the task. In this research, the multidimensional sportpersonship orientations scale (Vallerand, Brière, Blanchard, Provencher, 1997). was adapted as a research instrument and a tool to gather data from the respondents. It was proven valid and reliable to use for students. It assessed three different types of sportspersonship orientations.
Particularly, they were respect (1) for the opponent, (2) for rules and officials, and (3) for social conventions. It contains 15 items (5 items per subscale). It used a 4-point Likerttype scale ranging from 1 (strongly disagree) to 4 (strongly agree). The data were gathered through the survey-questionnaire. It was interpreted using descriptive statistics. This gave the researcher the data needed in realizing the goal of the study.

Data Analysis
The data gathered from survey-questionnaire were interpreted using descriptive statistics. Particularly, it utilized mean. This gave the researcher the data needed in realizing the goal of the study. Respect for social conventions is the first variable observed in this study. It is evident from the results that students perceived themselves in this criterion as low. They rarely congratulate the opponent when they lose, and infrequently shake hands with the opponent's coach. The respondents got high with the interpretation of doing always in the following activities: congratulating the opponent for good performance, acknowledging him or her for doing good work, and shaking the hands of opponents.

FINDINGS AND DISCUSSION
It can be concluded from the results garnered from the category, respect for social conventions, that they did not like losing. Such could increase the likelihood of them doing unsportsmanlike behaviors. When they lose, they commonly do not follow social conventions. This is in alignment with the observation of Fox (2008) that in the context of sports, fear of failure is manifested.
This also was seconded by Hanin (2008) expressing fear of failure is an intrinsic part of sport competition. Hence, when it turns to reality, students may become jealous, disagreeable, and even hostile. Therefore, it should be emphasized to the students that winning is not the prime award that they can get from playing sports or other physical education activities. The fun of playing should be put forward. Character building in this arena must be strengthened, also.

Table 2. Respect for The Rules and The Officials
Respect for the rules and the officials is the second variable in this research. The students often respect the referee even he or she is not good. Majority of the activities got high scores which mean that they always do practices like obeying the referee, respecting the rules, obeying all rules in all his or her sports, and respecting the official decision of the referee.
This is in conjunction with Cecchini and colleagues' research (2007), which cogently argues that the rules can reduce harsh playing, contact failures and a negative sportive behavior, since they indicate individual and social responsibility. In addition, these rules restrict behavior regardless of the sport itself (Vamplew, 2007). It is anticipated that fair play shall be manifested.
Reinforcing the respect for the rules and the officials indeed plays an indispensable role toward the development of physical education among students. Students must be bound to act in a sportsmanlike way toward fellow players. This idea must be advanced by the teachers, school authorities, and parents. This could help them show how to master their feelings and to handle to social conduct during games that would urge to react likewise.

Table 3. Concern for the Opponent
Respect and concern for the opponent is the last variable in this research, this got low scores from the respondents. They rarely do necessary actions when an opponent is unjustly penalized or hurt. They also do not want to let them borrow an equipment if an opponent forgot his material. They often help the opponent to get up. They also refrain from swearing when things do not go on their way.
This is similar to the results gathered by Stornes and Bru (2002) which shows that the participant adolescent players had low respect for their opponents. Similar to this, Bolter and Kipp (2018)'s findings indicate that the coach conduct has an effect with antisocial behavior toward teammates and opponents. Moreover, the lack of concern toward opponents is linked with the antisocial behavior toward teammates (Benson & Bruner, 2018).
The students got low scores on the category of respect and concern for the opponent.
They seldom do crucial actions when an opponent is unjustly penalized or hurt. The researcher suggests that physical education should execute programs or seminars that will aid them toward the development of positive character. It is not tolerable if the school will breed students who are excellent in sports but not in character. They should be taught on how to be concerned when unpleasant scenarios happened and must not utilize such for their own welfare.

CONCLUSION
In this research, it showed the perceptions of physical education students when it comes to the sportsmanship. The intuitive sense of the researcher led to the realization of this scholarly undertaking. Indeed, this variable must be scrutinized and be taken into observation. In actuality, it drove to the revelation that students' concerns when playing was more of winning only.
Their beliefs were explored and what emerged were certain aspects in which they needed to improve. To achieve victory does not solely undermine character. It can be a good avenue to harness physical improvement and love for the sports. However, when the virtue of sportsmanship is discounted and forgotten, the result can be disadvantageous or even disastrous.
Before sportsmanship can be manifested by the students, it indeed must first be evident from teachers, coaches, referees, and other stakeholders. One avenue in which they can learn such virtue is through modeling. When many people in the school display sportsmanlike behaviors then it becomes a culture in which students learn that through socialization. At a later time, it can lead to the positive change of the school climate.
The school must require competitors, referees and teachers to hold fast to key qualities of sportsmanship. It is proposed that they publish an improved code of conduct which are conveyed to students. Such can empower them to exercise restraint in undertaking unsportsmanlike practices. Plus, there should quantifiable standards for monitoring purposes.
Continuing personal and professional development for the teachers and students must be undertaken also. Furthermore, thorough checking and assessment of the effectiveness of sportsmanship education efforts should be undertaken. Future researchers may pursue to fill the research gap by replicating this research in different schools or environments.


Gauging Informal STEM Youth Program Impact: A Conceptual Framework and a Measurement Instrument

STEM education programs are often formulated with a "hands-on activities" focus across a wide array of topics from robotics to rockets to ecology. Traditionally, the impact of these programs is based on surveys of youth on program-specific experiences or the youths’ interest and impressions of science in general. In this manuscript, we offer a new approach to analyzing science programming design and youth participant impact. The conceptual framework discussed here concentrates on the organization and analysis of common learning activities and instructional strategies. We establish instrument validity and reliability through an analysis of validity threats and pilot study results. We conclude by using this instrument in an example analysis of a STEM education program.

Introduction
The aim of this report is to offer a new approach to gauging the impact of STEM-focused youth program activities on youth's active learning engagement. We begin by examining existing measurement tools commonly employed by informal STEM programs. Next, we propose a conceptual framework for an approach to program analysis differing from these existing methods. Then, we offer a new tool for measuring youth active learning engagement, exploring both the validity and reliability of this instrument. Finally, we offer an example of its application and interpretation of program analysis results.
Science-focused activities in both formal and informal education can help spark students' science interest (National Academy of Sciences, 2007). Thus, a better understanding of what aspects of these activities may influence students' preferences can help educators, researchers, and policy makers develop and improve various school-based and out-of-school time programs that purposefully involve customized elements to attract an increasing number of students in the STEM pipeline. A stronger grasp of children's preferences for specific pedagogical strategies can also serve to best match individuals' education selections with career options in the STEM workforce (Carnevale et al., 2010).

Background Interest in and Attitudes Toward Science
Much effort to engage children in science and engineering has concentrated on content, discovery-related hands-on activities, or a combination of these two elements. Indeed, educators often rely on the presentation of fascinating facts and images and/or discoveryrelated, hands-on activities that involve inquiry-based learning that may include projects and experiments (Swarat et al., 2012). In addition, a robust body of literature suggests students' interests and attitudes towards science are two dominant affective factors that can influence their career expectations and participation in science, technology, engineering, and mathematics (STEM) related fields (Koballa, Jr. & Glynn, 2007;Lent et al., 1997;Luce & Hsi, 2014;Simpson et al., 1994).

Science Interest Development
Students' interest in science is considered an essential indicator of the quality of science education, and a body of literature examines students' interest in science over time (Guzzetti & Bang, 2011;Schiefele, 1991). Interest can be defined as "a content-specific motivational characteristic composed of intrinsic feeling-related and value-related valences" (Schiefele, 1991, p. 299). School-based experiences are considered an important influence on youth science learning interest. In their longitudinal study of 33 high school students, Aschbacher et al. (2010) concluded that students most often attributed their interest in science, engineering, or medicine majors or careers to science teachers or early school-based experiences. This study also revealed the possibility that an individual's interest in science may change over time; findings suggested that some students who showed initial interest in science-related fields later abandoned the STEM pipeline in high school due to their discouraging school experiences (Aschbacher et al., 2010). Several of the interviewees, who were originally interested in science but later lost their interest, mentioned that their teachers discouraged them, or their curriculum contained too few hands-on science activities (Aschbacher et al., 2010). In their investigation of different learning environment elements in science classrooms, Swarat et al. (2012) found that instructional pedagogy that incorporates hands-on activities and allows for engagement with technology can pique youth science interest. These studies identify teachers and school-based activities as influences on students' science interest. However, more work is necessary to understand how instructors and learning activities exert their influence.
Beyond formal settings, research has found that out-of-school time science activities can strongly influence students' interest in science or help them maintain interests they developed in more formal school settings (Aschbacher et al., 2010;Bachman et al., 2008;Chang et al., 2009;Fields, 2009;Johnson, 2011). In her study investigating high school students' perceptions of their participation in a summer astronomy camp, Fields (2009) found that students who took part in the camp gained a deeper understanding of science as well as a stronger projective identity in science whereby they began to develop future involvement with science-related activities. These findings have been corroborated by other researchers who have also found that out-of-school time programs can reinforce science identities and improve, particularly for girls, students' interests in science (Barton et al., 2013;Farland-Smith, 2009).

Attitudes Toward Science
Attitudes toward science play an important role in the affective dimension of science learning.
Although the construct is not well defined in the literature (Osborne et al., 2003), attitudes can refer to "cognitive and emotional opinions about various aspects of science" (Kind et al., 2007, p. 873). While research supports that students' attitudes towards science and scientists are developed in the formative years between kindergarten and 12 th grade, studies have also found that as students mature, they tend to have more negative attitudes towards science (George, 2006;Majumdar et al., 1991). Like the shifting patterns of student interest in science, students' attitudes towards science, particularly girls' attitudes, decline significantly as they progress from elementary schools to middle schools (Barmby et al., 2008).
The reasons for this decline are not well understood, although a line of research has found a relationship between the way science is taught in school and student attitudes towards science (Bhattacharyya & Mead, 2011;George, 2006;Myers & Fouts, 1992;Talton & Simpson, 1986).
Work by Barman (1999), Ebenezer & Zoller (1993), Häussler & Hoffmann (2000), and Krajcik et al. (2003) has sought to examine the ways in which instructional practice influences students' perceptions of and attitudes toward science. Results have largely found that there is a compelling relationship between the type of pedagogy employed by teachers (e.g., teachercentered didactic instruction, collaborative learning, problem-based learning, contextualization of content) and students' science attitude development. Ferreira & Trudel (2012) found that problem-based learning in science can lead to significant positive increases in students' science attitudes. This result is supported by other research (Bhattacharyya & Mead, 2011;Robbins et al., 2005).
Studies employing quantitative methodologies have found that participation in out-of-school time programs with hands-on activities have a positive impact on students' STEM attitudes (Elam et al., 2012;Newell et al., 2015). Specifically, in her study evaluating the attitudes of high school students who participated in robotics competition and those in a comparison sample, Welch (2010) found that participating students held more positive science attitudes in four ways: social implications of science, normality of scientists, attitude towards scientific inquiry, and adoption of scientific attitudes. Additional research has offered evidence that students' attitude toward science correlates with achievement in the science classroom (Germann, 1988) and career expectations (Gibson & Chase, 2002). In light of these findings, it appears that developing measures of students' science interest, attitudes, and preferences is the next step. Biology, Chemistry, and Physics (Adams et al., 2004) Students' opinions about multiple facets in learning biology, chemistry, and physics respectively: knowledge connection, social significance, problem solving, scientific thinking, and enjoyment in learning these subjects.

Relevance of Science Education
(ROSE) (Talisayon et al., 2004) Multiple aspects of students' opinions about science and science learning: their interest in very specific science related topics, career expectations, within and out of school science experiences, as well as social significance of science and technology.

Survey Items of Situational
Interest (Hulleman & Harackiewicz, 2009) Whether classroom activities will increase ninth and 10 th grade students' motivation in learning science, and asks for students' perceptions of expectancies for success, interest in science, and utility value of science.
S-STEM (Unfried et al., 2015) Measure's student attitudes towards science, technology, engineering, and mathematics and interest in STEM careers for Grades 4-5 and Grades 6-12.

Existing Measurements for Interest and Attitudes Toward Science
To date, numerous measurements for students' interest in and attitudes towards science learning have been developed. Osborne et al. (2003) summarized five main approaches to measuring students' perceptions of science learning: (a) subject preference studies, asking students to rank their liking of school subjects; (b) attitudes scales, asking students' opinions about statements with Likert-scale response options; (c) interest inventories, asking students to select items of their interest from a list; (d) subject enrollment, collecting data on students' enrollments to various subjects; and (e) qualitative methodologies, collecting qualitative data from individual and group interviews. Among those approaches, attitudes scales were the most used in collecting large-scale data that can also present specific changes in students' interest in or attitudes towards science (see Table 1).

Theoretical Framework
There are several theories of learning and motivation that could be applied to unpack the ways in which students develop preferences for learning activities and experiences. The theoretical insights imparted by rational choice theory and expectancy-value theory are of relevance as they pertain to student learning preferences and choices. These two theories afforded us a model with which to better understand our data.
Rational choice theory (RCT) is a broader model rooted in sociology and behaviorist psychology that provides a useful frame for understanding how an individual develops and/or can change their preferences toward certain activities. RCT helps to explain the idea that all action is fundamentally "rational" in nature and that individuals are continually assessing the potential costs and benefits of any action or decision before they make them (Coleman & Fararo, 1992;Homans, 1961;Scott, 2000). These calculations lead to the development of individual preferences, which can be influenced by many factors such as individual personal habits and commitments, external variables such as availability or existence of alternative options, and assessments of the "positive or negative evaluations individuals attach to possible outcomes of their actions" (Wittek, 2013, p. 688).
Deitrich and List (2013) further elucidate the process of developing preferences and further explain also how they can indeed change or shift over time depending on individual variables and external circumstances. They assert that, according to RCT, preferences are not fixed but rather are based on certain "motivationally salient properties" of alternatives over which preferences are held (p. 613). This means that an individual's preferences may shift as different alternatives are presented and/or become-or cease to become-salient. It is these alternative choices, along with an individual's interaction with their social and cultural environments, which continually shape and re-shape their preferences (Loveland, 2003;Sherkat and Wilson, 1995).
In this regard, RCT provides a useful frame for us to better understand what aspects of STEM learning activities may influence students' preferences.
RCT alone does not embody intentions and attitudes, nor does it fully explain preferences (Opp, 2019). The closely related theoretical lens of expectancy-value theory (EVT) helps to flesh out how individuals formulate positive or negative associations and/or expectations of future outcomes related to certain learning experiences. EVT is a widely accepted model developed by Eccles, Wigfield, and colleagues (e.g., Eccles et al., 1983;Eccles & Wigfield, 1995;Wigfield, 1994) whereby an individual's expectations of success on a learning task and the value they place on the task are central determinants of their motivation to learn (Choi et al., 2010). These two motivational beliefs-success and value-can influence an individual's motivation and behavior.
Research applying EVT to learning activity preference and performance helps to further shed light on the research aims in our paper. In their examination of components of EVT via random assignment study of 70 mental health patients to assess their performance as they interacted with certain arithmetic learning activities, Choi et al. (2010) found that beliefs of content mastery can predict the degree of performance improvement on challenging cognitive tasks, even to a greater degree than general cognitive ability. These findings are reinforced by the exploratory work of Cooper et al. (2017), who developed a conceptual framework of EVT that is closely aligned with the research aims in this paper concerning learning activity preference.
They applied the theoretical lens of EVT to evaluate student perceptions of active learning after participating in active learning in a science course. Interview data collected from 25 first-year biology students who had participated in learning biology content through 40 hours of active learning revealed self-reported changes in the direction of increased engagement and selfefficacy, increased perceived value, and decreased resistance toward active learning. They conclude that by increasing the perception of the value of active learning-by offering a student-centered pedagogy and a variety of different active learning activities, for instanceinstructors can help increase student motivation to do well in active learning. Ultimately, the work by Cooper et al. (2017) is also supported by the work of other researchers examining how EVT impacts student perceptions of and preferences for active learning curricula (Schoor & with which to examine student preferences for different learning modalities and types.

Development of a Conceptual Framework Focused on Active Learning
This theoretical framework distinguishes between two modes of learning: receptive and active.
Forms of receptive learning include watching, listening, and reading-receiving information.
While youth may ask questions or make comments, the aim of receptive learning is to deliver information. Forms of active learning include making and discovering new things, collaborating, and performing. In this paper, we focus on forms of active learning and how these active learning formats may be used as measures of youth engagement and programmatic impact.
The instruments in Table 1 primarily target students' general attitudes towards science, impressions of the importance of science, specific interests in science, and motivation to learn science. All these measures are important, but the information they provide do not give youth STEM program developers and instructors the type of information necessary for reevaluating program impact on youth STEM engagement and making modifications. caretaking, and (g) teaching/tutoring (see Figure 1). Here we offer a concise discussion of how we define each of these seven active learning modes.

Typology of Active Learning
The activity of collaborating requires communication, sharing, and comparing thought and ideas among group members working together on a project or task. The form and type of collaboration may vary, but at its core is the requirement that the activity necessitates the cooperation and engagement of two or more individuals. One example is two youth working together to build a mousetrap-propelled model cart. Another example is a group of youth working together on a robotics team. An important characteristic of successful collaborations is clarity among team members of their respective roles and responsibilities. For example, while some youth focus on the mechanics of building the robot, others work on the control software programming, and still other team members study the tasks their robot will be required to do Making activities apply ideas and materials to constructing an object. Building towers of drinking straws, making bridges from toothpicks, making containers to protect raw eggs to be dropped from different heights, programming a website, and writing code to operate a robot are examples. Making-type activities are strongly associated with discovering-type activities and in many ways, discovering is to making as science is to engineering.
Performing is commonly associated with presentations and audiences but performing at its core involves accountability for an outcome at a specific place and time, meeting a challenge. While presenting to an audience is clearly a form of performance, other forms of performance may not involve an audience. A solitary youth developing her/his skill at starting a campfire by using a bow drill to create an ember is performing. The outcome matters to the youth engaging in the activity. Youth planting gardens to grow vegetables for their families or to donate to food banks is another example.
Caretaking is a type of activity where youth are given responsibility for caring for others, animals, and even objects. Caring for animals, growing a garden, designing a wheelchair ramp, and engaging in trash recycling to help clean up a community park are all forms of caretaking.
Teaching is an activity where youth are engaged in helping others to learn. Teaching involves engaging the mind and attention of another individual or group of individuals. In a teaching activity, the youth in the role of teacher has knowledge or information that they are tasked with passing along to others or has skills he/she helps others to attain.

The Instrument and Its Development
The youth engagement survey is shown in Table 2. This instrument was designed to assess the preferences that youth might have for each of the seven types of active learning. In this paper, we examine validity with respect to the conceptual framework and the survey instrument.
These efforts included pilot testing of the survey instrument as well as focus group interviews regarding the conceptual framework and the instrument. Next, we examine the reliability of the survey instrument through a confirmatory factor analysis (CFA) using a large-scale survey of in two states. When I find out that an activity involves . . . I feel . . .

  ☺
Being in a group 1 2 3 4 5 Being in a competition 1 2 3 4 5 Making or building things 1 2 3 4 5 Discovering and learning new things 1 2 3 4 5 Presenting in front of lots of people 1 2 3 4 5 Taking care of animals 1 2 3 4 5 Helping people learn things 1 2 3 4 5 We want to know what you think about each of the statements below. If you strongly agree, then choose 5. If you strongly disagree, then choose 1.
(Please circle only 1 number for each statement below)


Working with others is more fun that working alone. 1 2 3 4 5 I like being part of a team. 1 2 3 4 5

Examining Construct, Consequential, and Content Validity
To establish instrument validity, we used focus groups, pilot surveys, and educational policy Based on the discussion of validity in the Standards for Educational and Psychological Testing (AERA, APA, NCME, 2011, pp. 11-17) and other sources (e.g., Gall et al., 1996;Messick, 1994), we address three facets of validity: construct validity, content validity, and consequential validity.
Construct validity concerns the degree to which the survey instrument measures differences in learning activity preferences. This fundamental characteristic of validity concerns the question of whether individuals acknowledge that there exist different types of learning activities and that they assign different preferences to these types of learning activities. To this end, we carried out a pilot survey and focus group discussion with a fourth-grade class of 17 students.
After completing the survey, students were asked about their responses and their opinions about the questions. When the researcher asked students in a focus group what they thought the survey was about, a student responded, "It seemed like you [researcher] wanted to know what we like to do." When asked what types of activities they like to do and what activities they did not like to do, student responses included: "I don't like competitions." "I like working in groups." "I really don't like talking in front of the class." In one particular focus group, a student noted that the survey questions were redundant, i.e., the same question appeared to be asked more than once and asked, "Was it because you [researcher] wanted to be sure we were really thinking about our answers?" It appeared from this interaction that this student understood that the survey questions were intended to check for consistency in the students' survey responses.
Our analysis of the survey responses from the class revealed that students' responses were selfconsistent across the items within each learning activity type. Subsequent focus group discussions and pilot surveys of youth across ages produced comparable outcomes. These findings suggest that the conceptual framework and the survey instrument possess a reasonable degree of construct validity.
Both consequential and content validity were ascertained by focus group discussions with Content validity was examined by engaging with STEM educators through invited talks and workshops. The researchers engaged with informal program providers, schoolteachers, school principals, and educational researchers. The interactions during the presentations and the discussions that followed offered insights into the perspectives of other educators and helped the researchers to refine both the conceptual framework and survey instrument as well as establish content validity of both the conceptual framework and the survey instrument. These discussions let to more expansive definitions of "performing" and "caretaking" than had originally been considered.

Examining Reliability, Discriminant Validity, and Convergent Validity
In this next section, we consider the technical performance of the survey instrument in terms of reliability as well as discriminant validity and convergent validity through the application of CFA.
We explore the quantitative performance of the survey instrument with respect to reliability as well as discriminant and convergent validity of the seven-factor conceptual framework through its fit onto the data collected from students across Grades 3 through 12.

Participants
The data in this study were obtained from a survey administered in the fall of 2012 asking for participants' opinions about many statements regarding their preferences for learning activities falling within each of the seven types described above. Four participating school districts spanned the range of urban, suburban, and rural areas. The survey included students from

Measures
The seven types of active learning were considered as seven latent factors in the analysis ( Figure 1). For each of them, we developed three to five statements with 5-point Likert scale response options (strongly disagree, disagree, neutral, agree, and strongly agree) in the survey.
For the questions with only two response options, the responses were assigned values of 2 and 4 on the scale of 1 to 5. Negative statements were reverse-coded to maintain conceptual and statistical consistency with the other variables. Table 3 has a description of the survey items and variable names for the statistical analysis to follow. I like an activity that involves "Being in a group".

Col2
Working with others is more fun than working alone.

Col3
I like being part of a team.

Col4
I learn better when I am working with others.
Competing Com1 I like an activity that involves "Being in a competition".

Com2
I get excited when I hear there will be a competition.

Com3
I enjoy competing against other people.
Com4 I like to focus on my own goals, rather than competing with others.

Making Mak1
I like an activity that involves "Making or building things".

Mak2
I like doing projects where I make things.

Mak3
Whenever I can, I make the things I need.
Mak4 I like building things.
Discovering Dis1 I like an activity that involves "Discovering and learning new things ".
Dis2 I like figuring out how things work.

Dis3
I like taking things apart to see what is inside.

Dis4
I like trying different ways to figure things out.

Dis5
I like solving problems. I like an activity that involves "Presenting in front of lots of people".

Pre2
Performing in front of other people is fun.

Pre3
I like telling people about my work.

Pre4
I like presenting my work to my class.
Caretaking Car1 I like an activity that involves "Taking care of animals".

Car2
Having a pet is a big responsibility, but something I like to do.
Car3 I like to take care of things like plants and aquariums.
Teaching Tea1 I like an activity that involves "Helping people learn things".

Tea2
Helping others to learn things is fun for me.

Tea3
I like teaching things to others.
Tea4 I feel good when people depend on me.

Data Analysis
We used CFA to investigate the robustness of the measure of the seven latent constructs described in the conceptual framework. In preparation, we examined the data for possible outliers and for multicollinearity among 28 indicators of the seven factors. Cook's distance (D i ) was used to identify any potential outliers in the data at the significance level of 0.05 in F (k, n-k) , where k is the number of variables and n is the sample size; we did not find a multivariate outlier in F (28,7354) . In terms of multicollinearity, we computed variance inflation factors (VIFs) for 28 indicators and applied the convention that only variables with VIF value greater than 10 indicate multicollinearity (Neter et al., 1989). In this study, the VIF values for all 28 variables were between 1.21 and 3.27 with a mean of 2.03, supporting the conclusion that multicollinearity was not a concern.
To begin our CFA, we set our fit parameters according to the effects coding method of scaling (Little et al., 2006). These fit parameters were then used to determine the following criteria for good fit for the approximate fit indices: root mean square error of approximation (RMSEA) < .05, comparative fit index (CFI) > .95, and standardized root mean square residual (SRMR) < .08 (Hu & Bentler, 1999); and adequate (or acceptable) fit: .05 < RMSEA < .08 (Browne & Cudeck, 1993) and .95 > CFI > .90 (Bentler, 1990). According to convention we also reported chi-square values, where a value greater than 0.05 indicated that we retain the null hypothesis, which provided evidence that the data adequately fit the hypothesized model.

Results and Discussion
A seven-factor model was fitted to the data using maximum likelihood estimation which included the following latent constructs and corresponding observed variables: collaborating  Table A1 summarizes unstandardized and standardized estimates of parameters including their corresponding significance values. Factor loadings of indicators were greater than 0.45 and significant at the level of .05 in the CFA using a fixed factor model, which indicates the construct validation along with a good fit of the seven-factor model.
To examine the convergent validity and the factor reliability coefficients, we fitted an equivalent CFA model with the marker variable model fixing the first factor loading as 1 onto the data. We used Raykov's rho as a measure of factor reliability (see Appendix for details on computing this statistic). Table A2 provides a summary of the factor reliability measure findings, which show that the collaborating (0.837), competing (0.840), making (0.828), performing (0.848), teaching (0.820), discovering (0.753) and caretaking (0.763) constructs were reliable at a criterion of 0.7 (Nunnally & Bernstein, 1994). These results indicate that all of the seven constructs displayed convergent validity. The factor reliability measures reported here are comparable to the more traditional Cronbach's α and are a more appropriate measure for confirmatory factor analysis (Brown, 2006). The factor correlations shown in Table A2 were all less than .80, further supporting the discriminant validity of the seven factors (Brown, 2006).

Limitations of the Statistical Analysis
There are some important limitations to note in this analysis. First, the sample of 7,382 included students from a variety of schools across four school districts. While we have argued that this sample is representative of the population, it was not obtained through stratified or randomized sampling. This said, the sampling did include all students present in each participating school, with negligible numbers declining to participate. Second, use of Likert-type summated rating scales may be somewhat subjective to survey respondents and raises questions of reliability from survey to survey. We have addressed this concern by using multiple items to triangulate responses for each learning activity type. Considering these limitations, the confirmatory factor analysis results show that instrument reliability, discriminant validity, and convergent validity standards have all been met.
Another important limitation is the impact of confounding learning experiences that youth may have outside of structured, informal STEM programs. Some youth have family and friends who routinely engage them in STEM-related activities outside of structured programs. The instrument has no means of gathering this kind of information. Informal STEM educators must rely on alternative means to assess the impact of these factors. However, one important caveat should be kept in mind: Youth with this type of experience and mentorship are rare, and given that the results are viewed in aggregate, the impact of a few youths with these types of experiences will likely have an exceedingly small, if not negligible, impact on the aggregated analysis.

An Example for Practitioners: Applying the Conceptual Framework and Survey Instrument
One of the authors [Tai] uses the instrument to gather information about the impact of his course on elementary science teaching methods each academic year. The analysis of the data from one of these courses is used as an example here. This course enrolled 33 students, 29 of whom provided complete survey data. The four students not included in this analysis were late enrollees. Only the student responses including both pre and post surveys were analyzed. The semester-long course typically met twice weekly for a total of 28 classes, each lasting 75 minutes. The students in the course were a mix of pre-service elementary teacher candidates (n = 21) and undergraduate students (n = 8) enrolled in other schools in the university who were majoring in disciplines such as biology, urban planning, history, mechanical engineering, psychology, sociology, anthropology, and mathematics. All students were asked to complete the survey instrument on the 1 st day of class and again on the last day of class.
We began by surveying the students in the course to determine their active learning preferences. Table 4 includes the pre-course average preference values across the seven different types of active learning. How might a course instructor use the pre-course average preference values? The scales for each of the preference scores range from 1 (highly negative preference) to 5 (highly positive preference), with a score of 3 indicating no preference. On the lower end of the preference values, the average initial preference score for competing was 3.23 with a standard deviation of 0.80 indicating that the students had nearly no preference for engaging in competitions on the average. On the high end, the initial preference score for teaching was 4.87 (sd = 0.37). This result indicates that activities involving competition should be approached with care, since it appears that many students in the class are either ambivalent or report negative preferences for competing-type activities. To take this one step further, the individual student responses may be examined to look for the specific students who report strong negative preference scores. As a result, an instructor would have a heads-up regarding which students might potentially struggle with a particular assignment. It is important to note that active learning type preferences do not carry with them a negative stigma. Not preferring to compete or not preferring to collaborate is a status in a particular student's preferences which engagement in learning has the potential to change.
The teaching preference score was 4.87-nearly at the top of the scale. The aim is for the course activities to maintain this high preference score while still engaging the students in learning new content. Next, let us turn to the course activities. On the final day of the semester, students were given the same survey again to measure their post-course active learning preferences. The course averages for the preference scores are reported in Table 4 and compared directly in Figure 2.
The conceptual framework discussed earlier in this report is used to examine six course activities shown in Table 5. Not all course activities were included in this discussion. These course activities were designed to be used with college students to engage them in firsthand active learning experiences but may be modified easily for K-6 learners. The students' active learning experiences with these lessons were followed by class discussions aimed at engaging them with the pedagogical choices made by the instructor. Note that all seven active learning types are applied in the course activities in various combinations.

Figure 2. Graph of the Class Means for the Seven Learning Activity Types with Error Bars Displaying Standard Errors
Active Learning

Modes
Learning Activity Preference Ratings 123 The results for collaborating show a significant positive shift in students learning activity average preference. We can see in Table 5 that collaborating is included in five of six course activities. This result suggests that the course activities had a positive impact on students' average preference for engaging in collaborations, while the results also showed that six of the seven active learning preferences average scores (competing, making, discovering, performing, caretaking, and teaching) were static according to the t-test analysis used to compare pre-post course averages also shown in Table 4. To better understand these results, we take a closer look at the average scores themselves. Note that for teaching, discovering, and caretaking, the preference scores were high (i.e., above 4) and they remained high. This result indicates that the course activities were successful in maintaining students' overall positive preferences for teaching, discovering, and caretaking. For making, the average preference score was below 4 and remained below 4 which suggests that more work needs to be done to engage students in making-type activities. The characteristics identified in Table 5 indicate that making was included in four course activities: Slime and Silly Putty, Rube Goldberg, Stomp Rockets, and Egg Launch. Efforts to improve the making-type activities would be concentrated on these four course activities.
The weakest results were for performing and competing. The average preference scores were below 3.5. While the performing score remained nearly unchanged, the competing score average did show some improvement. These results suggest that efforts to improve engagement in performing-type activities should concentrate on the following course activities: Bean Plants, Hoverdisk, Rube Goldberg, Stomp Rockets, and Egg Launch. As for competing, the relevant course activities were: Hoverdisk, Rube Goldberg, and Egg Launch. Note that this analysis reveals that participants' two least preferred activity types (competing and performing) intersect in both the Rube Goldberg and Egg Launch activities. This result indicates that these two activities should be the ones that require the most scrutiny. This analysis resulted in these two course activities being modified with Rube Goldberg no longer engaging students in competing, but rather focusing on performing, while the students were given more time to complete the Egg Launch activity as well as having a class session devoted to testing their initial designs.

Conclusion
In this report, we have described a conceptual framework and survey instrument that can be useful to informal and formal STEM educators in evaluating and understanding the impact that their instructional programs have on the youth science learning engagement. The conceptual framework can be used to analyze program composition and structure. Pre-program surveys can identify the types of active learning that participants prefer or have reservations about. This information provides some actionable information for program facilitators and instructors. Youth who express strong positive preferences for active learning types included in a program have a clear path toward engagement and are likely to be highly engaged. On the other hand, youth who have strong reservations (negative preferences) are the ones to watch. They are the ones more likely to become disengaged during program activities they don't want to do. This information allows program facilitators and instructors to focus their attention on youth most likely to fall through the cracks.
After a program has been completed, post-program survey outcomes can be compared with pre-program survey outcomes to offer program facilitators and instructors insight into how their programs may have changed participating youths' preferences. Specific types of active learning may be shown in these pre-post survey comparisons to have a negative impact. This result, when coupled with the program analysis, will provide program facilitators and instructors information about which program activities may require reevaluation and modification.
Often informal STEM program evaluations include general science attitude questions, for example, "Science is helpful in understanding today's world" and "Science is something I enjoy very much" (Weinburgh & Steele, 2000); "I am sure of myself when I do science" and "I know I can do well in science" (Unfried et al., 2015). The responses to these questions offer some general participant impressions, but no cues from participants on where improvements might be made. When asked for suggestions on how to improve programs, many times participants are hesitant, concerned that their responses might be misconstrued as criticisms. The approach here offers a pedagogically focused means of analyzing programmatic impact on participant engagement with enough detail to allow for more focused programmatic reevaluation and improvement.

Factor Reliability Calculation: Raykov's Rho
The factor reliability was computed using the following formula originally given by Raykov (1997,2004) with notations in Klein (2011): . is the sum of the estimated unstandardized factor loadings among indicators of the same factor (Table A1), is the estimated factor variance (Table A2), and is the sum of the unstandardized error variances of those indicators. The factor reliability measure findings are summarized in Table A2.