How CETYS Engineering is Preparing Its Students for Success

The Centro de Enseñanza Técnica y Superior (CETYS University) is a private institution established in 1961 in Baja California, Mexico under the auspices of a group of visionary entrepreneurs committed to education. CETYS Engineering recognized in the spring of 2016 that it needed a formal organization to provide third-party, external feedback for the advancement of the CETYS College of Engineering. An Engineering Advisory Council (EAC) was formed that spring. At the May 2017 meeting of the EAC it asked: “What competencies are needed for the engineer of tomorrow?” “How is CETYS preparing its engineering students to meet this challenge?” These two simple questions lead to a year-long study conducted by the CETYS College of Engineering. With a broad industry survey resulting in a sample size of 42 subject matter experts (SME’s) we determined what are the most important Domains of Attributes, Skills and Abilities an engineer needs. Using this domain prioritization, we then learned “What are the biggest competency gaps?” that industry is finding in entry-level engineers today from all universities. Using this gap analysis, we then looked at the CETYS curriculum and determined that almost all of the gaps are overtly addressed in multiple classes. Finally, the survey led the CETYS College of Engineering to take action on how address those gaps.


Introduction
CETYS Engineering understands its responsibility to provide the best talent possible to the private and public sectors of Baja and beyond. Universities do this in two fundamental ways: 1. By educating those who will define the future.
2. By generating the ideas and discoveries that can transform the present and build a better world.
But to do this mission effectively, we need to understand the competencies required by the student in the dynamic world of ours. Competency-based education is a modern trend based upon the premise that students must be able to demonstrate mastery of key concepts, attributes, skills and abilities. This is not a time-based educational platform as has been present in the past ~ e.g. "take 140 credit hours, average a "C" or better and you will graduate!" The U.S. Department of Education proclaims "competency-based learning leads to better student outcomes because the pace of learning is customized to each student."

Thinking with the End in Mind
In order to capture what industry sought, a detailed Engineering Competency-based Strategic Assessment Tool (CompSAT) questionnaire was developed by the study team at CETYS. We anticipated that the questionnaire results would serve many valuable roles: 1. Use the survey to drive "industry-prepared" CETYS graduates.
2. Use the survey to help CETYS hiring of new faculty.
3. Use the survey to support the Baja development of research clusters. 5. Use the survey to demonstrate the industry involvement in academic program creation in support of future accreditation efforts.
The questionnaire targeted the following fundamental question. Looking out 3 -5 years: "For an entry level engineer in your industry sector, he or she should have the following competencies upon graduation from CETYS" (In this case we define competencies as domain-specific knowledge that pertains to a specific industry sector.) By competencies we mean domain-specific knowledge that pertains to that industry. For example, under the competency of "DESIGN" • An entry level engineer entering the aerospace sector may need to know Design Thinking, requirements capture, product / process definition, Design -Manufacturing integration, mCAD, etc.
• An entry level engineer entering the maquiladora sector may need different competencies. He or she may need to know Design -Manufacturing integration but may only need to know Drafting rather than mCAD.

The Instrument Theoretical Foundation
The plan to develop this assessment began with a literature review. The literature taxonomy easily clustered into three categories: 1. What is the impact of competency-based learning?
2. What is the impact of a competent workforce in industry?
3. What assessment tools have been successful in developing answers?
We address each of these briefly.
Competency-based learning has existed for many years. The U.S. Department of Education (DOE) states "competencybased learning leads to better student outcomes because the pace of learning is customized to each student." The National Academy of Sciences emphasizes that the "engineer of tomorrow will be educated at institutions that can provide customized, tailored, and tuned education to the learner." Industry has been pushing competency-based learning because they need critical thinkers. But as the Wall Street Journal points out in 2017(Wall Street Journal, 2017), many colleges are failing to improve the critical thinking skills. "The challenge of working on complex problems will require systems designers with skills in basic science and math, multicultural and multidiscipline teamwork and a solid grounding in the humanities, social sciences and economics" was articulated by the National Academy of Sciences in 2004. Dano points out (Dano, E.B., 2019) that "corporate success mandates that a well-trained, experienced and competent workforce be employed." The Organization for Economic Co-operation and Development (OECD) published two reports highlighting the educational needs in Mexico: • In 2015 the OECD states that in Mexico "to unleash the country's potential requires a comprehensive programme to improve the skills" • In 2016 the OECD reinforced their comments by quantifying the skills gap in Mexico at 45% of the total workforce. This compares to 40% in the United States and 25% in China.
Finally, the Institution of Engineering and Technology (IET) in the United Kingdom Skills Survey of 2019 showed that: • 60% of companies responding said recruitment of engineering and technical staff with the right skills is the biggest barrier to achieve business objectives • Yet only 20% expect the supply of engineering skills coming into industry will improve.
As far back as 1999 have assessment methodologies been studied by Lang, et al.( Lang, James D, Susan Cruse, Francis D. McVey & John McMasters, 1999). Lang states that "quantitative methodologies such as formal surveys and structured interviews can be used to capture industry expectations of the needed attributes." Serna and her team in 2018 [14] found that many assessment methods exist. Their focus was on student inputs and impacts. The CETYS study complements this by using industry inputs and impacts. Finally, there was one caution that the CETYS team took seriously. Cruz, et al. (Cruz, Mariana Leandro, Gillian N. Saunders-Smits & Pim Groen, 2019) in 2019 point out that they "found 63% of the curriculum surveys used questionnaires." But they highlight that "questionnaires alone are not a good practice and should be used in combination with other methods" to validate the results. As you will find later CETY did just this by reviewing all the findings with the EAC in a formal review.
The CETYS Engineering Advisory Committee (EAC) study team developed a knowledge base of dozens of competency assessment instruments based upon the literature. What evolved was a composite of many of these. We call this composite instrument the CompSAT (Competency-based Strategic Assessment Tool). The underlying theory behind the CompSAT has been used in various forms for over a decade throughout North America to identify the gaps for a particular topical area. For example, we use the same format and gap analysis to assess the strategic process gaps in industry, the leadership gaps in industry, the next generation of manufacturing called "Industry 4.0" preparedness, the supply chain gaps, etc. Each of these instruments develops a customized list of questions to be asked of subject matter experts (SME). For each question we ask the "SME" to provide three (3) values: • Where are you Currently (C)on a scale of 1 to 5 what is the current value assigned to this question. A "1" represents a very basic knowledge of the topic while a "5" represents an ability to synthesize and evaluate from a deep understanding. The rubric is a modified Bloom scale.
• What is your Desired state (D)-again using the rubric of 1 to 5 what should this response be in three to five years? This rather long-range horizon was necessary because for the academic supply chain to effectively address any gaps, it will take some time to change the curriculum followed by a longer time to matriculate the students.
• If there is a Gap between the Current and the Desired, what would be the Importance ( I) to fill the gap?again using a rubric of 0 to 5 what is the value to the customer and the company if the gap between the Current and the Desired is closed? A value of 0 says there is no importance, a value of 1 would represent a very low value of importance while a 5 would represent such an important value that this could lead to a competitive advantage.
We then capture all the inputs and do a very simple calculation:

CompSAT Value = (Desired -Current) * Importance
Since this survey was targeting the engineering competencies needed by industry in Baja, Mexico we limited our focus to only the STEM professionals. (Science, Technology, Engineering and Math) professionals using the current vernacular.

Conducting the Survey
After we have identified our subject matter experts (SME's) we send them the questionnaire. We used Survey Monkey as the data capture method.
In Section 1 we asked the respondent to identify themselves. This series of questions included their industry segment which we used later to stratify the results.
In Section 2 of the questionnaire we ask the SME to review all of the technical competencies we have identified. The questions for each technical domain competency addressed: is this technical domain needed by your sector ~ and if so, how IMPORTANT is this STEM domain to your sector's success? If the "SME" thinks this is important enough to give the Domain a 3, 4 or 5 then we asked them to continue by expanding the Domain into the various Competencies.
In Section 3 of the questionnaire we ask the "SME" to review the ability of your STEM professionals to problem solve and think critically. Most of the SME's at companies have highlighted these skills as mandatory for a successful STEM professional.
In Section 4 of the questionnaire we ask that the SME review all of the academic skills STEM professionals will need as well. Here a similar two-part series of questions will be asked.
In Section 5 of the questionnaire we ask that the SME review all of the personal and workplace competencies a STEM professional may need.
In the questionnaire we ask the following: For each Domain: we are seeking the IMPORTANCE of this body of knowledge in your industry sector in THREE (3) TO FIVE (5) YEARS. The scale we ask you use is as follows: 0. No importance to our sector at all. IF the "SME" gave the Domain an Importance value of 3, 4, or 5, we asked them to dig deeper. For those Domains of importance (3, 4, or 5) we then asked which Competencies that make-up the domain are of the highest importance and where are the competency GAPS that we must fill. An example of the detailed competency assessment questions is found below for Sections 2, 3, 4 and 5. A complete competency questionnaire can be provided upon request.

Survey Participants by Industry Sector
Survey responses came from 42 subject matter experts (SME's) in a variety of industry domains in Baja, Mexico. The breakdown of these respondents is as follows:

Findings and Assessments
The response shown has 45% coming from aerospace. The reason is that this sector is actively involved in the competency definition and gaps. Although heavily skewed toward the Aerospace sector, the Subject Matter Experts (SME's) are known by the university personnel and are qualified in many of the sectors listed. Using the data collected we shall proceed to answer six questions with the data below: • What Domains of competency are found to be the most important for an entry level engineer?
• What are the single biggest gap(s) in competency for each of the prioritized Domains?
• What is the prioritized list of largest competency gaps?
• How does this prioritized list of Domains compare to TUEE?
o "TUEE" is the American Society for Engineering Education "Transforming Undergraduate Education in Engineering" series of workshops.
• What competencies are needed for the engineer of tomorrow?
• How is CETYS preparing its engineering students to meet this challenge?

Most Important Domains
These are the prioritized Domains ranked by their "I" Importance score. We used this finding to be the overarching focus for the survey. These industry sector Important Domains must drive our planning for the future. Domain Importance Assessment: With such broad technical knowledge requirements by different industry sectors, the specific industry Importance values focused more on overarching principles. The logical top priorities across all engineering disciplines are the ethical, teamwork, communication and professionalism domains. These are also found in all ABET and other accrediting body evaluation criteria for Schools of Engineering. We did a sector-by-sector analysis of Importance and concluded that the overarching principles of the "Whole Engineer" represented in Table 1 were reflective across sectors.

Ranking the Biggest Competency Gaps
These are the specific competency gaps within all the highest Importance I-score domains found above. This is an important, cross-cutting list comparing all the biggest gaps for all the Important Domains. Table 3. Rank Ordering the Biggest Competency Gaps for All Important Domains Biggest Competency Gap Assessment: The overall LARGEST GAP was "Understand How to Make Change Happen". This is a fundamental take-away from the Competency Assessment. We must infuse not only the analytical skill sets into our curriculum but add to this a deep understanding of how to make change happen in order for real, measurable accomplishments to be achieved.

Comparison of Domain Importance with TUEE Domain Importance
As introduced earlier TUEE (Transforming Undergraduate Education in Engineering) is a study conducted by the American Society of Engineering Education and the National Science Foundation. We compare the prioritized CETYS The results shown in Table 5 reinforce the coverage found in the CETYS Engineering curriculum. To get to the answer of these two EAC questions took a thorough analysis of Domain Importance and Competency Gaps found by our SME's. The conclusion is that CETYS Engineering can find multiple courses covering all of the highest priority Competency gaps except for three. These three are listed below with the CETYS actions to address.
1. Understand How to Make Change Happen -to incorporate how to make change happen into the curriculum is being addressed by introducing this topic and discussion in many of our classes. We have introduced in our undergraduate classes a complete lecture on this topic. But to validate if the student has mastered this is very difficult in an academic setting.
2. Works Well Alonethis study was conducted before the Corona Virus COVID 19 struck. Since classes were transformed to a remote learning format, we have established a living laboratory of CETYS students working alone. Feedback from the students has been mixed ~ many are flourishing in a "study at home" environment but several find the need for time management required when their cohort attends classes together. This experiment will continue into the Fall of 2020 with meaningful modifications. Feedback from the faculty was initially to push back ~ "students cannot learn the material from home". But this mindset has changed dramatically as the real need for remote learning materialized. Faculty of engineering embraced the change and made the transition to remote learning in a seamless manner. Currently CETYS is formalizing its plans on the role of remote learning in the total educational ecosystem.
3. Emotional Intelligence (EQ)just like the comments on Making Change Happen above, we are preparing our students to successfully work in teams, deliver to timelines and be able to support interpersonal relationships in a proper manner. The outcomes of the team projects, interactions with industry on internships and external assessments demonstrate that our students are emotionally sensitive and stable, for the most part.

The Overall Assessment
The CETYS Engineering Advisory Committee ask of the engineering program two questions: "What competencies are needed for the engineer of tomorrow?" "How is CETYS preparing its engineering students to meet this challenge?" These required a very detailed look at the competencies needed and how CETYS is meeting those needs. A rigorous assessment using a CompSAT instrument that allowed 42 SMEs in Baja, Mexico to contribute lead to findings of great value. Summarizing some of these findings: • It is no surprise that Ethical Responsibility, Domain 5.3, is the most important domain expected of an engineer.
o What is encouraging is that industry rates Ethics as Very Important but they are finding that engineering graduates have mastered this competency.
• What industry is telling us is that there are clusters of gaps found in engineers today: • These prioritized gaps are aligned closely with the TUEE study results.
• We find we can cluster these gaps into four (4) logical categories: o Cultivate the ability to make change happen o Develop the abilities to Solve Problems o Develop the abilities to communicate in many technical forms, e.g. assertive writing and analytical reading • Comparing these gaps with how CETYS's new competency-based program is designed we are delighted to find that almost all of the Top 21 gaps are formally addressed.
o The comparison of gaps versus CETYS formal coverage of the material shows only three of the twenty-one highest overall industry gaps found in entry level engineers are not addressed in multiple classes.
▪ Understand how to make change happen ▪ Works well alone ▪ Emotional intelligence o CETYS has launched efforts to determine how the three can be pedagogically addressed by intertwining new ideas and methods into the curriculum.
Per the recommendation of Cruz (Cruz, Mariana Leandro, Gillian N. Saunders-Smits & Pim Groen, 2019) the CETYS study team conclude the study with a formal review with the Engineering Advisory Committee. Prioritized domains, competency gaps and CETYS Engineering course gaps were discussed and validated.
In conclusion, the two "simple" questions from the CETYS EAC have been formally addressed in a rigorous manner. Priorities in terms of domains of competencies have been prioritized and found to be consistent with external studies such as TUEE. Looking deeper into the specific competencies found lacking by entry level engineers at various companies in Baja, Mexico we find that CETYS Engineering formally addresses all but three of the top 21 gaps. Plans to address these three have now be developed because the College of Engineering knows where to focus its attention.
The two questions from the EAC have been answered to the EAC's pleasure.