DEVELOPMENT AND IMPLEMENTATION OF PROGRAM OUTCOME ASSESSMENT TOOL FOR POLYTECHNIC COURSES

In the present scenario every polytechnic is mandatory to apply for NBA (National Board of Acreditation), so every course teacher has to find out concerned level of PO (Program Outcome) attainment for the concerned theory as well as practical courses. This paper proposes a simplified tool for determining PO attainment level using performance in direct methods viz. test examination, semester examination and practical examination and in indirect method rubrics undertaken for various activities as per CO (Course Outcome), PO mapping matrix which has been designed by concerned faculty. In order to find out the PO attainment, outcomes of a particular course are mapped with PO (PO1 to PO7). This mapping is a co-relation between CO’s and PO’s of each course over the scale of 1 to 3. The percentage attainment levels are defined over the scale of 1 to 3 to compute the values of Direct Assessment of each course by using CO-PO Matrix. In order to justify the proposed tool a typical case-study has been presented. The Attainment levels obtained with Average Scores of Indirect Assessment with Rubrics was higher for the courseFeedback Control system as compared to that of Indirect Assessment without Rubrics for each PO’s from PO1 to PO7. Keywords— Program Outcome, Course Outcome, CO-PO Matrix, CO-PO Attainment levels, rubrics assessment.


I. INTRODUCTION
Education is a form of learning in which the knowledge, skills and information are transferred from teachers to students. The transition from output based education to outcome based education is the real need and demand of the 21 st century learning system. Outcome Based Education (OBE) system has the ability to measure what the students are capable of. OBE is a student centered learning approach that focuses on empirical measurement of student performance (Rao and Nayak, 2015). In order to perform empirical measurement, learning objectives and outcomes are defined wherein objectives outline the expected results of teaching activities and outcome indicates the actual results that can be demonstrated and measured at the end of learning period (Md.Kamal and Latip, 2009).
NBA is a permanent signatory member of the Washington Accord (Abhijit kumar et al., 2019) that measures the competence of Indian technical institutions and ensures compliance with international standards. NBA has adopted OBE and provides accreditation to technical institutions. National Board of Accreditation (NBA), India, expects that the assessment of student's specific knowledge and skill should be based on the assessment and evaluation of the course outcomes (COs) and program outcomes(POs). Memon et al., 2009 reported that the accreditation process is to realize the value-addition in transforming students admitted to the program into capable technocrats, having sound domain knowledge and a satisfactory level of professional skills and attributes for ready employment in technical world. Assessment is very important in OBE. The overall achievement of OBE requires assessment of Programme Education Objectives (PEOs), Programme Outcomes (POs) and Course Outcomes (COs). The CO assessment forms the first step in calculating the assessment of POs and PEOs. Different tools such as examinations (verbal/written), assignments, mini projects, quiz etc. are used for the assessment of COs. Ideally question wise mapping of CO must be done and assessment must be done at that level.
The course outcomes are narrower statements that describe what students are expected to know and be able to do at the end of each course i.e. subject (Polimetla et al., 2014). Expected course outcome statements refer to specific knowledge, practical skills, areas of professional development, attitude, higher-order thinking skills, etc. that faculty members expect students to develop, learn, or master during a course. The course outcomes are mapped to Programme Outcomes which are subsequently mapped to Programme Education Objectives (Suskie, 2004).
Program outcomes (POs) are statements that the graduate of a formal engineering program should have the knowledge, skills and attitudes (attributes). POs are defined by aspects of behavior, and encompass multiple learning experiences. Present assessment of engineering education is mainly based on student's academic performance in the final examination, however, in light of OBE overall development of student is required to be assessed. Apart from including more assessment tools such as assignments, mini projects etc. framing of the assignments, examinations should be such that it will be able to assess the defined course outcomes. Further course outcomes should be assessed by direct and indirect assessment methods. In the direct assessment method the average academic performance of all students in the course is considered while in indirect assessment method surveys are taken and rubrics are designed to assess the course outcomes (Reddy and Andrade, 2010). A commonly used definition for rubric is a document that articulates the expectations for an assignment by listing the criteria or what counts, and describing levels of quality from excellent to poor (Andrade 2000;Stiggins 2001;Arter and Chappuis 2007). Rubrics help to make implicit assumptions and expectations more explicit. Rubrics offer a clear insight into the elements, assess, find reason and motivate students towards developing the competencies. Rubrics have been shown to provide high-level feedback (Nordrum et al. 2013). Catherine Hack (2013) found that the rubrics very helpful in clarifying performance and promoting self-assessment, whilst the tutors felt that it was a time efficient and informative method of providing feedback. William et al (2012) assessed the graduate attributes of problem analysis, design, individual and team work, communication skills, and economics and project management using rubrics. Present paper deals with development of a tool for Implementation of Program Outcome Assessment Tool for Polytechnic Courses which will assist course teacher to determine PO attainment levels. In following sections proposed evaluation criteria, guidelines for Rubrics assessment and a typical case study for demonstration of the proposed tool have been presented followed by Conclusions in the last section.

II. PROPOSED EVALUATION CRITERIA
The criteria for rubric evaluation system has divided in to seven Program-wise PO's i.e. PO1, PO2, PO3, PO4, PO5, PO6 and PO7. The parameter wise PO's were organised in PO1-PO7 and the students performances were evaluated based on the three point scale i.e. 1 indicates Slight (Low), 2 indicates Moderate (Medium) and 3 indicates Substantial (High). Concerned course teacher shall undertakeplan and monitor various activities / rubrics and evaluate as per following criteria: PO1: Basic and Discipline specific knowledge: Apply knowledge of basic mathematics, science and engineering fundamentals and engineering specialization to solve the engineering problems.
PO1 indicates the evaluation related to Basic and discipline specific knowledge, in this the students were assessed based on their ability to apply the knowledge of basic mathematics, science and engineering fundamentals and engineering specialization to solve the engineering problems based on the 3 point evaluation rating. Table 1 shows the evaluation about the students assessment based on basic and discipline specific knowledge. PO2: Problem analysis: Identify and analyses welldefined engineering problems using codified standard methods.
PO2 indicates the evaluation related to problem analysis, in this the students were assessed based on their ability to identify and analyse the well defined engineering problems using codified standard methods based on the 3 points evaluation rating. Table 2 shows the evaluation criteria about the student assessment based on problem analysis.

PO3
indicates the evaluation related to Design/development of solutions, in this the students were assessed based on their ability to Design solutions for welldefined technical problems and assist with the design of systems components or processes to meet specified needs based on the 3 point evaluation rating. Table 3 shows the evaluation criteria about the students assessment based on Design/development of solutions. PO4 indicates the evaluation related to Engineering Tools, Experimentation and Testing, in this the students based on their ability to Apply modern engineering tools and appropriate technique to conduct standard tests and measurements based on the 3 point evaluation rating. Table 4 shows the evaluation criteria about the students assessment based on Engineering Tools, Experimentation and Testing. PO5 indicates the evaluation related to Engineering practices for society, sustainability and environment, in this the students based on their ability to Apply appropriate technology in context of society, sustainability, environment and ethical practices based on the 3 point evaluation rating. Table 5 shows the evaluation criteria about the students assessment based on Engineering practices for society, sustainability and environment. PO6: Project Management: Use engineering management principles individually, as a team member or a leader to manage projects and effectively communicate about welldefined engineering activities.
PO6 indicates the evaluation related to Project Management, in this the students based on their ability to Use engineering management principles individually, as a team member or a leader to manage projects and effectively communicate about well-defined engineering activities based on the 3 point evaluation rating. Table 6 shows the evaluation criteria about the students assessment based on Project Management.  PO7 indicates the evaluation related to Life-long learning, in this the students based on their ability to analyses individual needs and engage in updating in the context of technological changes based on the 3 point evaluation rating. Table 7 shows the evaluation criteria about the students assessment based on Life-long learning.

III. RUBRICS ASSESSMENT
The Rubrics undertaken for individual course (Feedback Control System (Theory and Practical)by concerned course teacher includes various activities viz. Course related seminar/ task, Surveys, Activity for remembering the topic, Online test, Wall chart preparation, Project, Model / prototype preparation, Content beyond curriculum reading and Component /machine identification and learning specifications which requires planning, monitoring, evaluation and relevance with PO's (i.e.PO1, PO2, PO3, PO4, PO5, PO6 and PO7) and its average score. Table 8 shows the rubrics undertaken for individual course.

IV. ILLUSTRATION OF PROPOSED TOOL USING A CASE STUDY
In order to justify the proposed tool with a typical case study has been evaluated. Table 9 and Table 10 presents the mapping of course outcomes with program outcomes on the basis of their relevance based on three-point rating i.e. 1, 2 or 3 for the course: Feedback Control System (Theory &Practical) of fifth semester Diploma Program in Instrumentation Engineering. The average of all the mapped Course Outcomes (CO ' s) and Program Outcomes (PO's) has been taken and their levels have been decided by rounding up the rating to the nearest higher number.   Table 11 shows the details of Academic year, Semester, Total number of students, average marks of students, set target, students above set target, percentage of students above set target for the course Feedback Control System (Theory), course code (DIN 3101). The attainment level is presented in Table 11 as follows: as per set procedure the Attainment level 1 i.e. 40% of students scoring more than 40% marks; Attainment level 2 i.e. 45% of students scoring more than 40% marks; Attainment level 3 i.e. 50% of students scoring more than 40% marks. As presented in the earlier section the students have to appear for the Practical examination (is of 50 marks) as per Dr. Babasaheb Ambedkar Technological University norms. Pratical examinations were conducted in the dates in adherence to the academic calendar. Table 12 shows the details of Academic year, Semester, Total number of students, Average marks of students, target, students above target, percentage of students above target for the course Feedback control system (Lab Practice) and course code(DIN 3106).The attainment level is taken in Table 12 as follows: Attainment level 1 i.e. 50% of students scoring more than 60% marks; Attainment level 2 i.e. 55% of students scoring more than 60% marks; Attainment level 3i.e. 60% of students scoring more than 60% marks.
Generally, the indirect assessment is based on Alumni feedback, parents feedback and present student feedback. As per the improved method of analysis using the Rubrics which also incorporates the many activities i.e. like assignment, quiz and surprise test may add-on the PO attainment level which results into the better attainment. The indirect assessment by adding rubrics (by conducting various activities) and averaging it, which showed increase in PO attainment level. Table 13 shows PO attainment levels for the course Feedback Control System (Theory) and Feedback Control System (Lab Practice) after taking addition of 80% (average of two courses) marks of direct assessment and 20% marks of indirect assessment. It also indicated that the indirect assessment using rubrics by conducting various activities viz. assignments, quiz and surprise test, is 1.9, and considering 20% of this indirect assessment, then it will be 0.39.  Figure 2 shows the comparison of Average Scores of Indirect Assessment with Rubrics and Indirect Assessment without Rubrics for various POs. It can be seen from the graph that the scores obtained with Average Scores of Indirect Assessment with Rubrics was higher as compared to that of Indirect Assessment without Rubrics for each PO's from PO1 to PO7. Therefore the Indirect Assessment with Rubrics results into the better attainment level for all the PO's.

V. CONCLUSIONS
Paper proposes a tool for assessment of PO attainment levels, taking account of direct and indirect assessments, which will assist polytechnic faculty members. Case-study has been presented for easy illustration of the tool which reveals that the attainment levels obtained with Average Scores of Indirect Assessment with Rubrics was higher for the course Feedback Control system as compared to that of Indirect Assessment without Rubrics for each PO's from PO1 to PO7.