Assessment Unit in Medical Education: Structure and Function

Assessment is a vital part of any educational process through which the desired learning outcomes are measured. Therefore, assessment, particularly in health professions education, should be designed, implemented, and evaluated meticulously. This requirement necessitates a particular entity to govern the assessment process in medical schools. The aim of this article was to explore the reflection of the authors in establishing, conducting and monitoring the assessment of students, and to explore the related structure, function, challenges and areas for improvement. The authors reflected on their experience establishing an assessment unit (AU) in a college of medicine and explored its structure, function, challenges, and areas for improvements. The main aim of an AU should be the supervision of the assessment process in the college, monitoring student achievements that fulfill the programme’s learning outcomes, and developing appropriate policies and procedures for student assessment to ensure fairness and validity of the assessment process. Several challenges facing the AU were identified, including devoted full-time personnel and sufficient fund allocation for this unit to build a holistic assessment practice using advanced technology. The establishment of an entity to assess and evaluate medical students should be a high priority. An AU or a similar entity should lead all assessment activities. It should have the authority to plan, create, and evaluate assessments and other evaluative tools. Moreover, it should serve as a consultant for all disciplines in the college and provide clear guidance and training in assessment practices.


BACKGROUND The Assessment Unit: Importance and Rationale
Assessment is a vital part of any educational process. Through assessment, the desired learning outcomes are measured. In medical education, assessment is a crucial issue because graduates of medical schools are responsible for human lives (1,2). Assessment helps medical colleges to ensure that their graduates can safely care for people's lives. Therefore, an assessment of medical students should be designed, implemented and evaluated meticulously (3,4). Monitoring student assessment is similar to monitoring the curriculum (5). The best way to prove that intended learning outcomes of a curriculum are met is by showing rigorous evidence that describes how the assessment was performed for every outcome (6)(7)(8). Managing the entire assessment process means optimising quality standards, which are essentially the validity and reliability of the assessment (9). Validity refers to the presence of evidence to prove or disprove an implication about a specific characteristic an assessment intends to measure (10), while reliability refers to consistency in the obtained data from the assessment (11).
The assessment concerns two dimensions: formative and summative. Formative assessment should be continuous periods of different assessment modalities at different times throughout a given course (12). Its purpose is to familiarise students with these modalities and provide immediate feedback on their strengths and weaknesses. In contrast, summative assessment is designed to filter those students who master the intended learning outcomes in that particular course. Both types of assessment require a continuous supply of exam items from a question bank and logistic support from the exam control center (13,14). The suggested functions of an assessment unit (AU) should entail these two essential assessment continuums.
Trends and paradigms in medical education are strongly required to establish a particular entity to govern the assessment process in a medical school and storage facility for documents. Therefore, AU was established in the Qassim College of Medicine. Its main aim is to supervise the assessment process in the college, monitor student achievements that fulfill the programme's learning outcomes, and develop appropriate policies and procedures for student assessment to ensure the fairness and validity of assessment. This unit is expected to resolve issues in student assessment and suggest appropriate solutions; this may include the addition, deletion, or modification of some assessment modalities. Moreover, it handles the storage of all required documents and evidence for validity and reliability of the assessment. In this article, the authors reflect on and share their experiences with the development of the AU and describe its structure and function.
The aim of this article is to explore the reflection of the authors in establishing, conducting, and monitoring the assessment of students; and to explore assessments' structure, function, challenges, and areas for improvement.

METHODS
The authors reflected on their experience establishing an AU in a college of medicine and explored the organisational structure of the AU, functions of assessment committees within the AU, framework of the AU, human resources within the AU, physical resources of the AU, challenges and opportunities in establishing an AU in a medical school, and areas for improvement.

Organisational Structure of the AU
In the University of Qassim College of Medicine, an AU was established as a new entity under the Dean for Academic Affairs and chaired by the Head of the Medical Education Department to achieve the following mission: "ensuring the provision of fair and appropriate assessment that enhances achieving the intended learning outcomes of the programme." The AU comprises the following three committees: (a) the Student Assessment Committee (and its Formative Assessment Subcommittee), (b) the Central Exam Committee, and (c) the Question Bank Committee (Figure 1).

Functions of the Assessment Committees within the AU
Each of the abovementioned committees has its specific scope; they are functionally interrelated, and the unit as a whole oversees and monitors student assessment throughout the programme. The main functions of the different committees are shown in Table 1.  Table 1: Functions of the assessment subcommittees A. The Student Assessment Committee ensures validity and reliability of summative assessment to test the achievement of the intended learning outcomes. It involves the exam review process, pre-exam to evaluate the appropriateness of exam items, removal of any flawed items, and post-exam to evaluate the exam item analysis. It performs the following functions: 1. Develops an understanding of the role of the various tools for student assessment.
2. Provides guidance and support for appropriate assessment. 3. Ensures reliability and validity of the assessment methods, including both the pre-and post-exam. 4. Performs a psychometric analysis for every exam throughout the programme. 5. Suggests a pass/fail standard setting for each exam for the relevant department or committee. 6. Provides feedback to the relevant entities and encourages faculty to provide effective formative feedback to the students about their performance.
The Formative Assessment Subcommittee works under the guidance of the Student Assessment Committee. Its main role is to provide different formative assessment modalities that promote learning and serve to enhance the depth of understanding of students' knowledge through the provision of assessment for the learning modules. This subcommittee also ensures the coverage of areas in which student achievement is suboptimum. The functions of the Formative Assessment Subcommittee are as follows: 1. Helps and supports course organisers to provide different modalities of formative assessment. 2. Ensures frequent formative assessment questions and answers through online forums that address the core knowledge of students. 3. Encourages faculty to provide effective formative feedback to the students about their performance. 4. Plans for re-emphasis and subsequent coverage of areas in which student performance on the exam was below target. 5. Submits an annual report to the unit board.
B. The Central Exam Committee manages the logistics of the exam process and secures the storage of exam papers. The functions of the Central Exam Subcommittee are as follows: 1. Provides facilities to print and copy exam papers (by the course organiser or who he nominates) in a secure manner. 2. Allocates the appropriate locations for exams upon prior request of the course organisers. 3. Nominates faculty or staff for the invigilation process. 4. Supplies stamped bubble sheets for multiple choice questions (MCQs) before the exam and facilitates machine checking after the exam. 5. Arranges for a secure place or environment to review exams. 6. Maintains a copy (hard and soft) of each exam with model answers and a scoring guide. 7. Retains the exams of students for the previous two years. 8. Preserves records of the results (a soft and an officially-approved hard copy) of exams. 9. Scrutinises student appeals related to the assessment results and brings their outcomes to the Vice Dean for Academic Affairs. 10. Submits an annual report to the unit board.
C. The Question Bank Committee builds a systematised archive of high-quality exam questions to be reused as necessary. The functions of the Question Bank Subcommittee are as follows: 1. Establishes criteria for questions to be maintained in the bank and tags the questions. The criteria should be approved by the board of the AU. 2. Maintains a collection of questions with proven reliability and different levels of difficulty that cover all the learning objectives throughout the programme. 3. Ensures appropriate archiving of the questions so that a specific question can rapidly be retrieved when needed. 4. Adopts the necessary policy to ensure the security of the questions. 5. Supplies the Student Assessment Committee (or the task forces for each exam) with the necessary questions. 6. Encourages the faculty to prepare questions that target specific objectives as needed.

THE FRAMEwORK OF THE AU
The AU committees have independent and overlapping roles. The Student Assessment Committee is the key entity that handles the design and evaluation of the assessment format. Upon receiving exam items, either as "raw" from the exam item writer or from the question bank committee, these items undergo different levels of review and editing (15). The first level occurs at the block/course committee where a group of corresponding block/course teachers meet and review the received items. They agree on the initial draft of the exam after considering the quality guidelines for construction of the exam items. The second level involves a specific task force reviewing the exam appropriateness for the learning outcomes and the blueprinting weight ( Figure 2). This group is composed of independent members from academic courses to observe the exam from a different viewpoint without bias to the course. The main task of this group is to evaluate the distribution of exam items among the course learning outcomes. Additionally, this group has the authority to change some items to fit the learning outcomes.
After this level of the review, the exam is ready for final proofreading. This step is performed by other members who have a strong English background to determine any language errors. Once the exam is approved, it is sent to the Central Exam Committee for logistics preparation for a given exam. At this step, the exam is printed and securely stored. Exam logistics preparation is a continuous process throughout the entire academic year and begins at the end of the previous academic year. It includes preparation of the exam calendar with the dates and times of every exam in every course in the entire programme throughout all study levels. Then, exam locations are reserved, and invigilation assignments are distributed to all faculty members. During the exam, this committee is responsible for providing supervision during the exam and ensuring that the exam runs smoothly.
After the exam, students' answers are collected and reviewed under the guidance of members from the Central Exam Committee and Student Assessment Committee. If there are any concerns about the exam analysis, it is discussed with the corresponding exam item provider. Once the exam item analysis is clarified, exam results are approved. Exam items with good analysis results are redirected to the Question Bank Committee to be sorted and tagged appropriately for future use, while items on which students did not perform well, even when they addressed essential learning outcomes, are distributed to the Formative Assessment Subcommittee to be uploaded in the Blackboard platform in a discussion forum. The Formative Assessment Subcommittee continuously supplies block/course organisers with certain assessment modalities to be provided to students either in Blackboard or after each problem-based learning session.
By this way, the AU framework emphasises the role of the formative assessment as a powerful tool to enhance personalised education -moving away from standardised outcomes as practiced in summative assessment. After the results are announced, appeals are directed to the Central Exam Committee. Hence, a specific group is assigned to extract exam papers from storage and investigate the appealed issue.

Personnel within the AU
For the proper function of the AU, the following is required: one full-time staff member to serve as the secretary for the unit and its committees and faculty nominated as heads of the unit's committees that devote 20% from their workload to the unit. Members of the committees are expected to participate in 10% of their activities. Moreover, coordination with the faculty development units or medical education departments or centres is required to conduct workshops and provide training sessions in all areas related to assessment. In addition to sessions included in the faculty development programme, as-needed sessions should be delivered upon request by different entities in the college.

Physical Resources of the AU
In addition to the human resources, an office with a round meeting table (for five persons) is necessary for exam review and proper functioning of the AU. One room including a copy machine and a scanner for the response sheets is needed for the secretary. Furthermore, one large meeting room is required for evaluation, workshops and similar activities, and one secure room is necessary for the archive.
The following are also required: computers (for the exam bank and office work), stationary materials, a black-and-white printer, a colour printer, laptops for objective-structured practical exam, photocopiers and a display screen.

Challenges and Opportunities in Establishing AUs in Medical Schools
Many challenges, and at the same time many opportunities exist for the AU. Regarding challenges, the first and most important challenge is the resistance from departments in term of modifying or deleting some test items that come from these departments. To overcome such challenges, the AU should have great support from the dean and leaders of the medical college. The AU has the authority to finalise the exam to meet the learning outcomes and adhere to the item writing guidelines. Another challenge is that some AU members are biased to their departments. This bias becomes clear during the final selection of exam items. It was overcomed by careful selection of AU members, extensive training to avoid any bias interference, close monitoring by leaders, and continuous internal audits. Other challenges are obtaining full-time personnel devoted to the AU and extensive experience in student evaluation. These challenges remain an issue of concern (16) because recruiting and financial constrains. To overcome this issue, the Medical Education Department takes the upper hand to manage human resource in the AU. Some of its members were devoted to handle majority of AU's tasks. Another aspect is the financial challenge of allocating sufficient funds to this unit to build holistic assessment practices using advanced technology (17). Technology is required to minimise the use of papers and connect all assessment practices. To handle this issue, the AU uses Microsoft Excel to build master blueprint, calculate psychometric properties of test items, and enter and analyse test scores.
The AU has many opportunities. AUs can be independently audited to prove the quality aspects of student measurements (18,19). These audits can be performed internally and externally. The merit behind audits is the ability to monitor and evaluate the major task of AUs: "assessment of medical students" (14). Currently, there is an international recognition programme for excellence in education, called ASPIRE. ASPIRE provide criteria for excellence in different areas of medical education including assessment (6). These criteria create a competitive environment for medical schools to prove good quality practices in assessment. Another opportunity for this unit is collaborative work with either the faculty development unit or any entity that offers training to provide many training sessions in the assessment continuum. Assessment-related training needs are highly demanded in medical schools. Moreover, the AU should be the leading entity in research on student assessment and evaluation (20).

CONCLUSION
The design, implementation and evaluation of assessment in medical schools is challenging due to the importance of decisions made by assessment having the ability to affect the graduation of safe doctors. Establishing an entity for the assessment and evaluation of medical students should be a high priority. AUs or similar entities should lead all assessment activities. They should have the authority to plan, create and evaluate assessments and other evaluative tools. These units should also emphasises the role of formative assessment and continuously supply the relevant assessment items that should be considered during instruction. Moreover, they should serve as consultants for all disciplines in the college and provide clear guidance and training in assessment practices.