EFFECTS OF RESPONSE LATENCY ON TEST ITEM PARAMETERS USING DIFFERENT SCORING PROCEDURES By
The study determined the prevalence of response latency behavior among undergraduates and examined its effect on item difficulty and discrimination indices 50 items EDU311 e exam across four scoring procedures as well as the difference in proportion of students that passed the test across scoring procedures. These were with the aim of profitably considering and integrating response latency into scoring process. The study adopted ex post facto research design. The study population comprised part three students of a university of education in Nigeria. The sample consisted of 1200 students that were selected using stratified sampling technique with college serving as strata. The research instrument consisted of dichotomously scored 50 items computerbased multiple choice EDU311 e exams. Collected data were analyzed using Confidence Interval, Frequency count, Percentages, Descriptive and ANOVA statistics. The results showed that that there is prevalence of response latency among undergraduate students in the studied university of education with 25% of the students engaging rapid guessing at least once. The result also showed that the difficulty (p^ and discrimination (rpb) indices values across the four scoring procedures differed significantly, (F (3, 196) 11.25, p 0.05) and F (3, 196) 8.57, p 0.05) respectively. Furthermore the results showed that the confidence intervals for the percentages of students who succeeded in the e exams were not significantly different from each other at the confidence level 95% Introduction Assessment of student learning is an important part of any educational process. The effectiveness of instruction as well as the instructional decisions largely relies on the ability of teachers to construct and select tests and assessments that would bring about valid, reliable, and fair measures of learning outcomes (Linn Gronlund, 1995) The nature and the quality of gathered information can control the educational development efforts and direct the instruction. Assessment of student learning in any educational settings is mainly aimed at measuring students' achievement as well as making series of decisions based on students' performance. The accuracy of such decisions may depend on the educational consequences to the students, the school, the society, policy makers, stakeholders or even the parents/guardian. Response latency (Rapid guessing) behavior is often observed when examinees do not put forth sufficient effort and time to answer test questions. When examinees are less motivated, they ISSN: 2411 5681 www.ijern.com 94 are unlikely to give adequate effort throughout the test which creates problems in presenting what they know and may result in biased estimates for ability and item parameter estimates. If a student does not put forth an adequate required effort, assessment results may underestimate the proficiency level and thus, it may lead to invalid interpretations of the obtained scores. Education policymakers and professionals often desire to use tests for multiple purposes, such as monitoring the educational system, aiding instructional planning, motivating students to perform better, acting as a mechanism to change instructional content, and holding schools and educators accountable. In addition, they use tests for certifying students as having attained specific levels of achievement (Hamilton, Stecher, Klein, 2002) Making inferences about a student's performance goes beyond the specific test that is used. Sometimes, teachers would like to know the degree of a student's understanding of specific concepts based on the score that he or she obtains on achievement tests, which corresponds to the knowledge and skills the student learns in the usual school subjects. Achievement tests have commonly been used to measure students' educational progress, but the number of purposes they expect to serve has grown substantially. Research reports overtime had made efforts toward explaining the inconsistency between student scores and the actual student's knowledge of the subject matter. Measurement experts intend to obtain valid test scores from each test administration. Whenever the examinee perceives that his or her performance has little or no consequences, it is more likely that he or she will put little effort into the assessment. Consequently, their scores may not accurately reflect their true abilities (Wise DeMars, 2003, 2005) This case represents a direct threat to test score validity, and experts have a responsibility for taking corrective action. Technology growth and recent developments in computer usage in the practice of assessment is providing great opportunities in exploring new ways that are geared towards improving the quality of assessment data (Klein Hamilton, 1999) Collection of additional information relating to the interaction between individual examinee and each of the items that made up the test is relatively made easy with Computer Administered Tests CAT. The time required for a response can give us a good indicator of the student's effort while testing in CAT. This is referred to as the "response latency." Low effort responses can be rescored in different ways to affect items' and persons' characteristics which may have an impact on educational decisions (Hadadi Luecht 1998) The use of computer to administer tests to a large extent expands the range of performance tasks that can be included in standardized tests and the information gathered during the testing session. The use of computer in test administration can easily provide additional data at the item level called "response latency," and behaviors related to the answer can be recorded. Response latency is the time used by an examinee when responding to an item. The capability to measure response latency at the item level appears to be easy and valuable. These response latencies can be used for both assessing the degree to which examinees give effort to the test and assessing the degree to which items receive good effort (Wise, 2006) Under a typical testing setting, some examinees will not reach the last questions on a test because of the time limit. Items of this sort are referred to as not reached. Even when items are reached and read, some examinees may appraise the item's content and decide for their own reasons not to respond. Items of this sort are referred to as omitted items. On the other hand, if the examinee realizes that there are no direct personal consequences based on his or her performance, the examinee may engage in random guessing response behavior (Mislevy and Wu, 1996) Wise (2006) shows that there are three primary choices of effort measure: examinee self reports, person fit statistics, and response time based measures. The use of self report instruments is the first method to detect examinee efforts, which seeks answer to questions such as, "I am not concerned about the score I receive on this test." International Journal of Education and Research Vol. 4 No. 5 May 2016 95 Administering and scoring self report instruments are easy; however, there may be problems when inferences are drawn based on student responses. For example examinees may be giving socially acceptable responses that do not match with what they actually believe. The information gathered through self reports may not correspond to the student's actual behavior during the examination. Also younger children may not understand the self report questions and be incapable of giving accurate responses. In some cases, students who did not try to do well on the test might falsely report giving good effort because of fear of disapproval or punishment from the test giver. Other students who believe they did not perform well on the test might underreport their effort because of a predisposition to attribute failure on the test to lack of effort over lack of ability (Pintrich Schunk, 2002; Wise, 2006) And, of course, students who choose not to expend the necessary energy on the test for a valid score may also choose not to expend the necessary energy on the selfreport instrument for a valid score. Examination of person fit statistics is the second method that can be used in detecting examinee effort. This method compares the examinee's response pattern with a theoretical measurement model. This method has a clear advantage of being based on an observed item response pattern rather than self report questions. The use of person fit indices, as an exploratory technique, applied typically in situations in which the kind of aberrant responses can be expected, is unknown, leads to numerous interpretations. According to Wise and DeMars, (2006) concluding unambiguously that a particular instance of misfit is due to lack of effort may be difficult. Misfit may indicate that the examinee engaged in cheating, creative responses, careless responses, guessing, or other unusual response behaviors. This is because it cannot always be confident of the kind of aberrant responses underlying test performance because different forms of aberrant behavior can have multiple explanations and may result in the same kind of item response pattern (Meijer, 1996) Thus, it becomes difficult to rely solely on person fit statistics to measure examinee effort. As a result of the short comings of the first and second methods of detecting examinee effort, Wise and Kong (2005) developed an index named Response Time Effort (RTE) to measure the examinee's test taking effort in computer based settings. The developed RTE was based on series prior research reports (Schnipke, 1999 and Schnipke Scrams 1997, 2002) that differentiates; appropriate solution oriented behavior (where the examinees actively seek to determine the correct answer to test items) from rapid guessing behavior (where the examinees determine the answer by rapid guessing; perhaps because they do not have enough time to fully think about the item) Wise and Kong in their study reported t