Answers.Body: <p>I use a lightly modified mastery learning system in my own classroom to make it more tenable in a regular classroom that has a curricular schedule.</p>
  <p>I'd recommend, as painful as it is, using the labs primarily for practice, and leaving the bulk of the grading to exams.  This is to fully allow students to utilize the labs as learning opportunities, and to allow for the fact that cheating has now become both incredibly easy to do, and fully impossible to detect.</p>
  <p>I'd then allow the labs to have unlimited opportunities.  This will give a chance for the strugglers to get as much out of them as they possibly can, even if they ultimately use the submission system as a bit of a crutch.</p>
  <p>What I do for mastery learning is to create an unlimited exam system, though with a caveat that there is a grade ceiling on each iteration of the exam.  The first exam has a ceiling of 100%, the second has a maximum grade of 92%, the next of 84%, and so forth.  The 8% step distance was calibrated to my current students and school - you may need a different step level to motivate your students.</p>
  <p>This ceiling is not a deduction, just a maximum, so if, on first retake, the student &quot;scores&quot; 96%, they earn the same 92% that someone who scores &quot;100%&quot; gets. The rationale is that subtracting makes it too difficult for students who struggle to feel like progress is being made while they do all of this extra work.  A student who earns a 50% the first time, studies, and then manages only 60% the second time retains their full 60%.  I do not average these grades - the later test simply wins.</p>
  <p>As a practical matter, I continue to move along while the students that need this extra support do their re-studying.  This allows me to cover all of the material I need to cover during the course term.  This is anecdotal, but in my experience, I have found a strong (perhaps unintuitive) result to all of this: the later material largely helps students who struggled with earlier material, and the students generally do not have substantial additional difficulty during the next unit of study.  The retesting makes sure that, by the time we've moved two units along, virtually everyone has caught up on the older material anyway, so the &quot;tail&quot; of weak understanding is kept acceptably short.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-26T16:35:24.357Z
Answers.Text: Thank you!  Great stuff!  How do you avoid the temptation to "brute-force" the answer to labs?  When labs are auto-graded, it's likely that there will be many multiple-choice questions.  When there are multiple-choice questions and students get unlimited opportunities and the auto-grader shows when their answer is correct, it is easy to re-submit multiple times, once with each possible answer, until you get it right.  The fact that it is so easy to do creates a temptation towards unhealthy practices.  My question is asking how to avoid this "attractive nuisance" (unhealthy temptation).
Answers.UserDisplayName: D.W.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-26T17:59:56.833Z
Answers.Text: @D.W. I guess I make my labs largely about coding, and my m.c. questions are largely on exams.  But in the spirit of "labs are purely for learning," I'd approach it by overtly mentioning to students that this is a way that they can approach the m.c. questions that way, but point out that the cost of submitting answers without fully understanding your own work comes later, on the exams. The labs provide carefully curated practical experience, and also help prepare students for the exams, and they should approach them in that spirit. Students who ignore this still have to pass the tests, so, .
Answers.UserDisplayName: Ben I.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-26T15:55:48.763Z
Answers.LastActivityDate: 2024-08-26T15:55:48.763Z
Answers.UserDisplayName: Ben I.
Body: <p>Consider a class where students submit their answers to assignments via <a href="https://guides.gradescope.com/hc/en-us/articles/23603175384717-Creating-an-Online-Assignment" rel="nofollow noreferrer">online assignments</a> in Gradescope.</p>
  <p>For pedagogical reasons, it is helpful to be able to <a href="https://guides.gradescope.com/hc/en-us/articles/23603175384717-Creating-an-Online-Assignment#h_01HN99V6ZD74YYZ0DPDD8QS71V" rel="nofollow noreferrer">provide explanations</a> to students about the correct answer, which are shown to the student if they select the correct answer.</p>
  <p>For auto-grading, it is typically most realistic to make most questions be multiple choice or select-all-that-apply, so that the student's solution can be auto-graded.</p>
  <p>For <a href="https://en.wikipedia.org/wiki/Mastery_learning" rel="nofollow noreferrer">mastery learning</a>, it is helpful to allow students to submit their solution at any point, with no deadline.</p>
  <p>However, this can lead to undesirable behaviour, where students brute-force the solution: they submit a solution, look at the feedback in explanations, and if they didn't get a question correct, try another answer and re-submit repeatedly with new answers each time until their answer is marked as correct.  This creates a temptation to behave in an unhealthy way.  It is not ideal to have classroom policies that create such temptations.</p>
  <p>Is there a solution that deters brute-forcing, without sacrificing the other benefits (feedback, mastery learning) for students who aren't trying to game the system?</p>
  
Tags: <grading><assessment><auto-graders>
Title: Immediate feedback in Gradescope, without gaming
ThreadText: # Immediate feedback in Gradescope, without gaming
  
  (Asked by: username_0 on 2024-08-22)
  
  Consider a class where students submit their answers to assignments via [online assignments](https://guides.gradescope.com/hc/en-us/articles/23603175384717-Creating-an-Online-Assignment) in Gradescope.
  
  For pedagogical reasons, it is helpful to be able to [provide explanations](https://guides.gradescope.com/hc/en-us/articles/23603175384717-Creating-an-Online-Assignment#h_01HN99V6ZD74YYZ0DPDD8QS71V) to students about the correct answer, which are shown to the student if they select the correct answer.
  
  For auto-grading, it is typically most realistic to make most questions be multiple choice or select-all-that-apply, so that the student's solution can be auto-graded.
  
  For [mastery learning](https://en.wikipedia.org/wiki/Mastery_learning), it is helpful to allow students to submit their solution at any point, with no deadline.
  
  However, this can lead to undesirable behaviour, where students brute-force the solution: they submit a solution, look at the feedback in explanations, and if they didn't get a question correct, try another answer and re-submit repeatedly with new answers each time until their answer is marked as correct. This creates a temptation to behave in an unhealthy way. It is not ideal to have classroom policies that create such temptations.
  
  Is there a solution that deters brute-forcing, without sacrificing the other benefits (feedback, mastery learning) for students who aren't trying to game the system?
  
  **Tags:** grading, assessment, auto-graders
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-08-26. Score: 1
  
  I use a lightly modified mastery learning system in my own classroom to make it more tenable in a regular classroom that has a curricular schedule.
  
  I'd recommend, as painful as it is, using the labs primarily for practice, and leaving the bulk of the grading to exams. This is to fully allow students to utilize the labs as learning opportunities, and to allow for the fact that cheating has now become both incredibly easy to do, and fully impossible to detect.
  
  I'd then allow the labs to have unlimited opportunities. This will give a chance for the strugglers to get as much out of them as they possibly can, even if they ultimately use the submission system as a bit of a crutch.
  
  What I do for mastery learning is to create an unlimited exam system, though with a caveat that there is a grade ceiling on each iteration of the exam. The first exam has a ceiling of 100%, the second has a maximum grade of 92%, the next of 84%, and so forth. The 8% step distance was calibrated to my current students and school - you may need a different step level to motivate your students.
  
  This ceiling is not a deduction, just a maximum, so if, on first retake, the student "scores" 96%, they earn the same 92% that someone who scores "100%" gets. The rationale is that subtracting makes it too difficult for students who struggle to feel like progress is being made while they do all of this extra work. A student who earns a 50% the first time, studies, and then manages only 60% the second time retains their full 60%. I do not average these grades - the later test simply wins.
  
  As a practical matter, I continue to move along while the students that need this extra support do their re-studying. This allows me to cover all of the material I need to cover during the course term. This is anecdotal, but in my experience, I have found a strong (perhaps unintuitive) result to all of this: the later material largely helps students who struggled with earlier material, and the students generally do not have substantial additional difficulty during the next unit of study. The retesting makes sure that, by the time we've moved two units along, virtually everyone has caught up on the older material anyway, so the "tail" of weak understanding is kept acceptably short.
  
  > **Comments:**
  > **username_0**: Thank you!  Great stuff!  How do you avoid the temptation to "brute-force" the answer to labs?  When labs are auto-graded, it's likely that there will be many multiple-choice questions.  When there are multiple-choice questions and students get unlimited opportunities and the auto-grader shows when their answer is correct, it is easy to re-submit multiple times, once with each possible answer, until you get it right.  The fact that it is so easy to do creates a temptation towards unhealthy practices.  My question is asking how to avoid this "attractive nuisance" (unhealthy temptation).  
  > **username_1**: @D.W. I guess I make my labs largely about coding, and my m.c. questions are largely on exams.  But in the spirit of "labs are purely for learning," I'd approach it by overtly mentioning to students that this is a way that they can approach the m.c. questions that way, but point out that the cost of submitting answers without fully understanding your own work comes later, on the exams. The labs provide carefully curated practical experience, and also help prepare students for the exams, and they should approach them in that spirit. Students who ignore this still have to pass the tests, so, .  
  
  ---
  
  