Answers.Body: <p>The <a href="http://rads.stackoverflow.com/amzn/click/B008PVQDCA" rel="nofollow noreferrer">Pedagogical Patterns</a> community has collected and published quite a lot of work on Feedback. The book referenced describes practices that have been vetted by the community. There are many more as well. Not all are related to exam grading, of course. Some that apply here are:</p>
  
  <ul>
  <li>Kinds of Exam  </li>
  <li>Self Test  </li>
  <li>Own Words</li>
  <li>Reduce Risk</li>
  <li>Differentiated Feedback  </li>
  <li>Mock Exam</li>
  <li>Grade It Again, Sam</li>
  <li>Fair Grading</li>
  </ul>
  
  <p>Other idea, that I don't think have yet been captured as patterns are thing like having students submit questions for an upcoming exam, probably with some reward for good questions. </p>
  
  <p>The pedagogical patterns community has continued the work also. Especially in the assessment area. A couple of workshops have been held in Europe, for example. Christian Köppe at the Han University in The Netherlands has been one of the drivers of the recent work. Newcomers welcome, of course. </p>
  
  <p>I agree, however, that a completely summative assessment has little value and gives you little guidance for improvement. </p>
  
  <p>But to the specifics of the question. If your class isn't too big, you could meet individually with those who need to improve. You could let students form small discussion groups (say 3 students each) to discuss answers on the exam. They need not reveal how they did, but could discuss how they approached the questions. Or you could meet with a small group rather than individuals to discuss the future and strategies for improvement. </p>
  
  <p>Of course, if the assessment vehicle is poor you need to work to improve that. It is possible for questions to be so bad that students are misled by them. I've written one or two of those in the past (alas). </p>
  
  <hr>
  
  <p>Some of the recent work can be found here: <a href="http://hillside.net/eduplop/patterns-publications/" rel="nofollow noreferrer">http://hillside.net/eduplop/patterns-publications/</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-04T20:11:15.840Z
Answers.LastActivityDate: 2017-07-05T09:57:14.313Z
Answers.LastEditDate: 2017-07-05T09:57:14.313Z
Answers.UserDisplayName: Buffy
Answers.Body: <p>First, I dispute your statement that <em>"the summative exam does nothing other than provide a coarse measure of the candidate's performance during the exam."</em>  Assessments are not merely chances for us to discover how the students are doing, they are also <a href="http://www.nytimes.com/2011/01/21/science/21memory.html?src=tptw" rel="nofollow noreferrer">useful in the learning process itself</a>.  They provide a focus for study, and they provide vital motivators for <a href="http://mrbartonmaths.com/resourcesnew/8.%20Research/Memory%20and%20Revision/The%20Critical%20Importance%20of%20Retrieval%20for%20Learning.pdf" rel="nofollow noreferrer">memory retreival</a>, and they provide the stakes that allow strong enough emotions to trigger memory.</p>
  
  <p><strong>Now, as for your question itself</strong>, I have also particularly struggled with summative assessments, and I ultimately concluded that I did not really believe in the model, and could not administer strict summative assessments in good conscience.  </p>
  
  <p>You've pinpointed the problem perfectly: once you've given a summative assessment, where do you go from there?  Instead, I utilize a test-retest model for all assessments provided throughout the year.   Every retest is provided with a lower maximum attainable grade.  </p>
  
  <p>At my current institution, I have calibrated this at 6 points per makeup.  (At a previous institution, I used 8 points.)  This is not a deduction, it is a cap.  So, earning a 65 on a first makeup will net you a 65, but earning a 98 will net you a 94.  My goal is to encourage targeted study, so I do not allow for further makeup penalties.  This means that the highest grade always wins, so you cannot lose points by taking a makeup.</p>
  
  <p>Now, I also create a practice quiz or test for every assessment.  This approach certainly involves a lot of work on my part (as I am <em>always</em> making 3 versions for practice-test-retest, and sometimes have to create 4, or even 5, assessments), but I love the net results I get in my class.  <strong>This system has virtually eliminated students falling seriously behind.</strong></p>
  
  <p>I do have a rule that students who fall below a certain grade (a) <em>must</em> take a makeup, and (b) must check in with me or a pre-approved student (roughly a TA) before they are allowed to take it.  This prevents the problem that some students have of simply taking makeup after makeup, with no appreciable improvement from one exam to the next.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-04T19:55:02.483Z
Answers.Text: I'm not so much in agreement with your first paragraph, but have successfully used a variation on your technique for years. It provides encouragement for the willing and a path past a brief brain-freeze. Total Approval overall.
Answers.UserDisplayName: Buffy
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-04T18:57:42.067Z
Answers.LastActivityDate: 2017-07-04T20:15:09.347Z
Answers.LastEditDate: 2017-07-04T20:15:09.347Z
Answers.UserDisplayName: Ben I.
Body: <p>Computer Science lends itself to a variety of different assessment methods - from the more practical assignment based programming problems, to the more formal final exams that cover the theoretical content. More formative assessments give students and educators the ability to improve learning and teaching whilst it is ongoing, but there is usually some concept of a "high-stakes" summative exam in most courses. I'm sure we can all agree that portions of one module (e.g. on data structures) will be exceptionally useful in later modules, and so it makes sense to do as much as possible to help students master these concepts.</p>
  
  <p>Often (from my own experience as a student), the "feedback" from the summative exam is a simple number or grade, which isn't broken down by assessment component. Further, there is usually no way to see the exam script or any form of mark scheme or question breakdown, which makes it difficult to improve in the general context of the university degree. The summative exam does nothing other than provide a coarse measure of the candidate's performance during the exam.</p>
  
  <p>Is this sort of status quo widespread throughout undergraduate level CS, and what can be done to give students the opportunity to learn from and reflect on these sort of assessments? Should a more formative approach be taken for the majority of each module?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-05T20:54:21.560Z
Comments.Text: Pupils do not improve when given summative feedback (it even cancels out the effect of any formative feedback), but they do when given formative feedback — I think I read this is evidence based teaching, G. Petty. Therefore give formative feedback: tell them what went well, and even better if, **but do not give them a grade**. Give them nothing that allows them to compare with each other. But give them info so they know which areas to study. I have an idea for a pie chart, where biggest area is proportional to study effort (not tried it yet).
Comments.UserDisplayName: ctrl-alt-delor
Tags: <undergraduate><assessment>
Title: How should students be encouraged to reflect and improve after summative assessment?
ThreadText: # How should students be encouraged to reflect and improve after summative assessment?
  
  (Asked by: username_0 on 2017-07-04)
  
  Computer Science lends itself to a variety of different assessment methods - from the more practical assignment based programming problems, to the more formal final exams that cover the theoretical content. More formative assessments give students and educators the ability to improve learning and teaching whilst it is ongoing, but there is usually some concept of a "high-stakes" summative exam in most courses. I'm sure we can all agree that portions of one module (e.g. on data structures) will be exceptionally useful in later modules, and so it makes sense to do as much as possible to help students master these concepts.
  
  Often (from my own experience as a student), the "feedback" from the summative exam is a simple number or grade, which isn't broken down by assessment component. Further, there is usually no way to see the exam script or any form of mark scheme or question breakdown, which makes it difficult to improve in the general context of the university degree. The summative exam does nothing other than provide a coarse measure of the candidate's performance during the exam.
  
  Is this sort of status quo widespread throughout undergraduate level CS, and what can be done to give students the opportunity to learn from and reflect on these sort of assessments? Should a more formative approach be taken for the majority of each module?
  
  **Tags:** undergraduate, assessment
  
  > **Comments:**
  > **username_1**: Pupils do not improve when given summative feedback (it even cancels out the effect of any formative feedback), but they do when given formative feedback — I think I read this is evidence based teaching, G. Petty. Therefore give formative feedback: tell them what went well, and even better if, **but do not give them a grade**. Give them nothing that allows them to compare with each other. But give them info so they know which areas to study. I have an idea for a pie chart, where biggest area is proportional to study effort (not tried it yet).  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2017-07-04. Score: 5 | Accepted answer
  
  First, I dispute your statement that *"the summative exam does nothing other than provide a coarse measure of the candidate's performance during the exam."* Assessments are not merely chances for us to discover how the students are doing, they are also [useful in the learning process itself](http://www.nytimes.com/2011/01/21/science/21memory.html?src=tptw). They provide a focus for study, and they provide vital motivators for [memory retreival](http://mrbartonmaths.com/resourcesnew/8.%20Research/Memory%20and%20Revision/The%20Critical%20Importance%20of%20Retrieval%20for%20Learning.pdf), and they provide the stakes that allow strong enough emotions to trigger memory.
  
  **Now, as for your question itself**, I have also particularly struggled with summative assessments, and I ultimately concluded that I did not really believe in the model, and could not administer strict summative assessments in good conscience.
  
  You've pinpointed the problem perfectly: once you've given a summative assessment, where do you go from there? Instead, I utilize a test-retest model for all assessments provided throughout the year. Every retest is provided with a lower maximum attainable grade.
  
  At my current institution, I have calibrated this at 6 points per makeup. (At a previous institution, I used 8 points.) This is not a deduction, it is a cap. So, earning a 65 on a first makeup will net you a 65, but earning a 98 will net you a 94. My goal is to encourage targeted study, so I do not allow for further makeup penalties. This means that the highest grade always wins, so you cannot lose points by taking a makeup.
  
  Now, I also create a practice quiz or test for every assessment. This approach certainly involves a lot of work on my part (as I am *always* making 3 versions for practice-test-retest, and sometimes have to create 4, or even 5, assessments), but I love the net results I get in my class. **This system has virtually eliminated students falling seriously behind.**
  
  I do have a rule that students who fall below a certain grade (a) *must* take a makeup, and (b) must check in with me or a pre-approved student (roughly a TA) before they are allowed to take it. This prevents the problem that some students have of simply taking makeup after makeup, with no appreciable improvement from one exam to the next.
  
  > **Comments:**
  > **username_2**: I'm not so much in agreement with your first paragraph, but have successfully used a variation on your technique for years. It provides encouragement for the willing and a path past a brief brain-freeze. Total Approval overall.  
  
  ---
  
  ### Answer by username_2 on 2017-07-04. Score: 2
  
  The [Pedagogical Patterns](http://rads.stackoverflow.com/amzn/click/B008PVQDCA) community has collected and published quite a lot of work on Feedback. The book referenced describes practices that have been vetted by the community. There are many more as well. Not all are related to exam grading, of course. Some that apply here are:
  
  * Kinds of Exam
  * Self Test
  * Own Words
  * Reduce Risk
  * Differentiated Feedback
  * Mock Exam
  * Grade It Again, Sam
  * Fair Grading
  
  Other idea, that I don't think have yet been captured as patterns are thing like having students submit questions for an upcoming exam, probably with some reward for good questions.
  
  The pedagogical patterns community has continued the work also. Especially in the assessment area. A couple of workshops have been held in Europe, for example. Christian Köppe at the Han University in The Netherlands has been one of the drivers of the recent work. Newcomers welcome, of course.
  
  I agree, however, that a completely summative assessment has little value and gives you little guidance for improvement.
  
  But to the specifics of the question. If your class isn't too big, you could meet individually with those who need to improve. You could let students form small discussion groups (say 3 students each) to discuss answers on the exam. They need not reveal how they did, but could discuss how they approached the questions. Or you could meet with a small group rather than individuals to discuss the future and strategies for improvement.
  
  Of course, if the assessment vehicle is poor you need to work to improve that. It is possible for questions to be so bad that students are misled by them. I've written one or two of those in the past (alas).
  
  ---
  
  Some of the recent work can be found here: <http://hillside.net/eduplop/patterns-publications/>
  
  
  ---
  
  