I have an education question for anyone with expertise in assessment:
Are there any studies examining the notion that marking all of one assignment/paper contemporaneously leads to more consistency?
It strikes me as intuitively true, but I’d love to find citations to studies that have examined this.
So far, I’m turning up empty.
Is it possible that NO-ONE has studied this claim to see if it actually has merit?
I’m not saying it isn’t true; I am asking how we can be sure it *IS* true?
For my part, I can say from 1st hand personal experience that I DO NOT grade the first exam in the exactly the same way as I grade the 20th, no matter how much I may try to.
Since I gave up using hard deadlines in my classes, I get assignments submitted to me throughout the term. Also, since fast turn-around of feedback is very important in my classes, I mark within a few days of those submissions. I have found that, at least, for me, I do the best job of marking on the FIRST assignment when I am doing a bunch at the same time. I don’t like that that’s true, but it is. When I get things handed in at various times, and when I mark them as they come in, I end up effectively marking EVERYONE’s assignment as if it were the first one.
Has anyone studied this?
Pingback: What?! No Deadlines on Assignments?! | The Becker Blog
I don’t grade exams, just quizzes and long design reports.
For quizzes, there is generally very little difference in grading quality from one student to another—I do the whole stack in about 2 hours and if I find myself changing my opinion about partial credit, I go through the whole stack to ensure consistency.
For long reports, which take me 30–60 minutes each to grade, there is a huge change between the first one and the 43rd one. The first one is generally not the best graded—that comes around the 10th one, when I’ve realized what the common misunderstandings are this time around but I haven’t burned out yet. My grading tends to get harsher if I’ve had a run of really bad papers in a row, and I often have to take deliberate breaks to reset after each bad paper.
Back when I had only 20–30 10-page papers to grade and they were on paper, I could go back over them, flipping through them quickly to regrade them for consistency. Now that I have 40 or more, and they are PDF files on the badly misnamed “SpeedGrader” system, it is way too slow to flip through the papers (changing which one I’m looking at takes 10–60 seconds, depending on the length of the file). So I find myself looking at only about a quarter of the papers a second time. Consistency has gone down, but I’m already spending so much of my time grading that it can’t be helped—I don’t have the time to do quality control any more.
I hear you!
I love your observation about “hitting your stride” around the 10th paper. I hadn’t thought of it that way, but I suspect I do some of that too.
I also think grading papers on paper is probably better than doing it online.
I have largely shifted away from grading long tomes, to shorter, easier to mark things.
I’ve also become a huge fan of having my students produce ‘post mortems’, where they talk about what went right and what they’d do differently. It’s interesting to see how often they already know the things we would often say as feedback.