{"id":6817,"date":"2018-02-08T09:00:27","date_gmt":"2018-02-08T16:00:27","guid":{"rendered":"http:\/\/minkhollow.ca\/beckerblog\/?p=6817"},"modified":"2018-02-07T18:07:39","modified_gmt":"2018-02-08T01:07:39","slug":"the-randomness-of-grades-what-is-an-a-student-2","status":"publish","type":"post","link":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/08\/the-randomness-of-grades-what-is-an-a-student-2\/","title":{"rendered":"The Randomness of Grades: Are all ‘B’s the Same?"},"content":{"rendered":"Approximate Reading Time: <\/span> 2<\/span> minutes<\/span><\/span>

Last time we looked at how different a score of 75% can be, depending on which 75% the student got right in an exam where the questions are ‘perfectly balanced’. BUT, what if the exam ISN’T perfectly balanced. What if, for example, we placed a heavier emphasis on the latter topics of the course? That’s a pretty common thing to do.<\/p>\n

\"\"<\/p>\n

Student 1<\/strong>:<\/h2>\n

The 2 graphics depict Student 1’s results on both exams. This student got 75% of the questions right in EACH topic. When we translate that to an ‘unbalanced exam, it’s not too bad. It’s possible to imagine that the first result indicates a similar level of competence as the second result. Both are ‘B’s.<\/p>\n

\"\"<\/p>\n

Student 2:<\/h2>\n

Student 2<\/strong> blew it in topics 2 & 6. Maybe they missed some classes or maybe they misunderstood, or maybe they were totally confused and too scared or embarrassed to ask for help (trust me, it happens).
\nIn this case, the total score is still close to the first score (155\/200 vs 150\/200), but in both samples it seems pretty clear that the level of competence is QUITE different from Student 1. Still, numerically, this is a ‘B’, which most would consider reasonably good.<\/p>\n

\"\"<\/p>\n

Student 3:<\/h2>\n

In this last example, the student ALSO blew 2 topics pretty completely BUT now the difference in score is far greater.
\nTHIS time, there’s a BIG difference. In both cases the student seemed to have completely missed or misunderstood Topics 7 & 8, but the score from the first test is 150\/200 = 75% (B), and the score from the second test is 115\/200 = 58% (D+).<\/p>\n

Further, we have NO IDEA from this result why<\/strong> <\/em>the student blew those two topics. Maybe, they got sick, or their child got sick, or their father died, or maybe, they just got bored and quit coming to class.<\/p>\n

The point here is that the\u00a0same<\/strong><\/em>\u00a0<\/em>student came away with a very different grade, and THIS student came away from the first exam with the same grade as the student who missed 75% of EVERY topic.<\/p>\n

\"\"<\/p>\n

Can you really claim to have confidence that all of these ‘B’ students have the same level of competence?<\/p>\n

Can you?<\/p>\n

So much for confidence in exams – especially in multiple choice exams.<\/p>\n

Note that I didn’t include a representation for what often happens with exam banks (especially those supplied by textbook writers) – when questions are chosen at random one often ends up with questions that ask the same thing, but really only differ in the way they are worded. A student who happens to know the answer to that question is at an advantage over the student who doesn’t.<\/p>\n

That said, I do think it is possible to come up with reasonable exams that fairly assess a student’s mastery of the topics in a course. It’s just that I no longer think most instructors take the time to make sure their exams really do that.<\/p>\n