{"id":6781,"date":"2018-02-06T09:00:27","date_gmt":"2018-02-06T16:00:27","guid":{"rendered":"http:\/\/minkhollow.ca\/beckerblog\/?p=6781"},"modified":"2018-12-22T14:50:37","modified_gmt":"2018-12-22T21:50:37","slug":"grades-the-random-factor-problem-2","status":"publish","type":"post","link":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/06\/grades-the-random-factor-problem-2\/","title":{"rendered":"Grades: The Random Factor, Problem 2"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Approximate Reading Time: <\/span> <span class=\"rt-time\"> 2<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span><p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-6788 alignleft\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/sick-with-exam-fear-student-trying-to-cheat-at-exam.jpg\" alt=\"\" width=\"444\" height=\"250\" srcset=\"https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/sick-with-exam-fear-student-trying-to-cheat-at-exam.jpg 640w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/sick-with-exam-fear-student-trying-to-cheat-at-exam-300x169.jpg 300w\" sizes=\"auto, (max-width: 444px) 100vw, 444px\" \/><\/p>\n<p><a href=\"http:\/\/minkhollow.ca\/beckerblog\/2018\/02\/05\/grades-the-random-factor\/\">Problem 1, as outlined yesterday<\/a> is that we really have no objective way of ensuring that our exams actually &#8216;cover&#8217; the course content.<\/p>\n<p>Now, what about exams from the student&#8217;s perspective?<\/p>\n<p>I know, I know, tons of people have talked about issues with exams, but bear with me for a bit.<\/p>\n<p>Think about the actual conditions for the exam:<\/p>\n<ul>\n<li>Is everyone focused?<\/li>\n<li>Who has another exam the same day?<\/li>\n<li>What day of the week is it?<\/li>\n<li>Early morning vs. late afternoon.<\/li>\n<li>Is everyone healthy?<\/li>\n<li>Regularly scheduled vs. deferred vs accommodated?<\/li>\n<\/ul>\n<p>Can we assume all of these things affect students exactly the same way?<br \/>\nIf not, then our exam is not the same for everyone, is it?<\/p>\n<p>What about the kinds of questions we ask?<\/p>\n<ul>\n<li>All Multiple Choice?<\/li>\n<li>Subjective Questions?<\/li>\n<li>Are questions JUST like what they\u2019ve been practicing?<\/li>\n<li>Graded on a curve?<\/li>\n<li>Supplied by Textbook Author?<\/li>\n<li>Thoroughly analyzed?<\/li>\n<li>Your OWN assumptions\/Biases (we ALL have them).<\/li>\n<\/ul>\n<p>Here&#8217;s the real kicker:<\/p>\n<h2>Since there is no way we can reliably &#8216;cover&#8217; all of the course content in our exams, it is clear that we will be choosing some SUBSET of the content to put in the test. That means we are testing students on a subset of the course content.<\/h2>\n<hr \/>\n<h3 style=\"text-align: center;\"><strong><span style=\"color: #993366;\">Choosing what goes into that subset is inherently SUBJECTIVE.<\/span><\/strong><\/h3>\n<hr \/>\n<p><em><strong>What if they study the wrong subset?<\/strong><\/em><\/p>\n<p>How many times have you heard students complain that they studied things that weren&#8217;t on the exam? Is that a problem?<\/p>\n<p><strong>*I*<\/strong> think it is. How does this accurately reflect a student&#8217;s mastery of the material?<\/p>\n<p><em><strong>Student X<\/strong><\/em> studies all the right stuff and aces the exam, while <em><strong>Student Y<\/strong><\/em> studies the wrong stuff and barely passes. Now, let&#8217;s further suppose that <em><strong>Student X<\/strong><\/em> did an adequate job on their assignments, and <em><strong>Student Y<\/strong><\/em> did exceptionally well on theirs. The final is worth 50% of the course, but the assignments are only worth 25% Let&#8217;s say they both got the same score on the midterm.<\/p>\n<p>That means\u00a0<em><strong>Student X<\/strong><\/em> got: 50% (final) + 15% (midterm) + 15% (assignments) = 80% = A-<br \/>\n<em><strong>Student Y<\/strong><\/em> got: 25% (final) + 15% (midterm) + 25% (Assignments) = 65% = C<\/p>\n<h2><em><strong>Can you REALLY be sure that Student X is an &#8216;A Student&#8217;, and that Student Y is only a &#8216;C Student&#8217; based on a single exam on a single day?<\/strong><\/em><\/h2>\n<p><em><strong>REALLY?<\/strong><\/em><\/p>\n<p><a href=\"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/07\/the-randomness-of-grades-what-is-an-a-student\/\">Next, we&#8217;ll look at whether an &#8216;A&#8217; is always an &#8216;A&#8217;.<\/a><\/p>\n<div class='wp_likes' id='wp_likes_post-6781'><a class='like' href=\"javascript:wp_likes.like(6781);\" title='' ><img decoding=\"async\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/plugins\/wp-likes\/images\/like.png\" alt='' border='0'\/><\/a><span class='text'>Be the first to like.<\/span><\/p>\n<div class='like' ><a href=\"javascript:wp_likes.like(6781);\">Like<\/a><\/div>\n<div class='unlike' ><a href=\"javascript:wp_likes.unlike(6781);\">Unlike<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Approximate Reading Time: <\/span> <span class=\"rt-time\"> 2<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>Problem 1, as outlined yesterday is that we really have no objective way of ensuring that our exams actually &#8216;cover&#8217; the course content. Now, what about exams from the student&#8217;s perspective? I know, I know, tons of people have talked &hellip; <a class=\"more-link\" href=\"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/06\/grades-the-random-factor-problem-2\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[3,9,352,369,373,24],"tags":[389,126,387,15,16,393],"class_list":["post-6781","post","type-post","status-publish","format-standard","hentry","category-academia","category-educational-technology","category-gamification-2","category-higher-education","category-students","category-teaching-learning","tag-educational-technology","tag-gamification","tag-gamification-101","tag-higher-education","tag-instructional-design","tag-teaching-learning"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4Hsb6-1Ln","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":5016,"url":"https:\/\/minkhollow.ca\/beckerblog\/2015\/07\/18\/8-part-series-on-gamification-as-reigeluths-post-industrial-paradigm-of-instruction-part-6\/","url_meta":{"origin":6781,"position":0},"title":"8 Part Series on Gamification as Reigeluth&#8217;s Post-industrial Paradigm of Instruction: Part 6","author":"Katrin Becker","date":"July 18, 2015","format":false,"excerpt":"Criterion Referenced vs Norm Referenced Poor Norm. He is doomed to be compared against whoever signs up for the same class as him. Doesn't matter what he can do, it seems, it only matters how he\u00a0measures up to what everyone else can do. Norm-referenced assessment is still the norm (pardon\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/07\/2012-01-08-14-00-45_wm.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/07\/2012-01-08-14-00-45_wm.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/07\/2012-01-08-14-00-45_wm.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":6817,"url":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/08\/the-randomness-of-grades-what-is-an-a-student-2\/","url_meta":{"origin":6781,"position":1},"title":"The Randomness of Grades: Are all &#8216;B&#8217;s the Same?","author":"Katrin Becker","date":"February 8, 2018","format":false,"excerpt":"Last time we looked at how different a score of 75% can be, depending on which 75% the student got right in an exam where the questions are 'perfectly balanced'. BUT, what if the exam ISN'T perfectly balanced. What if, for example, we placed a heavier emphasis on the latter\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/unbalanced-1.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/unbalanced-1.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/unbalanced-1.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/unbalanced-1.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/unbalanced-1.png?resize=1050%2C600 3x"},"classes":[]},{"id":6839,"url":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/11\/thousands-of-teachers-caught-cheating-in-exams\/","url_meta":{"origin":6781,"position":2},"title":"Thousands of teachers caught cheating in exams","author":"Katrin Becker","date":"February 11, 2018","format":false,"excerpt":"I have long thought that the way to change the problem of cheating in exams (and assignments) is to reduce the risk associated with any single task. Stay tuned for future posts on how to do that. It turns out that THAT is one of the most profound outcomes of\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.thetimes.co.uk\/imageserver\/image\/methode%2Fsundaytimes%2Fprod%2Fweb%2Fbin%2Fff6373f0-0ea4-11e8-b553-b6f31437c43b.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.thetimes.co.uk\/imageserver\/image\/methode%2Fsundaytimes%2Fprod%2Fweb%2Fbin%2Fff6373f0-0ea4-11e8-b553-b6f31437c43b.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.thetimes.co.uk\/imageserver\/image\/methode%2Fsundaytimes%2Fprod%2Fweb%2Fbin%2Fff6373f0-0ea4-11e8-b553-b6f31437c43b.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":6159,"url":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/05\/grades-the-random-factor\/","url_meta":{"origin":6781,"position":3},"title":"Grades: The Random Factor","author":"Katrin Becker","date":"February 5, 2018","format":false,"excerpt":"How much of your students' grades in your courses is subject to random chance? Is your first reaction a defensive one? \"Why, NONE, of COURSE!\" We all like to think we are assessing our students fairly, and that the grades they get are some sort of true, objective reflection of\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2016\/01\/study-300x257.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":6409,"url":"https:\/\/minkhollow.ca\/beckerblog\/2017\/04\/03\/gameful-learning-table-of-contents-for-gamification-101\/","url_meta":{"origin":6781,"position":4},"title":"[Gameful Learning] Table of Contents for Gamification 101","author":"Katrin Becker","date":"April 3, 2017","format":false,"excerpt":"Here is the current Table of Contents for Gamification 101 (Book 1) Gamification 101: An Inquiry Based Journey Part 1 - Background Prologue What s Gamification? A Description of the Course Part 2 - The Journal Reflecting on Previous Versions Module Maps Scoring It All Adds Up On The Randomness\u2026","rel":"","context":"In &quot;Educational Technology&quot;","block_context":{"text":"Educational Technology","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/educational-technology\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2017\/04\/snap03222-300x212.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":6794,"url":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/07\/the-randomness-of-grades-what-is-an-a-student\/","url_meta":{"origin":6781,"position":5},"title":"The Randomness of Grades: What is an &#8216;A&#8217; Student?","author":"Katrin Becker","date":"February 7, 2018","format":false,"excerpt":"We all have a sense of what an 'A' Student is. They are the ones who have earned our top marks, of course. But comparing our own personal 'A's with that of other faculty quickly becomes problematic. All those complaints about 'grade inflation' are effectively us complaining that other people\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=1050%2C600 3x"},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/6781","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/comments?post=6781"}],"version-history":[{"count":12,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/6781\/revisions"}],"predecessor-version":[{"id":7203,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/6781\/revisions\/7203"}],"wp:attachment":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/media?parent=6781"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/categories?post=6781"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/tags?post=6781"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}