{"id":6159,"date":"2018-02-05T09:00:31","date_gmt":"2018-02-05T16:00:31","guid":{"rendered":"http:\/\/minkhollow.ca\/beckerblog\/?p=6159"},"modified":"2018-12-22T14:39:25","modified_gmt":"2018-12-22T21:39:25","slug":"grades-the-random-factor","status":"publish","type":"post","link":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/05\/grades-the-random-factor\/","title":{"rendered":"Grades: The Random Factor"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Approximate Reading Time: <\/span> <span class=\"rt-time\"> 2<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span><p><img loading=\"lazy\" decoding=\"async\" class=\"size-medium wp-image-6773 alignleft\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2016\/01\/study-300x257.png\" alt=\"\" width=\"300\" height=\"257\" \/>How much of your students&#8217; grades in your courses is subject to random chance?<\/p>\n<p>Is your first reaction a defensive one?<\/p>\n<p>&#8220;Why, NONE, of COURSE!&#8221;<\/p>\n<p>We all like to think we are assessing our students fairly, and that the grades they get are some sort of true, objective reflection of their mastery of our course content.<\/p>\n<p><em><strong>Think again.<\/strong><\/em><\/p>\n<p>Let&#8217;s start with exams. Of necessity, you can&#8217;t ask questions about every single thing you covered(*) in class, assignments, and readings. It&#8217;s just not practical.<\/p>\n<p style=\"text-align: right;\"><em>(*It&#8217;s a handy fantasy to think that just because you &#8216;covered&#8217; it, they &#8216;learned&#8217; it.<br \/>\nIt just ain&#8217;t so.)<\/em><\/p>\n<p>Let&#8217;s imagine how it would look if you\u00a0<em><strong>did<\/strong><\/em> create a comprehensive test.<\/p>\n<p>This is a super simplified representation of the course content in my course. I have 8 topics, and each square represents the\u00a0<em><strong>stuff<\/strong><\/em> I covered.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-6774\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2016\/01\/2018-02-04-12_47_27-C1.2017-Grades-and-The-Random-Factor.pptx-PowerPoint.png\" alt=\"\" width=\"1080\" height=\"357\" srcset=\"https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2016\/01\/2018-02-04-12_47_27-C1.2017-Grades-and-The-Random-Factor.pptx-PowerPoint.png 1080w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2016\/01\/2018-02-04-12_47_27-C1.2017-Grades-and-The-Random-Factor.pptx-PowerPoint-300x99.png 300w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2016\/01\/2018-02-04-12_47_27-C1.2017-Grades-and-The-Random-Factor.pptx-PowerPoint-768x254.png 768w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2016\/01\/2018-02-04-12_47_27-C1.2017-Grades-and-The-Random-Factor.pptx-PowerPoint-1024x338.png 1024w\" sizes=\"auto, (max-width: 1080px) 100vw, 1080px\" \/><\/p>\n<p>If I create a truly comprehensive exam, then I need to ask things on my exam that touch on\u00a0<em><strong>every single<\/strong><\/em> <strong>Bit<\/strong> of content. Again, in my super simplified representation, each of my 8 topics has 25 <strong>Bits<\/strong> to it, and so my (let&#8217;s say Multiple Choice) exam should have 200 questions, each one addressing a different &#8216;<strong>Bit<\/strong>&#8216;.<\/p>\n<p>That&#8217;s completely untenable.<\/p>\n<p>So your exam, of necessity, picks on a small percent of the course content. Even in a 3-hour exam (which, quite frankly, I think is both ridiculous and cruel), you can only touch on a small portion of the course content. Let&#8217;s guess you manage to ask questions that touch on a third of the course content.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-6777\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/pic2.png\" alt=\"\" width=\"1390\" height=\"746\" srcset=\"https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/pic2.png 1390w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/pic2-300x161.png 300w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/pic2-768x412.png 768w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/pic2-1024x550.png 1024w\" sizes=\"auto, (max-width: 1390px) 100vw, 1390px\" \/><\/p>\n<p>Each &#8216;<strong>Bit<\/strong>&#8216; from the content appears in your exam. In the above image, each &#8216;<strong>Bit<\/strong>&#8216; that appears in the exam has been replaced with a white square in the original course content. Can you see how much of your content\u00a0<strong>didn&#8217;t<\/strong> make it into the exam?<\/p>\n<ul>\n<li>How do you decide which third to include in the exam and which 2\/3 to leave out?<\/li>\n<li>Does your textbook question bank have any influence on which questions you choose?<\/li>\n<li>Do you let an automated test bank choose random questions for you?<\/li>\n<li>How much do you trust it?<\/li>\n<li>Have you ever even thought about that?<\/li>\n<li>Do you know whether or not the author of your textbook did any statistical analysis on the reasonableness or predictive qualities of their questions?\u00a0<em>(It might surprise\u00a0you to know that many authors simply make up questions without ever testing them on real students.)<\/em><\/li>\n<li>Do you ask questions that you believe &#8216;cover&#8217; multiple <strong>Bits<\/strong>?<\/li>\n<li>Have you ever verified this?<\/li>\n<\/ul>\n<p>On top of that, also keep this in mind when you look at the &#8220;<strong>Bits<\/strong>&#8221; of your course content. Your course consists of:<\/p>\n<ul>\n<li>39 Lecture hours\n<ul>\n<li>Is everything you say in class relevant course content?<\/li>\n<li>If not, what isn&#8217;t, AND how do students know the difference?<\/li>\n<\/ul>\n<\/li>\n<li>13 Tutorial Hours\n<ul>\n<li>Are you doing your own tutorials?<\/li>\n<li>If not, what assumptions do you make about what&#8217;s going on in the tutorial?<\/li>\n<li>How knowledgeable are your teaching assistants w.r.t. the course content?<\/li>\n<li>How well do your teaching assistants match YOUR teaching style?<\/li>\n<li>Are all your teaching assistants equally prepared and dedicated to helping your students master the course material?<\/li>\n<li>Do you include material from the tutorial in your exam?<\/li>\n<\/ul>\n<\/li>\n<li>450 pg. Text Book\n<ul>\n<li>How much of the textbook is relevant course content?<\/li>\n<li>Is it only certain chapters, or is it certain parts of some (or all) chapters?<\/li>\n<li>Do your students know EXACTLY what parts you consider to be course content and what parts are not?<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>That&#8217;s only PROBLEM 1<\/strong><\/p>\n<p><a href=\"http:\/\/minkhollow.ca\/beckerblog\/2018\/02\/06\/grades-the-random-factor-problem-2\/\">Tune in tomorrow for Problem 2.<\/a><\/p>\n<div class='wp_likes' id='wp_likes_post-6159'><a class='like' href=\"javascript:wp_likes.like(6159);\" title='' ><img decoding=\"async\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/plugins\/wp-likes\/images\/like.png\" alt='' border='0'\/><\/a><span class='text'>Be the first to like.<\/span><\/p>\n<div class='like' ><a href=\"javascript:wp_likes.like(6159);\">Like<\/a><\/div>\n<div class='unlike' ><a href=\"javascript:wp_likes.unlike(6159);\">Unlike<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Approximate Reading Time: <\/span> <span class=\"rt-time\"> 2<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>How much of your students&#8217; grades in your courses is subject to random chance? Is your first reaction a defensive one? &#8220;Why, NONE, of COURSE!&#8221; We all like to think we are assessing our students fairly, and that the grades &hellip; <a class=\"more-link\" href=\"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/05\/grades-the-random-factor\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[3,9,352,369,373,24],"tags":[389,126,387,15,16,393],"class_list":["post-6159","post","type-post","status-publish","format-standard","hentry","category-academia","category-educational-technology","category-gamification-2","category-higher-education","category-students","category-teaching-learning","tag-educational-technology","tag-gamification","tag-gamification-101","tag-higher-education","tag-instructional-design","tag-teaching-learning"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4Hsb6-1Bl","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":6794,"url":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/07\/the-randomness-of-grades-what-is-an-a-student\/","url_meta":{"origin":6159,"position":0},"title":"The Randomness of Grades: What is an &#8216;A&#8217; Student?","author":"Katrin Becker","date":"February 7, 2018","format":false,"excerpt":"We all have a sense of what an 'A' Student is. They are the ones who have earned our top marks, of course. But comparing our own personal 'A's with that of other faculty quickly becomes problematic. All those complaints about 'grade inflation' are effectively us complaining that other people\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/course-content.png?resize=1050%2C600 3x"},"classes":[]},{"id":6781,"url":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/06\/grades-the-random-factor-problem-2\/","url_meta":{"origin":6159,"position":1},"title":"Grades: The Random Factor, Problem 2","author":"Katrin Becker","date":"February 6, 2018","format":false,"excerpt":"Problem 1, as outlined yesterday is that we really have no objective way of ensuring that our exams actually 'cover' the course content. Now, what about exams from the student's perspective? I know, I know, tons of people have talked about issues with exams, but bear with me for a\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2018\/02\/sick-with-exam-fear-student-trying-to-cheat-at-exam.jpg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":6409,"url":"https:\/\/minkhollow.ca\/beckerblog\/2017\/04\/03\/gameful-learning-table-of-contents-for-gamification-101\/","url_meta":{"origin":6159,"position":2},"title":"[Gameful Learning] Table of Contents for Gamification 101","author":"Katrin Becker","date":"April 3, 2017","format":false,"excerpt":"Here is the current Table of Contents for Gamification 101 (Book 1) Gamification 101: An Inquiry Based Journey Part 1 - Background Prologue What s Gamification? A Description of the Course Part 2 - The Journal Reflecting on Previous Versions Module Maps Scoring It All Adds Up On The Randomness\u2026","rel":"","context":"In &quot;Educational Technology&quot;","block_context":{"text":"Educational Technology","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/educational-technology\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2017\/04\/snap03222-300x212.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":7168,"url":"https:\/\/minkhollow.ca\/beckerblog\/2018\/12\/10\/is-it-what-straight-a-students-get-wrong-or-is-it-what-we-get-wrong\/","url_meta":{"origin":6159,"position":3},"title":"IS it What Straight-A Students Get Wrong, or is it what WE get wrong?","author":"Katrin Becker","date":"December 10, 2018","format":false,"excerpt":"It's not the students' fault. Not making marks random, and reducing the risks of failure are what gives students more room to really think. We owe them that. We need to get rid of compartmentalized grading entirely, and quit blaming the students for responding appropriately to an unreasonable assessment system.\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4991,"url":"https:\/\/minkhollow.ca\/beckerblog\/2015\/07\/13\/8-part-series-on-gamification-as-a-post-industrial-paradigm-of-instruction-part-1\/","url_meta":{"origin":6159,"position":4},"title":"8 Part Series on Gamification as Reigeluth&#8217;s Post-industrial Paradigm of Instruction: Part 1","author":"Katrin Becker","date":"July 13, 2015","format":false,"excerpt":"In his landmark paper describing what the new post-industrial paradigm of instruction should look like, C.M.Reigeluth outlines 8 core ideas: Learning-focused vs. sorting focused. Learner-centered vs. teacher-centered instruction. Learning by doing vs. teacher presenting. Attainment-based vs. time-based progress. Customized vs. standardized instruction. Criterion-referenced vs. norm-referenced testing. Collaborative vs. individual. Enjoyable\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/07\/2012-01-08-14-00-45_wm.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/07\/2012-01-08-14-00-45_wm.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/07\/2012-01-08-14-00-45_wm.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":7236,"url":"https:\/\/minkhollow.ca\/beckerblog\/2019\/03\/26\/what-traditional-classroom-grading-gets-wrong-and-how-to-fix-it\/","url_meta":{"origin":6159,"position":5},"title":"What Traditional Classroom Grading Gets Wrong &#8211; and how to fix it.","author":"Katrin Becker","date":"March 26, 2019","format":false,"excerpt":"Source: What Traditional Classroom Grading Gets Wrong (G)rading policies\u2014which appear to be an objective, fair, and accurate method to describe a student's academic performance\u2014often increase achievement gaps by infusing grades with teachers' implicit biases or by rewarding or punishing students based on their families' resources. Yup. They are also quite\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/6159","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/comments?post=6159"}],"version-history":[{"count":27,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/6159\/revisions"}],"predecessor-version":[{"id":7195,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/6159\/revisions\/7195"}],"wp:attachment":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/media?parent=6159"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/categories?post=6159"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/tags?post=6159"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}