{"id":4257,"date":"2014-08-20T11:34:43","date_gmt":"2014-08-20T17:34:43","guid":{"rendered":"http:\/\/minkhollow.ca\/beckerblog\/?p=4257"},"modified":"2019-04-26T14:08:24","modified_gmt":"2019-04-26T20:08:24","slug":"students-drop-a-bomb-at-the-gafesummit-vancouver-we-dont-like-rubrics-a-stick-in-the-sand","status":"publish","type":"post","link":"https:\/\/minkhollow.ca\/beckerblog\/2014\/08\/20\/students-drop-a-bomb-at-the-gafesummit-vancouver-we-dont-like-rubrics-a-stick-in-the-sand\/","title":{"rendered":"Students drop a bomb at the #gafesummit Vancouver: \u201cWe don\u2019t like rubrics.\u201d | A Stick in the Sand"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Approximate Reading Time: <\/span> <span class=\"rt-time\"> 3<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span><p>Do students really like rubrics? Are YOU using them right?<\/p>\n<p><a href=\"http:\/\/www.ovenell-carter.com\/education\/the-bombshell-at-the-gafesummit-vancouver-we-dont-like-rubrics-students-say\/\"><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-4261\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2014\/08\/CH043082.png\" alt=\"CH043082\" width=\"167\" height=\"320\" srcset=\"https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2014\/08\/CH043082.png 167w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2014\/08\/CH043082-156x300.png 156w\" sizes=\"auto, (max-width: 167px) 100vw, 167px\" \/>Students drop a bomb at the #gafesummit Vancouver: \u201cWe don\u2019t like rubrics.\u201d | A Stick in the Sand<\/a>.<\/p>\n<p>For my part I quite like rubrics, but I also think many, if not most teachers create unrealistic rubrics.<\/p>\n<p><a title=\"Authentic Education\" href=\"http:\/\/www.authenticeducation.org\/whoweare\/grant.lasso\" target=\"_blank\" rel=\"noopener noreferrer\">Grant Wiggins<\/a> (of <a href=\"http:\/\/www.authenticeducation.org\/ubd\/ubd.lasso\" target=\"_blank\" rel=\"noopener noreferrer\">Understanding by Design<\/a> fame, among other things) has written <a title=\"Granted, And (blog)\" href=\"http:\/\/grantwiggins.wordpress.com\/2013\/01\/17\/intelligent-vs-thoughtless-use-of-rubrics-and-models-part-1\/\" target=\"_blank\" rel=\"noopener noreferrer\">some must-read posts<\/a> about rubrics.<\/p>\n<blockquote><p><span style=\"color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; line-height: 23.399999618530273px;\">Consider how a valid rubric is born. It summarizes what a range of concrete works looks like as reflections of a complex performance goal. Note two key words:\u00a0<\/span><i style=\"border-width: 0px; font-size: 13px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline; font-style: italic; color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; line-height: 23.399999618530273px; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;\">complex<\/i><span style=\"color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; line-height: 23.399999618530273px;\">\u00a0and\u00a0<\/span><i style=\"border-width: 0px; font-size: 13px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline; font-style: italic; color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; line-height: 23.399999618530273px; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;\">summarizes<\/i><span style=\"color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; line-height: 23.399999618530273px;\">. All complex performance evaluation requires a\u00a0<\/span><span style=\"border: 0px; font-size: 13px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline; color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; font-style: normal; line-height: 23.399999618530273px; text-decoration: underline; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;\">judgment<\/span><span style=\"color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; line-height: 23.399999618530273px;\">\u00a0of quality in terms of one or more criteria, whether we are considering essays, diving, or wine. The rubric is a\u00a0<\/span><span style=\"border: 0px; font-size: 13px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline; color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; font-style: normal; line-height: 23.399999618530273px; text-decoration: underline; background-image: initial; background-attachment: initial; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;\">summary<\/span><span style=\"color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; line-height: 23.399999618530273px;\">\u00a0that generalizes from lots and lots of samples (sometimes called models, exemplars, or anchors) across the range of quality, in response to a performance demand. The rubric thus serves as a quick reminder of what all the specific samples of work look like across a range of quality.<\/span><\/p><\/blockquote>\n<p>The place where I see many (if not most) rubrics falling down is that they don&#8217;t actually summarize lots and lots of samples.<\/p>\n<p>Now, I can create a pretty good rubric for a programming assignment, but that&#8217;s because\u00a0<em><strong>I&#8217;ve seen 1000&#8217;s of programming<\/strong><strong> assignments<\/strong><\/em>. In order to be able to describe the various levels of achievement in a piece of work,\u00a0<em><strong>you have to have seen a very large number of samples of that particular<\/strong><\/em><strong> work.<\/strong><\/p>\n<p>I can&#8217;t speak for K-12, but I can say that education faculties LOVE rubrics, and when I teach a course in an education faculty, I am usually\u00a0to produce rubrics for every single &#8220;learning task&#8221; I create &#8211;\u00a0<em><strong>even if it&#8217;s a brand new<\/strong><\/em><strong><em> task<\/em>.<\/strong> To me this reveals a lack of understanding of what rubrics are and how they should be used. How can I possibly know what exemplary looks like if I have never seen any examples? Well, the answer is I can&#8217;t. What I\u00a0<em>can<\/em> do (which is what most teachers do when they are creating rubrics) is to invent a wish list of things we want to see. That&#8217;s not a rubric. That&#8217;s a wish list.<\/p>\n<p>Here&#8217;s an example of what I do now:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4259\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2014\/08\/COMP1103-Player-Stats-2014-2.10.png\" alt=\"COMP1103 Player Stats 2014 2.10\" width=\"779\" height=\"803\" srcset=\"https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2014\/08\/COMP1103-Player-Stats-2014-2.10.png 779w, https:\/\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2014\/08\/COMP1103-Player-Stats-2014-2.10-291x300.png 291w\" sizes=\"auto, (max-width: 779px) 100vw, 779px\" \/><\/p>\n<p>While this has many of the same ideas as are intended with a rubric, it allows for a great deal more flexibility &#8211; especially since I add comments on each of the categories.<\/p>\n<hr \/>\n<p>Update: April 26, 2019<\/p>\n<p>The original webpage that prompted this post appears to be gone, so i have taken the liberty of repeating it here (thanks to the <a href=\"https:\/\/web.archive.org\/web\/20140703014356\/http:\/\/www.ovenell-carter.com\/education\/the-bombshell-at-the-gafesummit-vancouver-we-dont-like-rubrics-students-say\/\" target=\"_blank\" rel=\"noopener noreferrer\">WayBack Machine<\/a>)<\/p>\n<h1 class=\"entry-title\">Students drop a bomb at the #gafesummit Vancouver: \u201cWe don\u2019t like rubrics.\u201d<\/h1>\n<div class=\"post-info\">On May 8, 2014 by\u00a0<a href=\"https:\/\/web.archive.org\/web\/20140703014356\/http:\/\/www.ovenell-carter.com\/author\/Braddo\/\">Braddo<\/a> With <strong>11<\/strong>\u00a0Comments &#8211;\u00a0<a title=\"View all posts in Education\" href=\"https:\/\/web.archive.org\/web\/20140703014356\/http:\/\/www.ovenell-carter.com\/category\/education\/\" rel=\"category tag\">Education<\/a><\/div>\n<div class=\"content column-1\">\n<p><em>Update: Here\u2019s a\u00a0<a title=\"Storify\" href=\"https:\/\/web.archive.org\/web\/20140703014356\/https:\/\/storify.com\/braddo\/students-hate-rubrics\">Storify story<\/a>\u00a0with as much of the conversation on Twitter as I can capture:<\/em><\/p>\n<div id=\"attachment_1474\" class=\"wp-caption alignnone\">\n<p><a href=\"https:\/\/web.archive.org\/web\/20140703014356\/http:\/\/www.ovenell-carter.com\/wp-content\/uploads\/2014\/05\/rubrics.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-1474 size-medium\" src=\"https:\/\/web.archive.org\/web\/20140703014356im_\/http:\/\/www.ovenell-carter.com\/wp-content\/uploads\/2014\/05\/rubrics-300x300.jpg\" alt=\"rubrics\" width=\"300\" height=\"300\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">student\u2019s note from the panel discussion<\/p>\n<\/div>\n<p>This year we assembled\u00a0a panel of students at the\u00a0<a href=\"https:\/\/web.archive.org\/web\/20140703014356\/http:\/\/bc.gafesummit.com\/2014\">Google Apps for Education (GAFE) Vancouver Summit<\/a>\u00a0we hosted at Mulgrave School. The thinking was, it\u2019s about time we talked to our customers, so to speak.<\/p>\n<p>We had 5 high school students students from three local schools sitting round the table blue-sky-ing the future school. I\u2019ll be posting more of their thoughts later but I thought I\u2019d start with this bomb: the students hate rubrics!<\/p>\n<p>Rubrics are the bread and butter of assessment these days. The IB Diploma program, for example, depends on them. One student did say that on the upside, rubrics provide some security against teacher bias (real or perceived.)\u00a0But, all of the kids said, rubrics\u00a0feel too standardized: \u201cWe want something more personal than a number or letter that refers to some canned comment.\u201d<\/p>\n<p>Good rubrics are built by students and teachers, collaboratively, so this comment from the kids might mean we\u2019re not doing a good enough job consulting the major stakeholders in the assessment. My sense was, however, that this was not the problem and that students were touching on something much deeper. As Yong Zhao said in the summit keynote, so long as outcomes\u2013whether employable skills or university entrance requirements\u2013are prescribed by an external body, schooling becomes an exercise in squeezing all students through the \u201csausage-making machine.\u201d It\u2019s an impersonal activity, by definition, because in this structure we are not cultivating personal talents but rather bending and shaping people to meet some externally determined standard.<\/p>\n<p><a href=\"https:\/\/web.archive.org\/web\/20140703014356\/http:\/\/www.ovenell-carter.com\/wp-content\/uploads\/2014\/05\/zhao.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1476 size-large\" src=\"https:\/\/web.archive.org\/web\/20140703014356im_\/http:\/\/www.ovenell-carter.com\/wp-content\/uploads\/2014\/05\/zhao-1024x768.jpg\" alt=\"Zhao keynote\" width=\"904\" height=\"678\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>For an outstanding example of what cultivating talent looks like, watch this video of Benjamin Zander teaching a 15-year old student to play cello. It remains the best teaching I\u2019ve \u00a0seen. \u201cYou can give an \u201cA\u201d to anybody,\u201d says Zander. Listen to him explain what great\u00a0teaching and assessment look like at about 5:45 on.<\/p>\n<div class=\"fluid-width-video-wrapper\"><iframe id=\"fitvid956673\" title=\"Benjamin Zander - PopTech 2008\" src=\"https:\/\/web.archive.org\/web\/20140703014356if_\/http:\/\/player.vimeo.com\/video\/18625943\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\" data-mce-fragment=\"1\"><\/iframe><\/div>\n<p>I recall reading somewhere of a study that suggested that rubrics did not actual provide any more objectivity or consistency that good teacher judgement:\u00a0the same student essays were given to two different groups of teachers with one group using a rubric and the second, their good judgement. The study reported no statistically significant difference in the scoring. If anyone can get their hands on this or similar research I\u2019d be grateful.<\/p>\n<p>Also, five students are\u00a0admittedly a ridiculous small sample size. I think I need to at least send out a quick poll to increase the numbers and draw on a more representative sample.<\/p>\n<\/div>\n<div class='wp_likes' id='wp_likes_post-4257'><a class='like' href=\"javascript:wp_likes.like(4257);\" title='' ><img decoding=\"async\" src=\"http:\/\/minkhollow.ca\/beckerblog\/wp-content\/plugins\/wp-likes\/images\/like.png\" alt='' border='0'\/><\/a><span class='text'>Be the first to like.<\/span><\/p>\n<div class='like' ><a href=\"javascript:wp_likes.like(4257);\">Like<\/a><\/div>\n<div class='unlike' ><a href=\"javascript:wp_likes.unlike(4257);\">Unlike<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Approximate Reading Time: <\/span> <span class=\"rt-time\"> 3<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>Do students really like rubrics? Are YOU using them right? Students drop a bomb at the #gafesummit Vancouver: \u201cWe don\u2019t like rubrics.\u201d | A Stick in the Sand. For my part I quite like rubrics, but I also think many, &hellip; <a class=\"more-link\" href=\"https:\/\/minkhollow.ca\/beckerblog\/2014\/08\/20\/students-drop-a-bomb-at-the-gafesummit-vancouver-we-dont-like-rubrics-a-stick-in-the-sand\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[9,14,24],"tags":[388,41,346,393],"class_list":["post-4257","post","type-post","status-publish","format-standard","hentry","category-educational-technology","category-general","category-teaching-learning","tag-academia","tag-education","tag-rubrics","tag-teaching-learning"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4Hsb6-16F","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":3296,"url":"https:\/\/minkhollow.ca\/beckerblog\/2013\/03\/13\/intelligent-vs-thoughtless-use-of-rubrics-and-models-part-1-granted-but\/","url_meta":{"origin":4257,"position":0},"title":"Intelligent vs. thoughtless use of rubrics and models (Part 1) | Granted, but&#8230;","author":"Katrin Becker","date":"March 13, 2013","format":false,"excerpt":"YES YES YES!!! I can create a pretty good rubric for any given introductory-intermediate programming assignment. Know why? Because I've seen literally thousands of student solutions. Can I do that for a brand new style of assignment I've never done before? Heck no! Crappy Rubrics is one of my many\u2026","rel":"","context":"In &quot;Educational Technology&quot;","block_context":{"text":"Educational Technology","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/educational-technology\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2879,"url":"https:\/\/minkhollow.ca\/beckerblog\/2012\/09\/23\/where-ive-been-online-to-sept-22\/","url_meta":{"origin":4257,"position":1},"title":"Where I&#8217;ve Been Online (to Sept. 22)","author":"Katrin Becker","date":"September 23, 2012","format":false,"excerpt":"Educational Electronic Games Rubric Educational electronic games can be utilized in a variety of teaching situations. This rubric will help you evaluate your electronic game in an educational setting. If a criteria rating falls between two levels of achievement, you may score it with a point value between the two\u2026","rel":"","context":"In &quot;Educational Technology&quot;","block_context":{"text":"Educational Technology","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/educational-technology\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":7236,"url":"https:\/\/minkhollow.ca\/beckerblog\/2019\/03\/26\/what-traditional-classroom-grading-gets-wrong-and-how-to-fix-it\/","url_meta":{"origin":4257,"position":2},"title":"What Traditional Classroom Grading Gets Wrong &#8211; and how to fix it.","author":"Katrin Becker","date":"March 26, 2019","format":false,"excerpt":"Source: What Traditional Classroom Grading Gets Wrong (G)rading policies\u2014which appear to be an objective, fair, and accurate method to describe a student's academic performance\u2014often increase achievement gaps by infusing grades with teachers' implicit biases or by rewarding or punishing students based on their families' resources. Yup. They are also quite\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5738,"url":"https:\/\/minkhollow.ca\/beckerblog\/2015\/09\/24\/gamification-10114-5-ways-to-make-marking-easier\/","url_meta":{"origin":4257,"position":3},"title":"Gamification 101[14]: 5 Ways to Make Marking Easier&#8230;","author":"Katrin Becker","date":"September 24, 2015","format":false,"excerpt":"This is Part 14 in my continuing saga of my current iteration of a gamified course. A key aspect of\u00a0my gamified course design is that there be fast turn-around of assessment. It is not uncommon in more traditional courses for instructors to take one to two weeks to return assignments\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"comp 1103 2014 quest log","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/09\/comp-1103-2014-quest-log-235x300.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":6850,"url":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/24\/worth-sharing-the-misguided-drive-to-measure-learning-outcomes-nyt\/","url_meta":{"origin":4257,"position":4},"title":"Worth Sharing: The Misguided Drive to Measure \u2018Learning Outcomes\u2019 NYT","author":"Katrin Becker","date":"February 24, 2018","format":false,"excerpt":"Colleges should stop trivializing the transmission of knowledge. Source: Opinion | The Misguided Drive to Measure \u2018Learning Outcomes\u2019 I love having objective, clear-cut descriptions of what my students are supposed to learn, and nice, objective, clear-cut assessments that I can use to measure them. Unfortunately,\u00a0that's never going to happen. These\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5621,"url":"https:\/\/minkhollow.ca\/beckerblog\/2015\/10\/04\/gamification-10117-what-is-needed-in-a-gamified-grading-application\/","url_meta":{"origin":4257,"position":5},"title":"Gamification 101[17]: What Does a Gamified Grading Application Need to Have?","author":"Katrin Becker","date":"October 4, 2015","format":false,"excerpt":"This is Part 17 in my continuing saga of my current iteration of a gamified course. As I've said before, current course management systems don't have the flexibility required to accommodate a gamified design such as mine (here and here). Currently I, using Google Sheets, which works pretty well, but\u2026","rel":"","context":"In &quot;Academia&quot;","block_context":{"text":"Academia","link":"https:\/\/minkhollow.ca\/beckerblog\/category\/academia\/"},"img":{"alt_text":"snap02080","src":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/09\/snap02080-1024x511.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/09\/snap02080-1024x511.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/minkhollow.ca\/beckerblog\/wp-content\/uploads\/2015\/09\/snap02080-1024x511.png?resize=525%2C300 1.5x"},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/4257","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/comments?post=4257"}],"version-history":[{"count":7,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/4257\/revisions"}],"predecessor-version":[{"id":7274,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/posts\/4257\/revisions\/7274"}],"wp:attachment":[{"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/media?parent=4257"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/categories?post=4257"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/minkhollow.ca\/beckerblog\/wp-json\/wp\/v2\/tags?post=4257"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}