Gamification 101[20]: Bona Fide Criterion-Referenced Assessment

Approximate Reading Time: 4 minutes

The continuing saga of my current iteration of a gamified course.

Hang on to your hats, this one is radical…..

First, some context.

Theory is nothing without practice. I learn new things with each iteration of the course as I teach it. One of the things I’ve been giving a lot of thought to is the notion of criterion-referenced (CRA) or performance-based assessment (PBA). For me criterion-referenced assessment sounds a little broader in scope. Something that is performance based implies that students must actually do something whereas criterion-referenced includes more of those things we would normally categorize as higher-order thinking (i.e. stuff from the 5th and 6th ‘petals’ of Bloom’s Rose below).

Blooms rose
Here’s the smack on the side of your head.

If we really intend to assess people according to described criteria and if we really want to let go of the simpler time-referenced assessment, then we should be willing to accept ALL relevant evidence created by our students, right?

I know some of you are simply thinking, “Well, sure.” but hold on. Think about this. Are you really prepared to accept bona fide evidence, without regard to when, where, or why it was produced? 

I’ve given this a lot of thought over the last few years and my answer is YES. If we say that showing evidence of competence is the most important thing, then so long as we are confident that the work done is that of the student submitting it, then why should it matter when that work was done? Or where? Or even why?

Now let’s look at the ramifications of this as they constitute a sea change in traditional educational assessment.

What if it is work the student did in the past? What if it is work that was produced for another course? Should we accept it?
….. I say yes.

What if the student is repeating a course? Should we accept work they submitted to the course from the last time they took it?
….. I still say yes.

If we decide that the thing that really counts is that the student be able to show evidence that they have mastered the material we have assigned to them, it shouldn’t matter if they mastered it elsewhere. Why are we so often stuck on this notion that students must produce “fresh evidence”? Does mastery expire?

DBSK11Mostly not, but I’m willing to bet that the thought of accepting work produced in the past would make many educators squirm. The idea of accepting work in one course that has also been submitted for assessment in another course is a time-honored taboo in formal education. But why? Think carefully now. Our students are not submitting papers for publication to some learned periodical. Unless they are PhD candidates and as long as the work submitted is genuinely their own, then there really is no requirement that the work they submit be new.

There are exceptions to the idea that we should accept prior work, of course. Anything that includes formal regulations, laws, or other guidelines that are subject to renewal can indeed have expiry dates. Work that incorporates these must be current and so may legitimately be required to be “fresh”. Similarly things like peer review, and work that connects with current events or other activities that must occur in the present (such as discussions) must also be produced at the time they are required. There is a lot of work that does not fall into one of these categories, however.

What are the benefits of accepting prior work?

For the instructor, none, really. It may actually result in marginally more work because we may now have to assess work that is not the ‘cookie-cutter’ solution we are used to marking. To these instructors I have something to say. If you don’t want to do the work of your profession – then go do something else.

EDU5For the student the benefits are potentially great. Additionally, the potential effect on student engagement and motivation is considerable. If they know ahead of time that a particular piece of work could be used in more than one course, then, assuming they actually care about their grades, they could be motivated to work harder on that assignment. They could even end up learning more than they would by doing a more superficial job on two similar assignments. If we are honest about the realities of student life, then we should also recognize that students occasionally have to make choices about which course work they are going to concentrate on and which ones they are going to try and coast on, or even ignore. If they already have evidence that they have mastered something we need them to master, then what’s the benefit of making them do it again? Aren’t we just making them jump additional hoops? Why not let them submit something they’ve already done and let them spend more time on the things that challenge and/or interest them? Isn’t that what we keep saying we care about?

Here’s the other big plus for students. If we decouple their required work from our calendar, then that gives them choices. If they already have evidence of competence in something we are requiring, then they should be able to submit it right away. They could then get that work off their to-do list and go on to spend time on something that they need to spend more time on. Why should they have to wait until we have ‘released’ our assignment that goes with that unit? And, further, why should they have to wait for feedback until we get around to marking that batch? What is our justification for making them wait, aside from our own convenience, that is?

OK, so what about cheating, you ask? Well, that is always a risk, especially when students have very little contact with their instructors, as is the case at so many large institutions. But cheating is always a risk, and let’s be honest here. If you are teaching a first year class, chances are it’s going to be nearly impossible for you to come up with an assignment that hasn’t been done before. There are going to be some students who will cheat no matter WHAT you do. Wouldn’t it be better to free up your time and energy by letting up on largely pointless anti-cheating efforts so you could spend that time actually helping students? My experience over the years has been that the more I trust my students, the more trust-worthy they actually become. Most people will live up (or down) to meet your expectations, so why not set them high rather than low?


gamificationIf you are interested in following my course journal, watch for the “Gamification 101” heading.

Also, for more information on gamification, check out my website here.

Be the first to like.


Comments

Gamification 101[20]: Bona Fide Criterion-Referenced Assessment — 2 Comments

  1. “Does mastery expire?” Unfortunately, yes, it does. I did very well in a lot of math courses when I was an undergrad and grad student, that I now would be incompetent at, not having practiced those branches of math for 30 or 40 years. I can’t even remember what the Chinese Remainder Theorem is any more, much less be able to apply it to number theory puzzles. I had two years of Russian and 2 years of Japanese in college, and I can’t remember more than a handful of tourist phrasebook phrases in either language now (I retained more of my high-school German, but it is still way down from the days when I could read Goethe’s Faust).

    Student behavior is very tuned by cram-and-forget courses, to the point where students often can’t do something that they’ve “displayed mastery” on two weeks earlier, much less 2 years earlier.

    While I like the idea of giving students credit for whatever evidence of mastery they can show, I’m much more convinced by recent evidence that they still have the mastery, than by evidence that once upon a time they knew what they were doing. My assessments are usually based on programs, research papers, or circuit designs. The writing and designing skills I’m interested in are ones that get better with practice—I don’t care if students once wrote a decent paper or good program, but I do care that they continue to practice writing and designing, and continue to improve their skills.

    • Fair point. It isn’t possible to make any kind of blanket statement about how long mastery should or could last – it really needs to be taken on a case by case basis. On the other hand, if the mastery they demonstrate only lasts a few weeks, then I’d say we’re doing something wrong.
      I was actually thinking about the things a student did in the last semester or in the last year.

      Academia has a tendency to love rules just a little too much. No matter how much we may wish it to be otherwise, education is a subjective process. No matter what we do, 2 people who end up with the same marks can not be assumed to have the same skills or know the same stuff. Setting more rules and “standards” hasn’t really fixed that.

Leave a Reply