Another example of the Randomness of Grades

Approximate Reading Time: 2 minutes

I Can’t Answer These Texas Standardized Test Questions About My Own Poems | The Huffington Post

Source: I Can’t Answer These Texas Standardized Test Questions About My Own Poems | The Huffington Post

This is an example of the actual author of the poem in the question being unable to figure out what the right answer is supposed to be.

Think about that for a moment. Don’t you think the author should be the final authority on things having to do with the piece they created?

I’m reminded of the time I gave my grade 10 English teacher a copy of my brother’s poem. I did not get along with most English teachers, but this one in particular disliked me.

I actually ended up failing English that year and had to re-take it in summer school. Now, THAT was the best class I ever had – in my entire school career – and as you can see, we were NOT a typical bunch of kids. One of the kids in the class was even a grade 9 student taking grade 10 English to get ahead (can you guess which one?).

 

 

 

Anyhow, this is the poem I gave my teacher:

Yea, though the bush burneth amongst the oak trees,
Let not the cookie crumble into the sea,
Thus making a mockery of the turnip,
Who ate the dwarf.

My teacher found deep and profound symbolism in this poem, which I found very amusing, seeing as how I knew for a fact that my brother had written it late one night in a drunken stupor. The poem was the drunken rambling of a teenage kid trying to make his similarly drunken friends laugh.

Perhaps the lesson to take away from these is:

Intelligence Is Believing Only Half Of What You Hear; Brilliance Is Knowing Which Half.

 

Unfortunately, when we are in school, we rarely have the option of questioning our teachers.

Be the first to like.

The Randomness of Grades: Are all ‘B’s the Same?

Approximate Reading Time: 2 minutes

Last time we looked at how different a score of 75% can be, depending on which 75% the student got right in an exam where the questions are ‘perfectly balanced’. BUT, what if the exam ISN’T perfectly balanced. What if, for example, we placed a heavier emphasis on the latter topics of the course? That’s a pretty common thing to do.

Student 1:

The 2 graphics depict Student 1’s results on both exams. This student got 75% of the questions right in EACH topic. When we translate that to an ‘unbalanced exam, it’s not too bad. It’s possible to imagine that the first result indicates a similar level of competence as the second result. Both are ‘B’s.

Student 2:

Student 2 blew it in topics 2 & 6. Maybe they missed some classes or maybe they misunderstood, or maybe they were totally confused and too scared or embarrassed to ask for help (trust me, it happens).
In this case, the total score is still close to the first score (155/200 vs 150/200), but in both samples it seems pretty clear that the level of competence is QUITE different from Student 1. Still, numerically, this is a ‘B’, which most would consider reasonably good.

Student 3:

In this last example, the student ALSO blew 2 topics pretty completely BUT now the difference in score is far greater.
THIS time, there’s a BIG difference. In both cases the student seemed to have completely missed or misunderstood Topics 7 & 8, but the score from the first test is 150/200 = 75% (B), and the score from the second test is 115/200 = 58% (D+).

Further, we have NO IDEA from this result why the student blew those two topics. Maybe, they got sick, or their child got sick, or their father died, or maybe, they just got bored and quit coming to class.

The point here is that the same student came away with a very different grade, and THIS student came away from the first exam with the same grade as the student who missed 75% of EVERY topic.

Can you really claim to have confidence that all of these ‘B’ students have the same level of competence?

Can you?

So much for confidence in exams – especially in multiple choice exams.

Note that I didn’t include a representation for what often happens with exam banks (especially those supplied by textbook writers) – when questions are chosen at random one often ends up with questions that ask the same thing, but really only differ in the way they are worded. A student who happens to know the answer to that question is at an advantage over the student who doesn’t.

That said, I do think it is possible to come up with reasonable exams that fairly assess a student’s mastery of the topics in a course. It’s just that I no longer think most instructors take the time to make sure their exams really do that.

Be the first to like.

Kua Tale, as Retold by Helga Ingeborg Vierich

Approximate Reading Time: 6 minutes

Helga Vierich listening to Kua Stories in Botswana.

Written by Helga Ingeborg Vierich (*)

The Kua told me one of their sacred stories – of the beginning times – in which the creator spirit saw a human being marrying an animal instead of killing them for food.

“..So God was born as a human baby, so impatient that he cut himself out of the belly his mother. He healed her and after a few days of growing, asked her where he could find his older brother, the one who had married recently. The mother told him to go to the place of the buffalo people. His brother had married a buffalo and they were expecting their first offspring.

So the little fellow set off, eating masses of wild plants to make himself bigger. He even ate all the termites in great mound, hollowing it out. He grew bigger and bigger, and started to eat animals he killed with a look. He finally took parts of the hide of dead giraffe to make a bag. The bag was useful, so he could carry dried meat and other food to be cooked on a fire he made every night, instead of eating as he went along. He decided he could not waste time wandering around eating all day.

When he got to the place of buffalo people he found his brother, and was introduced to the big placid pregnant cow who was his sister-in-law. They shared a meal of the foods God had brought within with him. His brother was surprised at the novelty of cooked food – especially meat, and was fascinated by the making of a fire. His wife, however, only nibbled at the cooked vegetation, and she refused the meat of her cousin the giraffe. God/boy told them that his mother had sent him to meet them, and ask them to return with him to her place for the birth of the youngster they were expecting.

So they all set off. One night they camped at the place of the hollow termite mound. God sent his brother to fetch firewood. While he was gone, God killed his sister-in-law, the buffalo wife. When the brother returned he was horrified, but God sedately set about making up a fire to cook the meat. Then he pointed to the dead cow and told his brother – “that is a buffalo. She is also your relative, as are all creatures, but the buffalo people are not for wives – they are meat for you.”

His brother wept. So God took pity and said: “She died quickly and knows nothing. and it was me that made this happen and not you. Now we will butcher her and make a meal of her, and then every part of her body must be used. Nothing is to be wasted. Her death serves this unhappy purpose: because of her I must teach you that your place in the world is not the same as that of the buffalo people. You must kill some buffalo when they offer themselves to you, and eat their bodies, just as does your brother the lion. But unlike the lion you will know what you do to them. It is in your heart and this is why I made you. And unless you love these other people, like the buffalo, kill mercifully, and keep your heart in the world with them faithfully, they will all vanish, and you will be nothing and have lost your place here.”

So the brother began, weeping, to cut open the body of his beloved. “Her life and breath are gone”, said God: “I have taken them, they will be with me forever. When you die you will both be with me, and we will rejoice in this forever.”

The man wept, and told the dead wife: “wait for me at the door of God’s house. I will join you there soon enough”. and God said “she will wait for you.” Weeping still, the brother took the body of his child from her womb, and it died in his hands, and the waters of the womb ran away across the sand. Mixed with his tears, the waters ran all the way back to the buffalo people. The buffalo smelled the waters and knew immediately their daughter and sister had been killed.

The man and his little brother/God were almost done drying all the meat – God showed him how to cut it in strips and hang in the branches of the tree by the termite mound. It was three nights after the death of the buffalo wife, that they heard the thunder of many angry buffalo. “They are coming. Quick”, said God, “in here” …and opened the termite mound and shoved his brother inside. “I will handle this”.

Soon the buffalo were running around and around the fire and the mound and were very angry as they smelled the death of their sister and daughter and her unborn child. God calmly sad to them “it was not her husband who has done it, but me”. “You?” snorted the buffalo bull, the father of the buffalo wife, “you are just a small boy!”

“No” said God. “I am YOUR father. It was foolish of you take a man as a husband for your daughter.”

He told the buffalo that it was mistake they made because the man “smelled as a grass eating buffalo” and this had created foolishness and error. He then told them that he would show them a smell that proved humans were not buffalo, but creatures to be wary of, creatures who ate meat.

God, having eaten masses of meat for three days, now began to fart. They were terrible poisonous smelling farts. (NOTE: at this point, the story teller began to make suitable sound effects, and all around the campfire the children began to make similar sounds. Their relieved laughter took the place of the shock on their faces that had earlier attended the death of the buffalo wife and her baby.)…

The farts were so dreadful that some of the buffalo began to faint. Others ran away into the night. Soon all was quiet. “You can come out now” said God. And the trembling man emerged and stared around at the buffalo lying as if dead all around the fire. “They are not dead, don’t worry, why would I kill them?” said God, “and they will soon wake up, so we must hurry”.. and the brothers packed up all the dried meat in a bag made of the skin of the buffalo wife, (because the giraffe bag as not big enough to hold it all).

They returned to the place where the mother lived, to eat joyously with her. In the next days, after explaining to the mother that she could use the bags to gather vegetables, and how to make fire for cooking both vegetables and meat. God/child produced a human women to be the wife of his brother. He showed him how to hunt in ways that did not frighten the animals and to incapacitate them with poisoned arrows that made them want to lie down and rest. He told his brother to kill animals as if each was his own brother or his own beloved… Then he walked away from the fire and at last God returned thankfully to his rightful place outside of this material world…”

I remember sitting up late in my tent with a flashlight to remember the whole story and translate it as well as I could. I forgot some things just now repeating it from memory. One thing I forgot was that the God-boy, as he walked along that first day, plucked hair from his head as it grew in rapidly. He took the black tufts and threw them into the air, and they became birds. He sneezed and with each drop new insects came into the world. He… well, never mind, suffice it to say that this little “intervention in nature” produced a few rivers and pans, dung beetles, and quiet a few soil creatures as well as scorpions and snakes. And it was all the fault of the first foolish humans. I never did get told who the father of the elder brother was; my story teller did not know – or wasn’t revealing this to the likes of me.

Megan Biesele recorded similar story in N’gamiland among the hunter-gatherers there. In her story, the wife was an elephant. There were poisonous farts and a termite mound too, I think. I suppose this to be a very ancient myth, with many longer or shorter versions, and perhaps other embellishments another places.

So, youngsters become hunters, and they learn the lesson of kinship with animals from stories told to them in childhood. They learn why they should never waste any of the precious creature they have killed. So we “weep for her, she was like a person” one hunter explained to me. And, believe it or not, during my fieldwork I actually witnessed the ritual of the kill: hunters would repeat the prayer of thanks, and bid the spirit of the animal to wait for them at the door of God’s house, where they would meet again and be together as kinfolk forever. It always made me tear up. It does now. Still.

Having killed for meat myself, I learned that compassion is all you have left to give. A merciful and quick death is the only hope of sanity during such an act, I think. I shudder to imagine human beings who can kill with indifference, or actually enjoy creating suffering. Those who find only amusement or self-gratification in the murder or torture of another being, even a domestic animal, are dangerous, the Kua told me, for they will kill a human just as easily.

(*) I don’t normally do guest posts, but this is such a wonderful story, I really wanted to share it more broadly than just on Facebook. Thank you, Helga for giving me permission to share your (and the Kua’s) story.

Be the first to like.

The Randomness of Grades: What is an ‘A’ Student?

Approximate Reading Time: < 1 minute

We all have a sense of what an ‘A’ Student is.

They are the ones who have earned our top marks, of course. But comparing our own personal ‘A’s with that of other faculty quickly becomes problematic.

All those complaints about ‘grade inflation’ are effectively us complaining that other people don’t have our high standards – as if OUR standards were, well standard.

But, are they?

Let’s have another look at my super simplified content <=> exam graphic.

Let’s suppose a student gets 75% of the questions right. That’s often considered to be a ‘B’.

This is what a ‘B’ looks like:

We’ll call this Student 1

If we think of each of the squares as representing a single MC question, then the patterns could tell us something if our questions were not randomized. Even without that, consider these other ‘B’s:

Student 2

Student 3

Student 4

Student 5

ALL of these students got exactly 75% of the questions right. Is it fair to say they ALL have the same level of competence?

I’d say no.

NOW…..

What if the exam ISN’T perfectly balanced. What if, for example, we placed a heavier emphasis on the latter topics of the course?

Come back tomorrow to see how things might change.

Be the first to like.

Grades: The Random Factor, Problem 2

Approximate Reading Time: 2 minutes

Problem 1, as outlined yesterday is that we really have no objective way of ensuring that our exams actually ‘cover’ the course content.

Now, what about exams from the student’s perspective?

I know, I know, tons of people have talked about issues with exams, but bear with me for a bit.

Think about the actual conditions for the exam:

  • Is everyone focused?
  • Who has another exam the same day?
  • What day of the week is it?
  • Early morning vs. late afternoon.
  • Is everyone healthy?
  • Regularly scheduled vs. deferred vs accommodated?

Can we assume all of these things affect students exactly the same way?
If not, then our exam is not the same for everyone, is it?

What about the kinds of questions we ask?

  • All Multiple Choice?
  • Subjective Questions?
  • Are questions JUST like what they’ve been practicing?
  • Graded on a curve?
  • Supplied by Textbook Author?
  • Thoroughly analyzed?
  • Your OWN assumptions/Biases (we ALL have them).

Here’s the real kicker:

Since there is no way we can reliably ‘cover’ all of the course content in our exams, it is clear that we will be choosing some SUBSET of the content to put in the test. That means we are testing students on a subset of the course content.


Choosing what goes into that subset is inherently SUBJECTIVE.


What if they study the wrong subset?

How many times have you heard students complain that they studied things that weren’t on the exam? Is that a problem?

*I* think it is. How does this accurately reflect a student’s mastery of the material?

Student X studies all the right stuff and aces the exam, while Student Y studies the wrong stuff and barely passes. Now, let’s further suppose that Student X did an adequate job on their assignments, and Student Y did exceptionally well on theirs. The final is worth 50% of the course, but the assignments are only worth 25% Let’s say they both got the same score on the midterm.

That means Student X got: 50% (final) + 15% (midterm) + 15% (assignments) = 80% = A-
Student Y got: 25% (final) + 15% (midterm) + 25% (Assignments) = 65% = C

Can you REALLY be sure that Student X is an ‘A Student’, and that Student Y is only a ‘C Student’ based on a single exam on a single day?

REALLY?

Next, we’ll look at whether an ‘A’ is always an ‘A’.

Be the first to like.

Grades: The Random Factor

Approximate Reading Time: 2 minutes

How much of your students’ grades in your courses is subject to random chance?

Is your first reaction a defensive one?

“Why, NONE, of COURSE!”

We all like to think we are assessing our students fairly, and that the grades they get are some sort of true, objective reflection of their mastery of our course content.

Think again.

Let’s start with exams. Of necessity, you can’t ask questions about every single thing you covered(*) in class, assignments, and readings. It’s just not practical.

(*It’s a handy fantasy to think that just because you ‘covered’ it, they ‘learned’ it.
It just ain’t so.)

Let’s imagine how it would look if you did create a comprehensive test.

This is a super simplified representation of the course content in my course. I have 8 topics, and each square represents the stuff I covered.

If I create a truly comprehensive exam, then I need to ask things on my exam that touch on every single Bit of content. Again, in my super simplified representation, each of my 8 topics has 25 Bits to it, and so my (let’s say Multiple Choice) exam should have 200 questions, each one addressing a different ‘Bit‘.

That’s completely untenable.

So your exam, of necessity, picks on a small percent of the course content. Even in a 3-hour exam (which, quite frankly, I think is both ridiculous and cruel), you can only touch on a small portion of the course content. Let’s guess you manage to ask questions that touch on a third of the course content.

Each ‘Bit‘ from the content appears in your exam. In the above image, each ‘Bit‘ that appears in the exam has been replaced with a white square in the original course content. Can you see how much of your content didn’t make it into the exam?

  • How do you decide which third to include in the exam and which 2/3 to leave out?
  • Does your textbook question bank have any influence on which questions you choose?
  • Do you let an automated test bank choose random questions for you?
  • How much do you trust it?
  • Have you ever even thought about that?
  • Do you know whether or not the author of your textbook did any statistical analysis on the reasonableness or predictive qualities of their questions? (It might surprise you to know that many authors simply make up questions without ever testing them on real students.)
  • Do you ask questions that you believe ‘cover’ multiple Bits?
  • Have you ever verified this?

On top of that, also keep this in mind when you look at the “Bits” of your course content. Your course consists of:

  • 39 Lecture hours
    • Is everything you say in class relevant course content?
    • If not, what isn’t, AND how do students know the difference?
  • 13 Tutorial Hours
    • Are you doing your own tutorials?
    • If not, what assumptions do you make about what’s going on in the tutorial?
    • How knowledgeable are your teaching assistants w.r.t. the course content?
    • How well do your teaching assistants match YOUR teaching style?
    • Are all your teaching assistants equally prepared and dedicated to helping your students master the course material?
    • Do you include material from the tutorial in your exam?
  • 450 pg. Text Book
    • How much of the textbook is relevant course content?
    • Is it only certain chapters, or is it certain parts of some (or all) chapters?
    • Do your students know EXACTLY what parts you consider to be course content and what parts are not?

That’s only PROBLEM 1

Tune in tomorrow for Problem 2.

Be the first to like.

What’s the difference between serious games, educational games, and game-based learning? Episode 2

Approximate Reading Time: < 1 minute

I recently posted a table explaining the differences between games, serious games, educational, games, GBL, etc.

Copyright © 2016 K.Becker, Mink Hollow Media, Ltd.; All Rights Reserved

 

I thought a concrete example might help to understand the distinctions.

Say we are using The Parable of the Polygons to help a high school class learn about diversity. This is a wonderful little game that tells a story of how harmless choices can make a harmful world.

The Game for Learning (a.k.a. Educational Game) refers to the actual object: The Parable of the Polygons.

All Games for Learning are ALSO Serious Games (Games for Learning are a subset of Serious Games), and in turn, all Serious Games are also Games.
Note: All digital games are ALSO simulations, but that’s a story for another post.

Copyright © 2018 K.Becker, Mink Hollow Media, Ltd.; All Rights Reserved

 

The Game-Based Pedagogy is the lesson that the teacher created which uses that game.

Game-Based Learning is what the students are doing when they use the game to try and meet the learning objectives.

These two are essentially 2 sides of the same coin:

Copyright © 2016 K.Becker, Mink Hollow Media, Ltd.; All Rights Reserved

 

Gamification is not part of any of these. It is NOT a game, although it may use ideas, mechanics, and other elements that are common in games. For example, the idea of ‘unlockable content’ might be included in a lesson such that students need to earn some pre-determined minimum score on a test, before they can move on the the next unit. This was one of the ideas employed in the highly successful SRA Reading Labs.

Copyright © SRA Reading Labs

Does that help?

3 people like this post.

What’s the difference between serious games, educational games, and game-based learning?

Approximate Reading Time: < 1 minute

This question keeps coming up. Here’s a handy table to help.

This is an updated version of the table I created back in 2015.

Note: This image is not to be posted anywhere or re-used without my written permission.

Citation: Becker, K. (Feb 3, 2018). What’s the difference between serious games, educational games, and game-based learning? Retrieved from https://wp.me/p4Hsb6-1KS

(You are free to disagree with me, just don’t present this table as yours).

Addendum 2021: A newer version of this chart has been peer-reviewed and published on academia.edu.

https://www.academia.edu/45044609/What_s_the_difference_between_gamification_serious_games_educational_games_and_game_based_learning
Citation: Becker, K. (2021). What’s the difference between gamifcation, serious games, educational games,and game-based learning? Academia Letters, Article 209. https://doi.org/10.20935/AL209

4 people like this post.