Formative vs Summative vs Confirmative vs Predictive Evaluation

Approximate Reading Time: < 1 minute
0101-evaluation

© K.Becker 2016

Confused about the difference between summative and formative evaluation? Did you know there are two other kinds of assessment that we rarely talk about in education? This image from my upcoming book shows where they typically apply in the development lifecycle.

Robert Stakes analogy for formative vs summative evaluation is one of my favorites: “When the cook tastes the soup, that’s formative. When the guests taste the soup, that’s summative“.

  • Formative evaluation tends to focus on the process and normally takes place during development and  before the intervention has been deployed.
  • Summative evaluation normally takes place after deployment and focuses on its overall effects.
  • Confirmative evaluation is normally performed after the summative evaluation has been complete for some time, and its purpose is to confirm that the instruction is still effective weeks, months, and even years later. This is rarely done in formal education, partly (but not entirely) for practical reasons.
  • Predictive evaluation takes place before the instruction is even developed. This kind of evaluation can help us decide whether or not to proceed with a project before we have devoted too much time or too many resources to back out. It can also give us information we need to know to be able to proceed with confidence.
4 people like this post.

Who has a good anonymous blog?

Approximate Reading Time: < 1 minute

2009-03-22_13-22-25-001_wmA new semester means a new set of students doing my “internet stalking” quest. This an adaptation of an assignment created by Robert Runte (ULeth), where students choose an anonymous blog and try to find out as much as they can about the person/people behind it. This is an exercise about privacy. The point of the exercise is to discover just how easy it usually is to find out information about people even when they don’t share it freely. I am looking for suggestions: if anyone has or knows of a blog that does not identify the blogger(s) that they think might be suitable for this quest, please share!

2.02 Achievement Quest - Profiling Mission 1_Page_1

2.02 Achievement Quest - Profiling Mission 1_Page_2

Be the first to like.

There’s more to Gamification than Narratives and Competition

Approximate Reading Time: 2 minutes

Survival of the Fittest (Gamification)

by http://simplifyingradicals2.blogspot.ca/
Tuesday, May 10, 2016

I start class by explaining to the students that they have been cast on a reality TV show called “Survival of the Fittest”.  This is the letter I give them:  (all materials can be found in this shared document).

Dear Students,

In an effort to supplement my teaching salary, I have volunteered all of you to be part of a new reality show called Survival of the Fittest.  The producers of the show and I are very excited to begin this endeavor.
Survival of the Fittest is a game where you are a member of a team and need to only stay alive for the duration of the show in order to win.  There will only be one winning team in the game.    Winners will receive bragging rights along with extensive knowlegde of     ___________________.

In order to survive, you will need supplies such as water, food, weapons, and a bit of good luck.  Other teams may try to sabotage you, so be alert.  Anything could happen in this game as the “game-creators” are attempting to put on a good show and boost television ratings.

 To begin, you will be flown to a remote location without any communication devices available to you.  You and your team will have the opportunity to earn supplies to help keep you alive.  Guard these supplies with your life, as you may need them to survive.

 I wish you luck and may the best team win!
Sincerely,

 

I’m using a gamified design in my classes that’s working extremely well, but I don’t layer a narrative on top of my course.

I have two concerns with this design:
1. There is a risk that the narrative will overshadow the actual learning objectives of the course of unit. The ‘content’ is depreciated to become little more than the thing you need to do to win the game. Good gamification should do the opposite: it should focus attention on the content. I don’t have a problem with narratives per se, but they need to be specifically tailored to the content of the unit. A narrative into which we can “drop” course content is really not much different from a game wrapper. If there are ANY people in your class who don’t like the narrative, then it becomes a barrier to learning. If all you are doing is switching one group of disengaged learners for a different group of disengaged learners, you are not making progress.

2. Only SOME students like competing against each other. They tend to be the more extroverted students. Introverts are often put off by competition. I think punitive measures (like losing health) interfere with learning, unless the ‘failure’ is directly related to the thing they are trying to learn.

Be the first to like.

Teaching Out Loud vs Teaching Loud vs Teach-Aloud

Approximate Reading Time: 3 minutes
I'm afraid I can't remember who should get credit for this photo (it's not me). This is the partly why I came to think about education the way I do.

I’m afraid I can’t remember who should get credit for this photo (it’s not me). This is partly why I came to think about education the way I do. This was what my 1st year experience looked like.

Every student in every class deserves an answer to these 2 questions:

  1. Why am I doing this?
  2. What is it good for?

If you can’t answer those questions clearly and honestly, then you really need to take some time to think.

Warning: Some may find this inflammatory.

That’s not to say that everything you teach needs to be immediately and practically applicable, but if you can’t connect the dots for your students so they can see what’s valuable about what you are asking them to learn, then you haven’t given enough thought to your teaching. If you don’t give even that much thought to your teaching, then you shouldn’t be teaching.

I’m not the first person to coin the term “Teaching out Loud“, but I interpret it somewhat differently. The approach advocated by many of those who use the term seems to me to be closer to “Teaching Boldly” (or “Teaching Loud”) than it is to Teaching out Loud.

It’s time for educators to teach out loud. To have the courage to teach the way students learn. To have the courage to try new things. To have the courage to fight for their students.

Teaching out Loud

First let me say, there’s nothing wrong with this approach. I actually think it’s a great idea and I do this myself. BUT, this is not what comes to mind (at least for me) when I hear the phrase.

What comes to mind for me when I hear “Teaching Out Loud” is more in line with the concepts associated with “Think Aloud” idea in psychological and educational research. The basic idea is that the ‘subject’ says what they are thinking about as they complete a task. The goal is to learn about the thought processes the subject is using. Given that, “Teaching Out Aloud” should be about the teacher explaining their reasoning and thought processes while teaching.

My view of “Teaching Out Loud” (or: Becker’s Teach Aloud Protocol) involves letting my students know why I am asking them to do what I am asking them to do AND telling them what I want them to get out of it. This goes beyond the typical learning outcomes listed in course syllabi (most students never read those anyways). It requires me to be deliberate about every single thing I do in class. I have to be very clear on why I am asking them to do this, and how it will benefit them. I also have to be prepared to explain to my students why I am doing it the way I am. I am not only justifying the instruction – I am also justifying my pedagogy. In this way, I am treating them as fellow adults (I should be able to do this in university) by inviting them into my circle.

Of course, this too can easily become just another academic exercise in intellectual masturbation, so it requires an additional step: I must ask my students to critique what I’m doing, AND I must be willing to consider those critiques WITHOUT penalizing my students in any way. I need to ask:

How did it go?

How can I improve this so it works better? Perhaps, it didn’t work at all and I should completely re-think the whole exercise.

A different approach that is also part of my Think Aloud Protocol has had a surprisingly profound impact on my whole philosophy of education. It’s this:

What if I don’t?

In other words, what is lost if I DON’T do this? What is lost if I DON’T set a hard deadline? What if I drop the typical rule that says students can’t submit the same work in two (or more) different classes for credit? What do I or my students lose if don’t have a high-stakes exam?

This has changed so much for me that I have started applying it to everything I have ‘usually’ done in my classes. Occasionally, I decide that this strategy or exercise is indeed very useful, and I keep it. More often than not, I discover that the reasons for doing something, or for doing them a particular way are very thin, and I am able to let it go and replace it with something more thoughtful and deliberate.

The end result?

Check out some of my gamification posts.

1 person likes this post.

Worth Sharing: Teaching by Numbers — Learning {Re}imagined — Medium

Approximate Reading Time: < 1 minute

A curious thing has been happening in UK education and it’s a trend that’s occurring globally as the craft of teaching is being transformed into a “science”. The Economist recently published an article titled “How to make a good teacher” that proclaimed: “The premise that teaching ability is something you either have or don’t is mistaken. A new breed of teacher-trainers is founding a rigorous science of pedagogy. The aim is to make ordinary teachers great, just as sports coaches help athletes of all abiliti

Source: Teaching by Numbers — Learning {Re}imagined — Medium

Be the first to like.

Worth Sharing: New American Academy of Pediatrics Policy Statement on “Virtual Violence” is, Basically, Nuts: Why Parents and Policy Makers Can Ignore It

Approximate Reading Time: < 1 minute

The American Academy of Pediatrics (AAP) has a long history of releasing policy statements on media which are both wildly alarmist and grossly inaccurate. These include their controversial claims about “Facebook Depression” in 2011, their problematic and unrealistic recommendations on screen time, and past biased statements regarding media violence, which once infamously inflated the number of studies by a factor of about ten (mistakenly claiming 3500 studies existed when, in fact, there were a couple hundred).  In July, 2016, the AAP have added to their unscientific litany a new policy statement that reads as strangely defensive and frustrated (indeed, ironically enough, even “aggressive”) but distorts the research evidence as more consistent and worrisome than it currently is.

Source: New American Academy of Pediatrics Policy Statement on “Virtual Violence” is, Basically, Nuts: Why Parents and Policy Makers Can Ignore It

Be the first to like.

Worth Sharing: Donald Clark Plan B: Why the tablets in schools debacle is over

Approximate Reading Time: < 1 minute

iPadWhen will they ever learn? Had I said that iPads in the the schools were not the answer four years ago at the height of the love-affair with the iPad, I’d have been thought a crank.

Wait, I DID.

More than once.

(For more, just try the iPad or iStuff tag on my blog).

My former supervisor thought I was out of touch. People I met a conferences thought I was weird. Well, apparently, I was right (not for the first time). Gee, spending 35+ years in tech and teaching novices about computers and programming for most of that time actually taught me something. How about that.

After the California debacle, schools in five states (Virginia, California, Maine, Texas and North Carolina) are starting to swap out tablets for the laptops they should have purchased in the first place. It started with a survey in Maine, where teachers and students expressed a preference for laptops over tablets. (That’s a novel approach asking the users what they want, as opposed to shoving inappropriate tablets down their throats.)

To be exact, 88.5% of teachers and 74% of grade 7-12 students wanted laptops, not iPads. The observations were clear, that while iPads may be appropriate for young children, they are not suitable for older children who need to acquire writing and other more sophisticated skills using tools that don’t work on iPads,

“shortcomings for older students”

“provide no educational function in the classroom”

“students use them as toys”

“word processing near to impossible … I applaud this change.”

“largely students’ gaming devices”

“a disaster”“WE NEED LAPTOPS!!!” a student said, three times.

Apple has caved in and swapped the tablets for reduced price MacBook Air laptops.

Source: Donald Clark Plan B: Why the tablets in schools debacle is over

Be the first to like.

Worth Sharing: Teaching Math to People Who Think They Hate It

Approximate Reading Time: < 1 minute

A popular Cornell professor tries to help language-arts types learn how to “make math” instead of just studying it.

Twelve years of compulsory education in mathematics leaves us with a populace that is proud to announce they cannot balance their checkbook, when they would never share that they were illiterate. What we are doing—and the way we are doing it—results in an enormous sector of the population that hates mathematics. The current system disenfranchises so many students.

Source: Teaching Math to People Who Think They Hate It

Be the first to like.