These are public postings of my writings for the first course of the Graduate Certificate Program in Serious Game Design and Research at Michigan State University.
Please note: these posts are not intended as any kind of commentary on or assessment of the course I’m taking, or its instructor, OR of Michigan State University or the College of Communication Arts and Sciences, or the Department of Telecommunication, Information Studies and Media. They are solely my thoughts and reactions that stem from the readings.
Feel free to comment, disagree, or what have you.
Week 9
These are the readings for the week (Topics: Topics: Evaluation plan (feasibility, acceptability, usability, effectiveness; IRB and logistics):
- Pinelle, D., Wong, N., & Stach, T. (2008). Heuristic evaluation for games: Usability principle for video game design. Paper presented at the The 26th ACM Conference on Human Factors in Computing Systems (CHI ’06).
- Baranowski, T., Baranowski, J., Thompson, D. I., Buday, R., Jago, R., Griffith, M. J., et al. (2011). Video game play, child diet, and physical activity behavior change: A randomized clinical trial. American Journal of Preventive Medicine, 40, 33-38.
- Baranowski, T., Baranowski, J., Thompson, D. I., & Buday, R. (2011). Behavioral science in video games for children’s diet and physical activity change: Key research needs. Journal of Diabetes Science and Technology, 5, 229-233.
- Isbister, K., Flanagan, M., & Hash, C. (2010). Designing games for learning: Insights from conversations with designers. Paper presented at the 28th ACM Conference on Human Factors in Computing Systems (CHI’10)
This week my comments are about the Pinelle paper. It seems that I am not the only one as a number of other classmates have also had some comments.
My perspective comes from that of a long time computer scientist – someone who watched the field of HCI (Human Computer Interfaces) from its very beginnings. This is where theories of usability came from. The idea of studying users to find out how to design better interfaces is a laudable one, but in the interest of full disclosure, I will admit that I have not found the field to provide a great deal of insight.
On the one hand, the goals of the Pinelle’s study are really good: they want to look at usability in popular games in order to generate a set of heuristics others can use when they design their games. This is great idea. I also think they have a pretty good start, but that’s all they have – a start.
This study gathered data by analyzing game reviews. Other people in the class have already pointed out some of the shortcomings of that approach, not the least of which is that game reviewers do not look at games the same way players do. Player reviews would be better, but we are still talking about a self-selected group of people who care enough about games to voluntarily write their opinions and submit them.
For me, there are two things that really stand out as problems with this study:
- They did not do any actual use testing. In other words they did not watch people actually playing the games. I know that the developers do this themselves, but they rarely release any of their data, so it is inaccessible.
- It is not at all clear that any of the researchers actually understand games. I recognize a few names, both in the author list, and in the acknowledgements, and references. The names I recognize are academics who study HCI. To my knowledge, none of them have any game industry experience. Now, this is not necessarily a problem, but it does speak to credibility.
Ben Sawyer always asks his presenters to include a slide outlining what they are playing now in any talk they do for an organization with which he is involved. I think academics who write about what we should be doing to make batter games should similarly have to reveal their own game experience. Academic credentials aren’t enough.
