GDC 2010: Achievements

It's been too long since I posted here, but in my defense, I took some time off after the expansion launched, and at present I'm in San Francisco after attending GDC.  For those readers not in the industry, that stands for Game Developers Conference, and is a week-long conference consisting of discussion panels, round tables, lectures, a large display floor, and lots and lots of opportunities to mix, mingle, demo, meet, and generally discuss anything and everything to do with games.

I have been here for 3 days (Thursday, Friday, and Saturday) which is when the bulk of the talks/lectures are scheduled, and I've taken the opportunity to attend quite a few of them.  I had never been to GDC before so I decided to attend this year mainly to see what it's like.  Very interesting, is the answer!  Some of the talks were better than others, and inevitably there were a few annoying time slots when I wanted to attend two different things at once (or even three), but enough of the information was interesting enough to make me quite enthusiastic about the topics.  GDC is also a great chance to meet others in the industry, and catch up with old friends who may have moved on to new companies and/or cities.  This week I've had lunch, dinner, or drinks with people from Blizzard, Bioware, Ohai, 38 Studios, Trion World, and more; and listened to people from these companies and many more discuss their own perceptions of gaming, their own product's particular challenges or successes, and much more.

Two lectures particularly enthused me, Chris Hecker's talk on achievements, and Dmitri Williams' talk on what kind of useful information you can get out of your game data.  The second probably does not make for a very interesting blog post unless you are particularly enthused by business intelligence databases, but I'll take a stab at summarizing the first.

So, ever since Skinner and particularly since the 1970s, psychologists have done a lot of studies about what motivates (and demotivates) people.  There's a ton of data out there, however not all of it agrees on all points, since measuring people's behaviour is not always an exact science.  However, according to Hecker, over all the studies it was possble to pull out two key points on which they all agreed:

For interesting tasks:

1. Tangible, expected, contingent rewards reduce free-choice intrinsic motivation
2. Verbal, unexpected, informational feedback increases free-choice and self-reported intrinsic motivation

A couple definitions of terms here - intrinsic motivation means that the task itself is motivation to do it (like, I want to do a crossword puzzle because it's fun to do a crossword puzzle - the motivation is in the task itself). This is as opposed to extrinsic motivation, which means the motivation comes from outside the task (like, you pay me to do crossword puzzles).  Tangible reward = real, you can touch it, such as money, as opposed to verbal rewards like someone saying "good job".  And contingent reward means that whether you get the reward depends on what you do, for example knowing that every time you finish a crossword puzzle you get $1, as opposed to unexpected rewards which you can't anticipate when or whether you will get them.

To give some examples of the interesting task rewards listed above:

1. Giving someone a tangible reward they know about in advance which is based on their performance (e.g. read this book and I will give you $5) reduces people's motivation to do the task subsequently without a reward.  (e.g. they are less likely to decide they want read books on their own, for no reward, after you start paying them to).
2. Giving someone verbal feedback they aren't expecting makes them more likely to report they are likely to do the task again without a reward.  (e.g. unexpectedly telling someone "hey, you read a book! that's great!" may make them feel more positive about reading more books in the future).

In this talk, only interesting tasks were looked at.  For boring tasks, like data entry for example, promising rewards can have different effects on motivation because people are never really going to be motivated to do really boring stuff for the sake of doing really boring stuff (there is little or no intrinsic motivation to boring tasks). But the talk ignored boring tasks, since it's a sensible assumption that computer games are made with the intention of being interesting, and nobody intentionally makes a computer game to be boring.

There were many examples given of how offering rewards changes the amount of intrinsic motivation that people reported to do tasks.  Some examples:

• take a group of children drawing things.  Give half of them candy as a reward for drawing things, and don't give the other half anything.  Next time, the children who got the candy will not be as interested in drawing things.
• offer a child money to clean his room.  The room will never again be cleaned unless money is offered again.
• give people money to wear seat belts, they will wear seat belts less often.
• give kids a reward for reading, they will read less.

Once an external reward is offered, the focus changes.  The task itself is made to look less interesting than the reward, and people focus now on the reward instead of the task, which becomes just an annoyance to complete as a path to getting the reward.  There are numerous studies showing this (and even quite a few arguments published demonstrating that this is why investment bankers should not be rewarded with big bonuses).

Now, if you're at all familiar with achievement systems in computer games then you can probably see where this is going.  Achievements are an external reward system which is contingent on completing certain specified and usually predetermined tasks.  Whether you define an in-game reward as tangible or verbal is a matter for debate, since most of the existing studies looked at real world examples, which do not translate entirely directly into a virtual world, but it probably makes sense for an in-game item to count as a tangible reward.

So what are achievements doing to players' enjoyment of the games?  If we tell you that you'll get an achievement for killing 500 orcs, do you now curse and swear as you slog through joylessly killing 500 orcs just to get the achievement -- have we removed all the fun from killing orcs for the sake of killing orcs?

In EQ1, many many years ago, I spend hours and hours and days and days doing various crazy things just for the sake of doing them -- such as killing enough minotaurs and kobolds to be non-kill-on-sight in the gnome city even though I was a dark elf, or handing in endless numbers of bone chips to allow me to safely enter the city of Cabilis.  There was no reward for doing any of this other than being able to stand safely in those cities surprising the "normal" residents, and I spent days on it.  There was even one completely dedicated player I knew back then who set out to earn friendly faction standing with every single tribe of goblins in the world ... and eventually did so (even in Droga).  This was not a task of hours, or days, or even months, it was literally years and years of dedicated work.  There were no achievements back in those days, this was all done just for the interest of the task and the challenge itself.

In EQ2 or in any other MMO that I play nowdays, I can't imagine ever spending that amount of time to do anything similar.  Unless of course we put in an achievement for doing something like that ... in which case someone would do it, just to get the achievement.

Hecker's "nightmare scenario" was as follows:

1. You make an intrinsically interesting game.  Congratulations!
2. You decide to add extrinsic motivation (ie, achievements) to make your game even better.
3. Unintentionally, you destroy players' intrinsic motivation to play your game.
4. Because achievements are easily measurable, the metrics that you are using to measure player actions actually push you towards designs where extrinsic motivations work better and better - which becomes a less and less intrinsically enjoyable game experience.
5. As a bonus, women appear to lose intrinsic motivation to enjoy tasks even faster than men do, when extrinsic motivation is added.  So you also end up distancing your female players even more.

Nightmare result: you end up with a boring and unenjoyable game that people (mostly men) grind through just to get the achievements.

Hecker was the first to acknowledge that we need to do more research to see how the psychology research he referenced would translate into virtual worlds.  He is hoping that someone with access to the data needed will be able to produce more information, or at least make the data available for others to research it.  But there certainly seems to be enough real-world-based evidence to raise questions such as, are achievements actually making our games less intrinsically interesting?  Or are we implementing achievements in a way to ensure we don't do that?

In closing, until we can definitively determine how achievements are affecting gameplay experience, his final note of caution was to minimize the potential damage from achievements as follows:

1. Don't make a big fuss about them
2. Use unexpected rewards
3. Use absolute, not relative measures (e.g. "you killed 10 rats", not "you are the 9th highest rat-killer in the world")
4. Use endogenous rewards, not external ones
5. Make them informational, not controlling

Chris Hecker's website is at http://www.chrishecker.com, although unfortunately he hasn't updated the site yet with his talk on achievements.