Wednesday, July 22, 2009

But it's just not FAIR ...

I'm currently readon an interesting book by James Surowiecki called The Wisdom of Crowds. Basically, it's an examination of how groups of people make decisions, both good and bad. And many of the points that he highlights are quite different from what you might expect.

I'm only about half way through the book, but I came across the description of some interesting sociological experiments in chapter 6 (section II). Here's the part that particularly intrigued me:

The explanation for people's behaviour might have something to do with an experiment called the "ultimatum game," which is perhaps the most-well-known experiment in behavioural economics. The rules of the game are simple. The experimenter pairs two people. (They can communicate with each other, but otherwise they're anonymous to each other.) they're given $10 to divide between them, according to this rule: One person (the proposer) decides, on his own, what the split should be (50/50, 70/30, or whatever). He then makes a take-it-or-leave-it offer to the other person (the responder). The responder can either accept the offer, in which case both players pocket their respective shares of the cash, or reject it, in which case both players walk away empty-handed.

If both players are rational [in an economic sense], the proposer will keep $9 for himself and offer the responder $1, and the responder should accept it, since if he accepts he gets some money and if he rejects, he gets none. A rational proposer will realize this and therefore make a lowball offer.

In practice, though, this rarely happens. Instead, lowball offers -- anything below $2 -- are routinely rejected. Think for a moment about what this means. People would rather have nothing than let their "partners" walk away with too much of the loot. They will give up free money to punish what they perceive as greedy or selfish behaviour. And the interesting thing is that the proposers anticipate this -- presumably because they know they would act the same way if they were in the responder's shoes. As a result, the proposers don't make many low offers in the first place. The most common offer in the ultimatum game, in fact, is $5.

Now, this is a long way from the "rational man" picture of human behaviour. The players in the ultimatum game are not choosing what's materially best for them, and their choices are clearly completely dependent on what the other person does. People play the ultimatum game this way all across the developed world: cross-national studies of players in Japan, Russia, the United States, and France all document the same phenomenon. And increasingly the size of the stakes doesn't seem to matter much either. Obviously, if the proposer were given the chance to divide $1 million, the responder wouldn't turn down $100,000 just to prove a point. But the game has been played in countries, like Indonesia, where the possible payoff was equal to three days' work, and responders still rejected lowball offers.

It isn't just humans who act this way either.... primatologists Sara F. Bronsan and Frans B.M. de Waal showed that female capuchin monkeys are also offended by unfair treatment...

Capuchins and humans alike, then, seem to care whether rewards are, in some sense, "fair." That may seem like an obvious thing to worry about, but it's not... The responders in the ultimatum game are being offered money for what amounts to a few minutes of "work," which mostly consists of answering "yes" or "no." Turning down free money is not something that, in most circumstances, makes sense. But people are willing to do it in order to make sure that the distribution of resources is fair.

Does this mean people think that, in an ideal world, everyone would have the same amount of money? No. It means people think that, in an ideal world, everyone would end up with the amount of money they deserved. In the original version of the ultimatum game, only luck determines who gets to be the proposer and who gets to be the responder. So the split, people feel, should be fairly equal. But people's behaviour in the game changes quite dramatically when the rules are changed. In the most interesting version of the ultimatum game, for instance, in stead of assigning the proposer role randomly, the researchers made it seem as if the proposers had earned their positions by doing better on a test. In those experiments, proposers offered significantly less money, yet not a single offer was rejected. People apparently thought that a proposer who merited his position deserved to keep more of the wealth.

Put simply, people (and capuchins) want there to be a reasonable relationship between accomplishment and reward.

Anyone who has worked in game design is probably already familiar with this phenomenon as applies to players in competitive computer games such as MMOs. A frequent lament on MMO discussion forums is how the designers, or the game mechanics in general, are favoring one group over another. Group A complains when Group B gets too much attention, loot, or whatever. As a game designer and also as a player, I have frequently read such discussions with more than a bit of impatience and wondered why Group A is getting so upset about Group B getting something, when it really doesn't affect them personally at all. To pick just one example, if Fred got his Uber Sword of Wumpus-Slaying on day 1, and then we change the quest so that it's easier for George to get the same sword when he does the quest on day 365, why is Fred so upset? Fred still had the sword for a whole year before George got his, and had all the utility, fun, and boasting rights that came with that, and the fact that George now has his own Uber Sword of Wumpus-Slaying doesn't change Fred's game-play experience in the slightest. Yet this is one of the most common complaints on any MMO forum and comes up again and again.

I can't say that reading this study has provided me with any answers to the problem. But it does help me to see that this kind of behaviour is part of a bigger picture of human behaviour, not an exclusive characteristic of MMO players. And it definitely highlights that when developing an MMO, we need to keep in mind that player perception of fairness is perhaps even more important than it is to truly distribute those rewards fairly. "Fair" distribution of rewards according to some cryptic scheme that only the designers can see will not be perceived as fair, and will not be well received. Clear ways of measuring player accomplishments, and equitable rewards balanced in line with those, will go a very long way toward reducing general unhappiness and complaints of unfairness.

This may seem like an obvious statement, but in a complex game like an MMO where there may be many different types of accomplishment, it's far from simple even to equitably measure the different accomplishments, let alone to determine the "conversion rate" between one type and another. Is a "killing the dragon" accomplishment worth more than an "interior decoration" accomplishment if they both took the same player the same amount of time? Is a "raiding" accomplishment deserving of better rewards than a "solo questing" accomplishment? Simple measurement of play time invested is not sufficient to compare the two; and when games are looking more and more to entice casual players who want to be able to have fun and make progress in limited play time chunks, time is not a good measure of accomplishment. It seems almost like a standard that designers of new MMOs should sit down and define before they produce any gameplay content: which type of accomplishment is worth most, what defines worth, what defines the value of the reward that is earned.

Most MMOs I have played seem to have a very rough set of standards along these lines, though rarely comprehensive and often inconsistent. Where they are even more lacking is in making these standards accessible in some form to the players. Result: endless player debates over the comparative merit of one accomplishment over another, and thus, an equally endless debate over whether the MMO's rewards are "fair". Perhaps it's an overly simplistic wish on my part, but I'd love to find a game that defines these accomplishment measurement standards clearly and consistently, and without being so narrow in definition that large elements of the player gameplay experience are missed out entirely. One day! =)


  1. Two comments:

    I think the danger in formal ranking of the accomplishments is that you might drive away players whose gameplay doesn't make the top of the list. This would not necessarily be doing them a favor - maybe all they still enjoy about the game is complaining about it, but they still enjoy it enough to pay money and spend time playing it. Somehow, it's okay if (for example) raids get all of the best lore encounters and loot, but actually being told that raiders are the most important demographic at the game would be disheartening to the non-raiders.

    Also, you wrote:
    "As a game designer and also as a player, I have frequently read such discussions with more than a bit of impatience and wondered why Group A is getting so upset about Group B getting something, when it really doesn't affect them personally at all."

    My guess is that the disputes persist because things that one group gets actually do affect other groups in interconnected worlds like MMORPG's. As you may have noticed from the forum discussion on the locations of the new recipes you added to the last GU, we as players have a very limited understanding of the allocation of development time/resources. There is a perception - sometimes true and sometimes completely untrue - that time spent providing something to group A means less time for group B, that adding a recipe to a vendor is the same as adding a quest that rewards the recipe, etc. Both sides will try to argue that their group represents more of the silent "casual" masses in an attempt to get more stuff.

    There's also the potential for itemization conflicts if you're trying to set out solo quests, crafting, small group content, and raid rewards. Allowing the lower groups on the totem pole to continue to advance would require stepping on the toes of the next group up the ramp, while the alternative is for the groups at the back of the line to be left with nothing to work towards.

    The other issue, specific to your example of nerfing a quest for a sword, is that players do implicitly factor their expectations for how long a reward will be persistent into how hard they are willing to work for it. I've seen server-best guilds give up on raid content they had not yet completed because an expansion coming in a month would include a gear reset. Fred is complaining not just for the reason he says he's complaining, but because the knowledge that quest requirements may change affects his valuation of future rewards.

    If the next patch adds a superior sword of wumpus slaying for killing 100 wumpuses, should Fred get to the slaughter, or wait and see if it gets nerfed to 10 wumpuses a year later? Two months later? A day later, if Fred doesn't read test notes and only discovered that he spent the last week killing wumpuses for a sword that now requires a handful of them only when the change goes live? In the absence of a formal stated policy on how long players can expect their rewards to remain on the top of the game, players will make their own decisions (some will make an untenable argument that their loot should be the best in perpetuity) and argue that some sort of unspoken social contract backs their view.

    I think it was Tobold who pointed out that none of us individual players have the money to have an AAA MMORPG made to our specifications, so all we can do is try to play tug of war with the developers in the hopes of getting someone else's game to look more like the one we want. It's unfortunate, but it's the only model us players know at the moment.

  2. "all we can do is try to play tug of war with the developers in the hopes of getting someone else's game to look more like the one we want."

    I guess my thought was that if you knew from the start that RaidingQuest was a game that would always prioritize raiding accomplishments as the most valuable, or CraftQuest would always prioritize tradeskills, you would not need to "hope" the game would have the priorities you want, you'd already know. And if you chose to raid in CraftQuest anyway, you would be less upset (I hope) if your rewards were not as nice as if you'd completed the 20-step tradeskill epic quest. And if that didn't suit your playstyle you could go play RaidingQuest instead or some other game, but it seems the level of annoyance would be lower.

    At present, I think the tug of war exists in large part because neither the players nor the developers in many games even really know how to judge what is more valuable as an accomplishment nor how to measure that. Or if the developers do know, it's not at all clear to the players.

    As an example, by some standards of measurement, a level 30 carpenter in EQ2 who runs around giving strongboxes and free house decorations to every newbie they can find may BE more valuable to retain as a customer than a raider who only interacts with his own raiding guild, if the carpenter is making sufficient newbies happy and being a significant factor in causing them to stay in EQ2 and stay subscribed. From a company profit perspective you could certainly make that argument! On the other hand, the raiders would have a much different way of measuring accomplishment and how it should be rewarded, and that's not wrong either.

    Tug of wars result when everybody has a different measuring stick and nobody clearly understands which to use when. Players accomplish something they think is impressive, and they feel they deserve to be rewarded. If that doesn't happen because the game developers are measuring accomplishment in an entirely different way, frustration ensues.

  3. I know that there are a lot of reasons for the tug-of-wars. The one that seems most dug-in is the raider vs not-raider (not going there on hardcore vs casual, oh, no! ;). My take on it is that if the rewards for the raiders are the best in game, BUT only so when raiding, that that is 'fair'. I don't personally think that raiding is 'harder' or 'more worthy' of being rewarded than other playstyles, so it *does* irk me if the best lore and best non-raid-centric items go to raiders. Every stat I've seen shows that non-raiders outnumber the raiders, so like many I do often feel like I'm paying for their fun ;o

    EQ2 is one of the better games I've played for balancing that out though -- I find plenty of content to keep me happy, and other than sometimes missing out on the backstory (and perhaps a twinge about knowing I'll never get my Mythical) I don't really envy anything the raiders get. It's just as well it is not a pvp-centric game though -- that's where I've noticed the imbalance to most directly affect the non-raider playerbase.

  4. I think the problem here is often a conflation between means and ends. What I mean by that is that there seems to be an assumption that just because two or more groups fight over the same thing they want the same thing. That's just not true. For example, non-raiders don't want to be raiders. But when they see raiders get rewards they don't get they will fight those raiders for rewards because, as they see it, that's better than being left out in the cold. In this example, what the non-raiders want is equal treatment, equal attention from the developers. It's like the little child who hates ice cream complaining about why his sister got ice cream and he didn't. He doesn't really want the ice cream, he just want to make sure he's not ignored.

    To my mind this is often as much an issue of timing as it is an issue of substance. I think there would be a lots less QQ if, for example, every time the developers implemented a new instance, for example, they implemented a new smashing quest chain. If you have pets watch what happens when you feed one and not the other and compare that to the dogs behavior when you feed both at the same time. There's a lot more peace in the second situation.

    The point I want to add to your discussion is that whether a reward is perceived as fair is often as much about the timing, the order and sequence of events, as it is about the actual substance of what is being fought over. MLK once said that justice delayed is justice denied and people are very aware of that fact. Just because it makes more sense to you as a developer to do the instance first, it doesn't mean that non-raiders wont resent the opportunity cost imposed upon them by that decision.