A hair lip?

Design by Numbers: Simultaneous Perception

Perceived Simultaneity Threshold: 100 milliseconds

Let me tell you how I got my nickname, Jaime “Three Frames” Griesemer.

A hair lip?

Emily "Three Frames" Crazyhair

We were working on the melee animations for Halo 3, trying to get the timing right. There were already fast. Like, blink-and-you’ll-miss-it speed. The animators were having trouble because there wasn’t enough time to add a sense of anticipation and sell the impact properly. And for some unknown reason I wanted the damage keyframe to happen faster. “The melee animations can be as long as they need to be… as long as they’re exactly three frames.” We went back and forth about it, but eventually I traded three frame melee attacks for some extra flourish on the recovery after the hit. Everybody was happy.

Until the next day when I went back to try the new melee animations. I picked up the Assault Rifle. I pressed the B button. Three frames later, the butt of the rifle crunched into the face of a helpless alien invader. I picked up the Plasma Pistol. I pressed the B button. Three frames later, my left hook connected with a random marine that was looking at me funny. I picked up a shotgun. I pressed the B button. And for some unknown reason it didn’t feel right when I cracked open the head of a passing parasite. “Wait. That wasn’t three frames. Yeah, I’m sure, check the file. See… four frames!” (In hindsight, I’m lucky my uncompromising design demands didn’t earn me anything worse than a mocking moniker.)

What was the unknown reason? How could I tell the difference of only a single frame? Why did I insist on the speed at the expense of animation quality? The answer lies in our physiological limitations. What we experience as consciousness — as a stream of consecutive events in a clear and tidy order — is not nearly so neat. It’s a cacophony. A discordant mess. Your senses collect an incredible amount of information; they’re always on, always filling your nervous system with an unrelenting torrent of unfiltered data. It comes in at different speeds and at different times. The visual part is heavily processed, so it comes in late and is usually out-of-sync with the rest of your senses.

Change your mind?

Your brain at work

Your brain is used to this. It is, in fact, extremely good at taking all that information and combining it into a coherent experience. So good, it can take distinct events and merge them. Asynchronous events like pressing a button and seeing a melee attack. This feeling of simultaneity greatly increases the player’s connection with the action on-screen. Instead of a second-hand signal that transmits their intentions, pressing the B button becomes the act of dishing out a knuckle sandwich. But that requires your brain to blur over the delay and merge the movement of your thumb with the satisfying crunch.

Below a certain time threshold, the brain takes two events that are functionally separate — one causes the other and could not happen at the same time — and gives the impression that they were simultaneous. It turns out that threshold is around 100 milliseconds, or a little more than three frames. The reason four frames felt wrong was because the events were experienced as distinct, disassociated actions. The extra time allowed my brain to distinguish between them, in a sense inserting the controller as a step in the process and reminding me that I was not actually the one wearing metallic boxing gloves. So, for games where it is important to give the player a feeling of direct control, be careful of crossing the 100ms threshold.

Advertisements

GDC 2010: Design in Detail XIX


And there it is! The big detail!

It’s a large change. It’s very easy to convince yourself that you can feel tiny changes, but you will be fooling yourself. The balance never hinges on a 2% difference in a single value!

It was a smaller change than we tried initially. I think originally we changed it to 0.9, which broke the flow and wrecked the weapon, but did fix the problem, so we knew we were on the right track. In general, you want to overshoot and then come back. You have to make sure your change accomplishes your goal, and then dial it back.


There are lots of ways to verify that a change was successful. In this case, the Sniper Rifle didn’t get any less popular; People still use it whenever they can get it. But Optimizers stopped using it exclusively.


The other reason we could tell it was balanced was because we could compare how the behavior we were seeing in playtests matched the desired role we described in our paper design. It’s not quite an objective test, but it should help. And most importantly, I no longer got nervous when I watched people use the Sniper Rifle. You should verify a change with both your brains and your guts.


Ship it!

At this point during my GDC talk, the audience started clapping, cluing me into the fact that I was completely out of time.
So I just went with the flow and ended it there, which was fine. I had walked people through the balancing process, brought up the important principles, and applied them to the sniper rifle change. But I had also intended to mention a couple of things about the last stages of balancing, which I will get into next week!

GDC 2010: Design in Detail XVIII


Here’s what we didn’t do. We didn’t touch the Strength Knobs. In most cases, they aren’t the problem anyway. When a weapon is being used as intended, it should feel overpowered. So most imbalances come from using a weapon outside its role, in which case changing the strength knobs won’t fix anything, it will just make the weapon worse.

We also didn’t try to add a weaknesses. It often feels like the only option, but find something else! That sort of “a little hot, a little cold” design never ends. You’ll just chase your tail until you run out of time.

Your strategy should be to find where the element is being used outside of its designated role, and change the mechanics to constrain it better.


But if you can’t touch the Strength Knobs, you have to touch the Flow Knobs. (Remember, there isn’t anything else, because we removed all the extraneous mechanics!) The tricky part is to fine tune those Flow Knobs without losing the flow state you worked so hard to capture. Revisit them in light of what you now know about the game and don’t change them so far that you break flow.

Of the components of flow, cadence is the most flexible, so many problems can be fixed by adjusting cadence. Hopefully not so far that you lose the rhythm.

Most importantly, at this point you must not rely on your gut, it will steer you wrong! You need a very clear chain of cause and effect, so you can make as small a change as possible to fix the problem.


So what are the possible flow knobs we can tweak? What will achieve the balance objective with the least amount of ripple effect?

We could have changed the number of shots in a clip. That would have changed the cadence to cause the player to reload more often. But it would also have meant a sniper couldn’t kill two enemies without reloading unless he got a headshot. That would have ratcheted up the pressure quite a bit, and moved the Sniper Rifle out of the skill range of most players.

We could have increased the length of the reload. But dying because you can’t fire back is frustrating. To be honest, this change was a contender, but it felt too much like adding a weakness.


We could have changed the time to it took the Sniper Rifle to reach full zoom. We actually tried this as a solution (and found some bugs in the camera code, too!) But in the end, it encouraged players to fire without zooming, which broke the role worse than the original issue. In fact, it exasperated the problem of players using the Sniper Rifle at close range.

We also could have prevented the Sniper Rifle from doing headshots when unzoomed. Unfortunately, this removes a uniquely cool moment, which even average players can experience if they get lucky.


We could have changed the maximum total ammo. The problem is, this would have limited the overall effectiveness of the Sniper Rifle without changing the experience of any individual combat encounter. (For more on the perils of this technique, read Against Statistical Design.)

Finally, we could have changed the time between shots. We didn’t choose this option immediately; we tried, tested and reverted almost every value on this list. In the end, increasing the time between shots was the only one that fixed the balance problem with a minimum of side-effects.

Treating Iterationitis

Iteration is the key to good game design.  Everyone knows this, not just designers.  Artists, programmers, even the producers in charge of the schedule acknowledge that iteration is a necessary evil — a gullet of unknown appetite that must be sated.  However, this abstract understanding breaks down almost immediately when faced with this inflexible reality:  If our plan is to iterate on the design until the game is fun, it will not be fun when we begin, and will not be fun at any point in the iteration process until literally the moment before we are done.  Successful designers have accepted this reality to some extent — they have to.  But non-designers, deprived of both visibility into the iterative process and ultimate responsibility for how it turns out, have not been forced to come to terms with this uncomfortable truth, and so they get nervous.

It is in the designer’s best interest to minimize the impact of their own uncertainty on the rest of the development team.  Some of them do this through bravado — how dare you question my ability to produce results?!  Others present design as a black art, with arcane secrets known only to themselves — an illusion which a perceptive non-designer can dispel with a few well-considered questions.  Many use this uncertainty as camouflage, avoiding direct conflict by constantly iterating without revealing their ultimate goals.  The problem with all of these techniques is that they can be used not only to reduce the team’s discomfort with iteration, but to conceal a lack of design discipline or creative horsepower.  It becomes impossible to discern the difference between a good designer protecting the team from the difficulties of iteration and a bad designer hoping to stall long enough to get lucky.  What is required are methods to iterate quickly while minimizing the team-wide anxiety.

Not how iteration works

Continuity of Iteration

If a designer stops iterating on an element or system for more than a week, other team members will assume that they are done with it.  They will evaluate it as if it was finished and assume that the designer is satisfied, even though it is clearly not fun.  Left unchecked, this chain of assumptions will lead them to distrust the designer’s abilities and judgement.  This happens almost entirely subconsciously, so communication to the contrary is rarely effective.  The only real solution is to keep iterating, even when the correct next step is unclear.  Just change something so that bystanders will postpone their evaluations.  This also avoids the problem of inertia, where developers — even designers, sometimes — grow accustomed to an element’s broken state and no longer notice that it isn’t fun.

Tools and Systems for Iteration

Any non-designer that is an integral part of the iteration loop will be exposed to the chill of uncertainty on a daily basis.  So the simplest way to reduce their unease is to make sure the tools for iterating don’t require their constant support.  With the proper tools, a designer should be able to experiment entirely on their own, without external resources.  If these tools are unavailable, it may require the designer to broaden their skill-set until they can hack together what they need.  That’s why most experienced designers have a working knowledge of so many tools — 3D modeling software, source code editors, Photoshop, etc. — not because they can produce shippable content or code, but so they can try off-the-wall ideas without needing help from outside the design team.

Private... not on TV

Private Iteration

Another benefit of tools that designers can use on their own is that they allow for private experimentation.  Sometimes the iteration process is random and goes through an enormous number of bad ideas before discovering a good one.  Again, since most non-designers are not confronted by the constant, unmitigated stream of failures required by the iterative process, watching the ones responsible for the ultimate quality of the game appear to be blindingly incompetent for months on end can be… unsettling.  It takes an emotional toll that can be avoided when a designer does most of their failing in private, witnessed only by other sympathetic designers.

Herald Milestones

Of course, chronically working in private has one major disadvantage — nobody sees your successes, either.  Since a lack of perceptible progress can be as damaging to team morale as public regression, it is crucial to announce any concrete improvements as soon as they happen.  No need to wait for an element to be complete, so long as a decision is reasonably certain and isn’t going to change immediately, let people know!  “The speed of the tank has been decided!”  “Jump is definitely going to be on the ‘A’ button!”  Pointing out these small, but measurable, signs of certainty will give the team a sense that there is light at the end of the iteration tunnel.

We'll get there... eventually

Even permanent failures can result in more certainty.  “We have given up on the hovercraft!” “We are no longer working on the Shadow Beast!”  The objective is not to demonstrate the perfection of the design team, but to show an inexorable drive toward closure.  The team must understand — not just intellectually but in their bones — that if the iteration process continues as it is, at some point in the future, the game will be done.

GDC 2010: Design in Detail XVII


This is the point in development where we finally changed the Sniper Rifle. Now I will try to describe how all the work from previous passes informed this decision…


The Sniper Rifle was overpowered — that’s what we intended, remember – but it made the other aspects of the game feel weaker.  We couldn’t make the rest of the arsenal strong enough to keep the Sniper Rifle in line.  One way we could tell was because the players we had picked out as Optimizers were using it exclusively.  Role Players, on the other hand, were still not using it, but suffering for it.


Worse, the Sniper Rifle was being used at close quarters, which is clearly outside of its role.  And nothing the targeted player could do would allow them to avoid being sniped.


When something impacts you emotionally we say we were “moved”.  Emotions are what compel you to act — not graphs and data.  Use your Sense of Balance to feel when something is wrong and trust those instincts.

Nerf Herders?

Against Statistical Design

Statistical Design

I suggested that a good way of improving one’s design sense is by staring at Rorschach Tests, and here is a practical example of the importance of practicing pattern-avoidance.

To me it looks like a designer's brain in a vice

Stop seeing patterns!

This image is a heatmap showing where people most often die on Assembly, a Halo multiplayer map.  These heatmaps were first used by the Halo design team to analyze maps during testing, but were so interesting looking they became part of the bungie.net statistics pages.  This data is so rich — so detailed and specific — it must be useful to a designer in some way, right?  The problem-loving brain of the game designer latches on to this as The Solution and immediately starts searching for The Problem.  It is tempting, given a powerful tool like statistical analysis, to incorporate it into the design process somehow — especially since design is often stranded in a world of abstraction and uncertainty.  Having concrete numbers is a rare treat.

However, what does this data mean?  Are red areas bad?  Should dark areas be eliminated?  Does a well-designed multiplayer map have a symmetrical shape?  What percentage of a map should be yellow?  Something about high-contrast feels unbalanced, so perhaps the map should be revised so that the gradient from safe to dangerous is more continuous.  And areas where nobody dies seem wasteful, maybe they should be removed.  And obviously the red areas will be frustrating, so they should be made safer by limiting line-of-sight and adding cover.  Pretty soon we have a completely yellow multiplayer map, that we have tricked ourselves into believing is balanced because our data looks pretty.  We have fallen victim to statistical design.

Players Aren’t Statistical

Statistics are powerful tools because they aggregate a large number of unique instances into a manageable form so it can be analyzed.  It would be impossible to watch every death of every player across thousands of games and have any cohesive understanding of how often players were dying in a given area.  Given enough examples, we would develop an emotional feeling of dread or security associated with certain spots, but the brain uses a very unscientific method to determine these attachments.  Exciting experiences are weighted much too heavily, which is why the impartiality of statistics is useful in discovering imbalances.  Using statistics to find problems is fine; designers go wrong when they use statistics to evaluate solutions.

Players don’t engage with the game statistically — they experience it personally.  It doesn’t matter if more players are killed standing in a specific spot than anywhere else on the map, what matters is the unique experience of a player killed in that spot.  If they realize that they shouldn’t have crested the hill with no cover that is right below where the Sniper Rifle spawns, vow not to do that again and move on, there is nothing wrong with the map.  Even if they do it over and over, growing more and more frustrated at their repeated mistake and creating a bright red dot on the heatmap, the map is not unbalanced.  However, if players are forced to expose themselves at a single chokepoint, or get sniped through a hidden line-of-sight in an otherwise safe area, it doesn’t matter if it is a rare experience and there is no red, the map ought to be fixed.  Neither of these situations can be found through statistical analysis, and neither of them are fixed by a solution that merely addresses the probability of being killed in a given area.

Avoiding Statistical Design

Some systems can only be balanced statistically.  If there are three factions in the game, and one faction wins 43% of the time, the factions are not balanced.  If a map is intended to be used for two-flag CTF, but the bases aren’t mirror images of one another, then the two sides had better be perfectly fair.  The necessity of reverting to statistical methods is inherent in the design of the system itself.  The designer will be forced to make changes that do not change the unique player experience — or may even harm it– in order to fix a statistical imbalance.  Worse still, players are skilled at detecting when a system must be balanced statistically, but since they do not have access to hard numbers their personal experience will tell them that it isn’t balanced — even when the data says that it is!

Nerf Herders?

Nerf Paladin?

Well-designed systems do not need to be balanced through data-manipulation.  If there are 10 weapons in the game, and one weapon is responsible for 20% of the kills, there is probably not a problem.  If the unique player experience isn’t negatively impacted, the statistical difference isn’t a balance issue.  So, the easiest way to avoid the trap of statistical design is to avoid systems that must be balanced mathematically in favor of those that can be balanced behaviorally.  If a system requires a large amount of instrumentation and is extremely sensitive to tiny value changes, instead of obsessing over statistical patterns, try revisiting the system’s design and making it less brittle.

GDC 2010: Design in Detail XVI


Sometimes you have to let your head drive.

I’ve claimed in this talk that your rational brain is a feeble instrument and cannot handle the volume of input associated with balancing a multiplayer game. This is true, but that is not an excuse for presenting your design decisions as inexplicable black magic guided solely by intuition and guesswork. You need to have solid design values that you can point to as a general explanation for how you make decisions, and then provide specific reasons and supporting data when necessary to convince the team. Just because you don’t have time to carefully examine every detail doesn’t mean you can’t logically examine any decision. Other designers and developers have instincts, too; if they clash with yours take it as a sign that you need to look closer.


Here are some good design values to use when balancing a complicated game system. First, fix imbalance with physics when you can, and only revert to math when required. The player cannot directly experience the math behind the system, so tweaking elements by changing mathematical formulas is difficult to perceive, tough to evaluate and even if it does fix the problem in the long-term, it may not feel fixed. Of course, at their most basic level all video games software are math, so it may be impossible to address a problem without making changes to numbers and equations, but don’t resort to that immediately. I think a lot of designers got their start with board games and pen-and-paper RPGs, where the math is explicit, but modern games are simulations and respond more readily to behavioral changes than to statistical ones.


Hey look, a totally fair game… A totally boring, pointless, frustrating, fair game.


You cannot make a Sniper Rifle fair. The person being sniped cannot counter-attack, faces near-instant death, and usually doesn’t even know they are in a fight until it is too late! If you make fairness your goal you will end up removing all the interesting asymmetry from the game. Instead, focus on longevity. Create a Sniper Rifle that doesn’t make the person that was sniped want to quit, and you will succeed.