“We practice iterative design” You hear it in virtually every studio profile, every GDC design lecture, and it’s a buzzword game journalists equate with exceptional game design and a high level of polish. Apparently there is a magic formula for making good games, and it goes something like this.
- Start with a fun game
- Make a tiny change (usually after exhaustive debate)
- Playtest extensively to see if those changes made the game better
- If they did, keep them; if not, change them back
- Repeat until your publisher makes you ship
It’s a pretty straightforward process, anyone can understand it and imagine executing on it. Acceptance of this model leads to some pretty obvious ways to improve your design skills, too. Make smarter changes. Iterate faster. Find better test metrics. Take more time. And now we know why Valve and Blizzard make better games than anyone else, right? They take more time discussing changes, they have better tools for iteration, they playtest more than anyone, and of course, they ship when they are ready. And it’s true, to some extent; a game that went through several rounds of polish will be better than one that didn’t. But if you adopt iterative design as your primary design philosophy you will be doomed to making mediocre clones of better games…
First, let’s examine iterative design applied to other creative forms. Want to write the next great novel? Start with a book you really enjoy, like “The Count of Monte Cristo” and start iterating! Pick a random chapter, change a crucial detail, and then read the whole book again and see if it is better. Clowns are funny – let’s make Dantès a traveling circus performer instead of a sailor. Over the course of 1400 pages, assessing that change is virtually impossible, except that the occasional clown-lover might comment on it. Although no one change will be measurably worse, by the time enough alterations are made to create a substantively different novel, it will be an incomprehensible mess.
Pourquoi cet air si sérieux?
Let’s take a look at a concrete example of how iterative design can fail, specifically in determining the number of weapons the player can carry in a shooter. In Wolfenstein 3d, the player could carry 6 weapons. In order to improve this, an iterative designer would try playing through with 5 weapons and with 7 weapons. Clearly, 7 weapons is better than 5 because the player has more choices. So then the designer would try 8 weapons, which would also test better because “more” always tests better. At some point, the designer would probably decide the gains from adding an additional weapon no longer justified the expense, and the game would ship with the player carrying 25 weapons at all times. The iterative designer would never arrive at the solution introduced in Halo and adopted by virtually every shooter since, of reducing the number of weapons the player can carry, because the benefits of that system are not apparent until you reach 2-3. In mathematics, this technique is known as hill climbing and it can only be used to find a local maximum because it cannot cross gaps to reliably find the global maximum. Every designer knows that a game is never fun at first, so it is likely that even a good change is going to feel like a bad one for a short period of time, and an iterative process will reverse course too early.
You can't get there from here...
The main reason that the iterative process is a siren song is implicit in Step One, “Start with a fun game.” How can anything dramatically new be created if you are only allowed to start with something that is already successful? If you something is already fun, stop iterating on it and work on the parts that aren’t fun. And if something isn’t fun, the iterative process won’t get you there. In the end, iteration is a polishing technique, not a generative one.
Rewarding Play 
I’d be curious to hear the story about how the two weapons-at-a-time idea came about during development.
Hm, that’s not how I understand iterative design. You’ve read Lost Garden on the topic? There the concept of iterative design seems to be about picking chunks of gameplay, treating them as natural experiments (is this fun or not?), and expecting most of the experiments to fail. It promotes entirely original development over starting from a preexisting idea. So, my impression is that whatever you’re criticising here isn’t the iterative development I’ve learned about. Maybe I’m confusing terms, but what would you call danc’s design process?
Of course, Lost Garden was one of the blogs that inspired me to start my own. This post is not meant as a criticism of iteration itself; any cyclical process could be described as “iterative”, including Danc’s. It is meant to discredit the idea that you can make a great game with nothing but iterative design. That through a converging sequence of bite-sized improvements a designer can gradually arrive at a great idea with minimal risk. Sometimes, one must make an inductive leap to a completely new idea (which will probably not be immediately fun) and find some new creative spark _before_ the iteration process can begin.
Danc’s welcome to weigh in, but I believe he has written about taking an intial, reasonable risk by striking out to find something completely new, and then refining that new idea, as being preferable to grinding on an existing, proven idea (and making endless sequels and clones.)
If I appear to be disagreeing with him, it’s probably because I’m not as good a writer as he is. 8)
Ha ha, I think we agree then about what he’s written, and about the futility of grinding. Thanks for clarifying, that helped me a lot in understanding what you’re getting at.
I disagree that it’s as all-encompassing or as poor a process as you imply (although I’m not sure we actually disagree here). Yes, doing only iterative design would lead to Mario 6048. But I haven’t heard of that happening on a project level. Certainly, it wasn’t how Blizzard worked while I was there, like you imply. And yes, in an A/B testing reductionist test iterative design can be hill climbing, but iterative design can also jump hills. Iterative design can get to 25 weapons, true, if more really is better, but it can also take a step back. Done well, on any given iteration it can look at the game as a whole, and go “wait, the mental overload here is terrible” or “look, the player ramping is failing because it’s too much to learn” or even “wait, these choices here are false choices.”
For a good example of this process in a very public way, see what Blizzard did with Talent trees in WoW from launch through Cataclysm, discussed in Tom Chilton’s GDC talk.
Of course, there’s improvements that can be made – you can make better initial designs, you can make larger binary splits so that you aren’t only testing 1 weapon at a time, you can have more experience with what works and what doesn’t, etc. All those things except the last one, though, tend to lead to cause that exhaustive debate you were referring to, so the value has to be pretty heavily scrutinized relative to the quick iteration.
In other words, is it fair to be so (perhaps unintentionally) dismissive of iterative design?
@Dan – Of course, the title of the blog ought to be “In Jaime’s Opinion…” but it gets redundant. 8) I’m not really arguing against (or dismissing) anyone’s _actual_ process, I’m just trying to point out that that very often “iteration” is idolized as the source of good design when it is actually just one context in which good design can happen. The press and interested gamers are lead to believe that iteration is the magical force behind their favorite games and as developers we should probably stop encouraging this view. It eliminates the need for craftsmanship and reduces design to user research and convinces publishers that what they need to do is iterate more when what they often ought to do is hire/train better designers.
@Breath – I actually plan to write a number of these posts, trying to knock down the strawmen of game design and puhs back against over-simplified theories. It’s fine for a sound bite, or to make a point in a GDC lecture, but not to uncritically implement them as a design process when actually making a game.
Well said.
I’m curious how much iterative design (more as I discussed it then the popular press view) you value and like to use? And when do you know to take a step back/stop?