How can we wrap our heads around the chaos of game development? By understanding that the famous phrase “find the fun” implies something important: discovery. How do you manage the creative process? By acknowledging the latter word of the phrase: process. If you can understand how those terms related – and where they differ – you can appreciate something vital to effective production. That nothing we do in game development is completely devoid of process. And, if you can learn to separate the process from the discovery, then science becomes a weapon against the dark forces of development hell.
By Reading This Post, You Will Learn
- The Discovery versus Process framework for understanding your activities
- Why discovery is the source of much of our management woes in game development
- How scrutinizing and improving our processes can help us better contend with the risk of discovery
- The contents of the various parts of the “Game Planning With Science!” series
Discovery Versus Process: An Preface to “Game Planning With Science!”
By the best estimates of paleontologists, our genetic predecessors, the hominins, discovered how to make fire approximately 350,000 years agoº. By the time our homo-sapien ancestors entered the picture about 150,000 years later, that discovery, through repetition and iteration, became something else: process. An understood sequence of steps that produced an predictable result.
This observation leads to another: that all human activity falls along that same spectrum. On one extreme, you have activities that are pure process. Boiling water, for example. There’s no mystery there. You apply a certain amount of energy to a given amount of water and it will boil. On the other extreme end is discovery: the unknown. There lies the bleeding edge of science, the depths of space, and new forms of expression.
This spectrum has a few dynamics that are worth recognizing. First, there are few extreme examples of process or discovery. Most activities represent a balance between the two. Cooking for example. Many elements of cooking – fundamental combinations of ingredients, for instance – represent forms of process. But other forms, such as specific temperatures, timings, or novel ingredients, yield discovery.
The second dynamic to recognize is all activities start as discovery and, over time and repetition, move closer to process. This applies both globally and personally. Someone had to be the first person to create pasta sauce, but now it’s commonplace. Likewise, the first time you cooked your own pasta sauce from scratch, it probably involved a lot of discovery. But, by the hundredth time, it was probably closer to process.
Discovery is the Soil in which We Labor
As game developers, much of what we do is discovery. Hell, the phase “Find the fun” even has the word FIND in it. And that is the source of much of the pain we experience. Because discovery, by it’s nature, results in uncertain outcomes. And uncertainty creates risk. Or, as the data scientists call it, variance.
Because that’s what risk is: variance from an expected outcome. If you jump into the ocean with a cinder block tied to your ankles there’s no risk: you are definitely going to die. It’s not risk, it’s certainty. If jump into the ocean with the cinder block tied to your ankles and a knife in your hand, that’s risk. The spectrum of potential outcomes is far more varied.
The magnitude of the risk is the measurement of width of the spectrum of outcomes. An investment of $5 that could yield $10 in the best case has a fairly narrow variance: best case you’re up $5, worst case your down $5. An investment of $10,000 that could yield $20,000 has a much greater variance (-$10,000 to +$10,000) and thus a much greater risk.
So, if discovery produces so much grief because of variance, should we abandon it? Just stick to making games and features we’ve made in the past? Of course not. Discovery is what makes our industry great. It’s what makes game development rewarding.
Taming The Beast of Variance
But neither do we need to resign ourselves to being at the mercy of variance – if we accept two concepts.
First, variance compounds over subsequent activities. If Activity A directly precedes activity Activity B, then the expected variance of the former magnifies the expected variance of the latter.
Second, let’s also acknowledge that NOTHING we do is pure discovery. Even the most in the weeds, experimental, exploratory design is facilitated by some amount of process. Processes by which features are spec’ed out or the act of coding the features themselves. Or the processes by which those featured are submitted to source control repositories, or subjected to QA testing. Then there are processes for actually generating builds or deploying them to distribution platforms.
So, if we want to get a handle on the variance that causes us so much grief, we should endeavor to make any process oriented activity:
- Whenever possible, automated
Why The Fuss Over Process?
Curtailing the variance of our processes accomplishes two important goals. First, it reduces out overall variance. If, as I said previously, variance compounds across activities, then it also follows that reducing the variance of one or more activities in that sequence reduces the overall variance of the entire sequence.
Second, reducing the variance of some activities of some activities in the sequence increases out ability to absorb variance from the other activities.
And that notion is what “Game Planning With Science!” is all about: how to understand, measure, and compensate for variance. The series of articles covers nitty-gritty topics like statistics and operation science to higher-level concerns like management psychology and long term forecasting.
Here is a preview of what you will find in the “Game Planning With Science!” series:
The first post covers two of the most fundamental aspects of operation science: process flows and Little’s Law. The post shows you how to analyze any process flow pipeline by identifying its critical path and its bottleneck. I also define Little’s Law and how it defines the relationship between three key aspects of any pipeline process: the amount of units currently in the pipeline, the time necessary to complete any single unit, and the throughput of the pipeline.
Part 2 builds on Part 1 by taking those process flow fundamentals and using them to optimally staff a hypothetical character art pipeline. Using a simple tool called a “capacity chart”, you can quickly and easily analyze how staffing impacts the throughput of any process by impacting which activity is the bottleneck. Part 2 is a step-by-step walkthrough of how to assemble and interpret a capacity chart along with some special case calculations.
In order to understand and account for variance, you have to have a baseline understanding of statistics. That’s what Part 3 is all about: a crash course on statistical analysis. The post will clarify the difference between statistics and the real world and define a “probability distribution”. It also explains why calculating the arithmetic average of data only tells you part of the story and why it’s essential to pair that average with an understanding of variance.
Building on Part 3’s statistical primer, Part 4 digs into one of statistics’ most important concepts: the central limit theorem. The central lime theorem allows us to make predictions about outcomes even when we have limited data. In the post, I’ll show you, with in depth instructions how to use Excel to make meaningful estimates. For example, you’ll learn how to estimate the probability of hitting a particular delivery date and, conversely, what range of dates you could hit with a desired probability. I also show you how to calculate a “confidence interval” which provides a range out outcomes you can hit with a given probability. This is the most math centric portion of “Game Planning With Science!”, but hang in there because it’ll pay dividends down the line!
In part five, I show you how to greatly simplify your data gathering using a simple concept from scrum: story points. The post starts by drawing a distinction between accuracy and precision, and then move to an explanation of why forecasting with discrete units of time is inherently problematic. I then define what a “story point” is, why you should use it, and how to perform a proper story point estimate.
Having established the value of story points in Part 5, Part 6 describes the value of their counterparts, user stories. The post provides a formal taxonomy for a successful user story. It then moves to an argument in favor of the value of disciplined feature planning. I also define the concept of “omitted variable bias” and how you can use it to your advantage when estimating the scope of user stories. The post concludes by describing how to play “planning poker” and how to avoid the negative impacts of social proof.
Part 7 is a capstone post – it ties the prior 6 posts together and provides a comprehensive methodology for estimating the time to complete a given amount of work while taking variance into account. The post covers scheduling for both art assets and feature creation. It also provides guidance on how to make reasonable assumption if you’re just getting started and lack data.
Part 8 offers a counterpoint to Parts 1 through 7. Namely, all of the data and predictive models in the universe are no substitute for responding to real world change. So, as important as it is to gather data and plan carefully, it is just as important to throw those plans away when they no longer match reality. Part 8 offers multiple scenarios where it is important to think critically about what you see. Making games scientifically doesn’t just mean applying complex formulas and statistical calculations. It also means applying the scientific method in order to make the best possible decisions.
Enjoy and Remember: The Internet is Awesome
“Game Planning With Science!” represents months of work. And that’s not counting the time I spend in school learning all this stuff. As such, it represents the best case I can make for using these tools and concepts. I’ve tried to make these posts both as comprehensive and as digestible as I can. But the beauty of a blog is that nothing is set in stone. I will continue to iterate, edit, polish, and update these posts both as I improve as a writer and as I continue to learn.
The other beautiful thing about blogs is that they are a starting point for dialog. So please, as you read these posts, comment if the spirit moves you. Correct me, question me, challenge me. I’m always happy to talk shop and to learn new insights.
Now then, let’s get started! Click this link to go to Part 1: Video Game Art Pipelines
- All activities fall on a spectrum, with discovery on one end and process on the other
- Discoveries are activities that venture into the unknown
- Processes are activities that are well know and predictable
- Few activities are pure process or pure discover; most are a blend
- Discovery by necessity carries risk, which operation science calls variance
- Variance compounds across sequential activities
- Much of what we do as game developers is discovery oriented, but nothing we do is pure discovery
- For all of the things we do that are process, we should seek to make those activities efficient, predictable and automated whenever possible
- Reducing the variance in our process-oriented activities will, by extension, reduce the variance in subsequent activities
- It will also allow us to absorb more variance from those activities