Iterating towards value

It is commonly accepted that the user value in a game and most things has a statistical probability to increase with the number of iterations it has gone through. This goes for things such as fried meatballs, your DNA, a guitar solo, a sound and so on.

This takes on different forms in different areas of life, since I am interested in writing about game development I will focus this story on some of the forms involved with making games. The process relates to cooking and music production, but since this is a big topic I’ll leave most such fun references out.

The relationship between iterations and value is often plotted as a neat little graph.

Note the S-curve type of deal going on. That says something about the existence of “no value” iterations at the beginning of the process. You got to get to some point before you have anything, then eventually it is finished and stops getting any better. To get anywhere interesting with this topic we have to take a look at the definitions of the graph. What is the value you get as a function of # iterations?

What one iteration is

One iteration is the work done from the point where there is an idea about doing something to the point where what was done has resulted in useful feedback. This is a fractal concept and it has several types of zoom levels and dimensions. The one zoom level most commonly associated with game development is on the product level. One iteration on that level have its value measured by feedback originating from the potentially shippable product through various test procedures such as user testing, technical testing and getting accepted by the product owner. These are examples of something I will call explicit feedback.

Explicit feedback gives a quantifiable measure of value which is obviously visible to the involved parties. These summarize as “done” when everything pass defined criteria often called a “definition of done”. Generally if feedback from the work shows an insufficient or negative value increase the iteration has failed.

Another zoom level for one iteration with an explicit feedback loop is the creation of code data, commonly called programming. If the code is broken it will fail to compile and all kinds of feedback is generally built into the code pipeline to prevent broken code from entering the system. The programmer makes a new iteration of the code to improve the value of the code without first shipping the code to the end user to test if it has any value. These iterations are fast, some broken code can be iterated on in a matter of seconds. This time unit defined by the time it takes to iterate is sometimes called “cycle time”, which I will also use a fractal concept. I will call this comparably low zoom level of iterating for “creative process”, the programmer creatively create the code.

Another type of creative process is the creation of images. The artist uses an image creation tool such as photoshop to create image art. This is also a fast process. The artist will generate art data on a second to second basis, undo things that go wrong and create new pixels with a stylus without thinking of it as iteration. The artist also maintains and switches between multiple options throughout the production of one image. The feedback which closes one iteration in the creation of a pixel of art data is internalized by the artist. I will call this implicit feedback.

Implicit feedback is not obviously visible to anyone. It is to the observer a subjective measurement and often leads to all kinds of emotional distress for the involved parties when it comes time to analyze the value of the output of the creative process based on implicit feedback structures. To the professional artist it is not fully subjective as some principles of the arts guide the creative process but in large it’s a relative mess compared to the efficient explicit structure of code creation. It also happens that different artists use different principles for this process which often cause communication problems, but mostly everyone has learned to live with this process quite productively.

Now that we have a model to differentiate iterations between ones based on explicit and implicit feedback it becomes much easier to understand what things you can put in through the value creation process.

A guitar solo runs its first several thousand iterations as an implicit process before it reaches the point where it gets explicit feedback in the form of praise or complaints from band members, audience or other people involved with the music production. Improvisational jazz is an example of the solo going directly from implicit practice to the end user without explicit iteration, even driven by the goal to change between each gig! The details of the solo are guided by a lot of musical theory internalized by the jazz guitarist.

The development team for a product increase its value based on explicit feedback from the producer or the team itself, as well as the implicit feedback within each team member and their feelings towards doing good or bad work. Scrum is an example of a process which forces some of this feedback to be explicit. This is good in the sense of providing direction but as all game designers or players should know we humans have the property of “gaming” systems towards maximum reward for minimum effort, hence it is likely that if not guarded well the process is used by the team to optimize towards getting the positive explicit feedback at the lowest cost possible. This is in game design terms sometimes called “bottom feeding” and needs to be guarded against by the rules of the game which define what to do to get which feedback. Another game design term which becomes relevant in explicit systems is the “mastery problem” which means that the better players will dominate the feedback structures of the game and win at a cost to the other players. Neither is good for the value of the product and digging into these details may be a topic for the future.

When explicit and implicit feedback systems fail to agree on if an iteration is successful or not there is often all kinds of problems surfacing. If these problems happen often the whole product is most likely going to fail on the market.

Implicit equals difficult

Making enjoyable and fun interactive games is primarily based on implicit iteration. Some fundamental infrastructure such as code and having a development team can be considered explicit but getting the experience to a point where the market value motivates the investment goes through tons and tons of implicit iteration.

Many of the implicit structures in a game are based on established art disciplines such as creating images, animations, music and storytelling. The product trusts itself to these domain experts effectively collaborating as a team. There are tools to rapidly iterate these components in isolation and it is part of the expertise of the experts to be masters of these tools.

The real problems surface when the implicit feedback relates to the sum of the parts of the system in the form of information design and game design. To make the game play experience have any value these also needs much iteration. Yet again there are skills and principles in the design expertise that help this process but they all fall under the implicit feedback umbrella and thereby cause communication problems.

If you look at the game industry at large you will find that there are a lot more games in certain categories, this makes the game industry appear relatively narrow. Many games resemble each other and are based on each other. The games that have been successful become so by playing on their strengths. The things that can be iterated relatively fast are strengths while things that iterate slow are weaknesses. Most are built on development techniques which upfront know where the pipeline is fast and where it is slow. This gets successful when each domain expert in the area of design owns the iterative loop for the implicit creative process. The things that have a cycle time measured in minutes or seconds reach the point of noticeable value, while anything which has a slower cycle time measured in hours or days should be considered as experience-wise valueless infrastructure. Those slower components of the experience will not be what make the user reliably have a gratifying experience. You might get lucky, but to get from lucky to certain the fundamental trick is to get the cycle time down to the range of seconds.

Techniques for effective iteration

Now that we have gone through a somewhat too big background of the problem we can take a look at a solution.

Based on concept, product vision or creative direction take a deep look at the idea which is going to result in user value then list the main components of the user experience.

• Take a through, cross-functional, look at each of these pieces. If there are more than a few dozen and in the hundred or so range you probably mistook the user experience for something too granular or technical. Or your team might not understand what the experience is supposed to be which might be a sign for future trouble.
  
  
• With the list in hand maintain the cross-functional approach and categorize the pieces in cycle time as estimates in seconds, minutes, hours, days and weeks. Include dependencies in these cycle times if needed, for example if it is relevant that some animations blend properly in the feedback system consider the pipeline for animation blending in the cycle time for animation feedback.
  

Now make three categories

1. Usability Infrastructure, everything with a cycle time of days and weeks in here
   
2. Shoehorn User experience, the things with a cycle time measured in hours
   
3. Reliable User Experience, seconds and minutes
   

With these three categories freshly composed remove everything except the third, fastest, category and see how well the things in this fast list match up against the idea about the product. It probably won’t match up very well. Don’t panic! This is quite common when making interactive entertainment and it is to your advantage. Most of your competitors will expose their weaknesses if you can figure out where their user experience is bogged down in the infrastructure category. You can also save your project by avoiding dependencies on the user experience which resides in this category. You will have some, that’s unavoidable but at this early point you have the ability to shed complexity to move the right things to the category of reliable user experience.

A general trick for getting the interactive parts of the system to the level of reliable user experience is to make all feedback elements dynamically changeable. For example if you have a storyline with a movie in it you will have the experience defined by the movie as a feedback element for an event triggered by the user. It is now important that the person who knows how to make movies can change the movie inside the experience with fast cycle time. If the cycle time is slow for movies don’t make them as parts of the system. Put the movie in the beginning instead as an intro which has no dependencies of interactive nature.

This can also be done for the verbs of the interaction circuit. You can’t expect to add a new verb with a fast cycle time so the verbs themselves go in the category of infrastructure. You can however adjust the subject the verb interact with, such as the width of a hole you need to jump over, how fast the opponent is running in a shooter. The verb can be changeable too such as the target area of the weapon or the height of the jump, expose suitable adjustable variables in an easy to change and preview data model. Beware tho, don’t mess up all the level design by changing the jump variables after production of the levels has started!

Use a recursive strategy to tune the experience. Make sure that the user is protected by efficiently tweakable variables to lean against all the way along the user journey. This will allow you headroom to react to feedback on the user experience at any point during the production. The larger percentage of what the user experience you get in a fast iterative loop the better. It’s better for the user to play a tiny game with a flawless emotionally satisfying user experience than a bigger game with a few uncomfortable moments. Note here that proper pacing is not to equal “uncomfortable” because the player will need both peaks and throughs on the emotional side to enjoy playing.

Exceptions

There are some exceptions when we look at the established game industry. Mostly they are rooted in incredibly huge budgets. Someone is spending so much money on the business that you can move the infrastructure category one notch up to the week level and just pour calendar time and man hours on making slow iterations instead. This also tends to interact with established development pipelines such as game engines for specific types of interactions. This approach is however on its last legs or already dead but it made some good stuff the last decade.

Another type of exception is the undiscovered market. You usually don’t predict these, but it happens that a new market is discovered by a product developed with a random approach towards creating value. Usually the resulting product is soon dethroned or bought by a big budget competitor. Looking at the music industry we see these random productions rather often but for each successful one we see thousands and maybe millions of failures. Reliable and consistently successful musicians iterate more or less consciously towards a safe result including all relevant disciplines in the process.

A lot of products also have no ambitions of providing a valuable experience. Instead they provide something else of value which may be relevant information or a service with pragmatic and explicit solutions to specific user needs, however games are only partially explicit and rarely pragmatic experiences.

phew, thats a long post...