Agile Failures

There has been a lot of almost mainstream media whining about how projects fail despite using an agile development methodology. Lots of blog posts about how agile cause trouble and how scrum fails in companies.

This post aims to start a journey which looks at my perspective of this trend. The journey begins with looking at the businesses which invest in projects and we’ll begin with separating these into two major factions.

Faction 1: Profitable business.

The profitable business has found a way to reproduce a profitable sale. Steve Blank uses the phrase “repeatable sales model”. My interpretation of this is that there is a potential market value which is reliably converted into money through the business activities. The repeatability causes a reliable equation to play out where you have a high chance of staying on top of positive revenue through conducting very low risk maneuvers until the equation changes when you have the option to adjust to maintain your profits.

Faction 2: Not yet profitable business

The not yet profitable business has bigger costs than earnings. This is quite simple and includes everything which is not yet profitable. In many cases you can see a profitable business include portions of this in the profitable equation, for example by making design improvements or product line extensions. The more common scenario is however attempts at creating a profitable business out of an idea.

Now that we have these two major factions we have a tool for understanding several ways to achieve failures which will make any methodology fail. Since everyone has learned that the typical waterfall project is going to fail by default it’s not that interesting to talk about, but we’ll try and stick to general failures which can be blamed on Agile.

To predict how you will fail you can start with asking yourself: where does the money that finances the project come from? If it is financed by an already profitable business you have a particular case. If it is financed by venture capital you have another and if it is financed by yourself, often as some type of hobby, you have a third. Among these three candidates you have one in particular which spells uncomfortable failure louder than the others. The least loud failure is when you are paying yourself. No one expects success from a hobby. If you are funded by venture capital you are expected to fail about 90% of the time. No one is surprised when investments fail to find a return. The loudest failure is when an already profitable business invests in unprofitable projects. These also seem to be the ones which plays the blame game and tries to blame agile methodologies for their failures.

If you look at an already profitable business from a capitalistic perspective you would think they should not invest in a new “not yet profitable business” and rather send that money to their investors. But for various reasons they generally don’t operate this way but attempts to reproduce profitability in new innovative manners. These become complicated matters.

A profitable business has generally spent a great lot of effort to achieve their profits. As a game designer knows you will be conditioned by expending effort at achieving a goal. So the profitable business has been conditioned by its own success. This conditioning is cultural within that company. This culture tends to have an agenda which prioritizes protecting the profitable business against risks. The practical implementation of this protection is commonly called bureaucracy.

Bureaucracy causes risky product experiments to have such a great cost that they have to be cancelled before they reach the market. It is better to send the investment immediately into the trash bin than to use it to cause a potential harm to an already profitable business. This also has the benefit of creating plenty of work for bureaucrats who thereby can be hired to handle the paperwork.

Eventually this causes polarization within the company where creative forces such as marketing eventually is urged to brute force the bureaucracy to revitalize a now aged brand. They can see their market share slowly being poached by competitors. At this point in time the culture has set really well. The creative forces rally enough of a budget to overcome the cost hurdle set by the bureaucrats and begin their project.

Since times change and you learn that using “traditional methodologies” will cause failure this project must begin to work with an Agile methodology, they usually try to work with scrum.

The root of the problem here is that scrum does not remove the cultural conditioning which practically has become embodied within the bureaucratic parts of the company. There will be plenty of reasons to interpret the new project as a risk to the profitable business and thereby justifying interference.

I don’t know if there is any particular solution to this problem but I do believe that a better understanding of this structure will reduce the costs of failure for any business which is investing in any type of projects. And this is just the beginning.

To propose a practical solution to this I will recommend making investments into new business areas at a healthy distance from currently profitable businesses. Mere proximity might increase the cost of failure.

From my experience with scrum I interpret some of the rules as engineered to refuse interference from bureaucracy. These rules cause a kind of cultural power struggle within an already profitable business when scrum is introduced. Power is usually owned by the bureaucrats and scrum becomes the loser and an embodied target for the blame game.

The first hard Learning of the Cats

This post begins a journey of unpredictable length which describes things the Lean and Mean Cat Team has learned along the way towards achieving significance within the games market. We will begin with the hardest lesson first, then possibly move on to easier ones later.

A hard lesson for Expert Cat: Mastering Perspective

This is a lesson about communication and understanding where one of the biggest risks with your product really lies. Expert Cat has gone through a life long journey to develop an understanding for this type of situation so it will not be a nice and easy lesson to describe. But we shall give it a try, Expert cat is whispering his s33kritz into my ear as I try to write them down.

This is what Expert Cat looks like as he is trying to convey this message for me to write down.

As you can see he is sitting next to little blue dots and lines which say Node, Link and System. This might not seem like something which is a learning, but it is a pillar of the most fundamental learning he wants to tell you about.

He says that in the beginning he thought about the world as if it was made out of Nodes. That means things which has properties in themselves. It is an effective way of observing things to treat them as nodes and it makes life for the Expert Cat easier on the surface. He can describe them and point at them and people who listen to him has a decent chance to understand what he is talking about.

Links are a bit trickier to understand. They are the connections between nodes which describe that the nodes are interacting with each other somehow. This is a lot harder for Expert Cat to at first understand, even harder to make use of and almost hopeless to describe. How do you describe that the car and the road are connected through a link, you cant see it? Maybe the tires are a link, maybe it is the driver which wants to travel who is the link or maybe it is the engine which make the tires turn with force. Expert Cat almost goes crazy trying to describe this to other cats.

Systems are a bit easier to describe but harder to create. Expert Cat can lump a car, a road, an engine and a driver into a clump of things which together are a system. When he describes the whole system to other cats they intuitively understand that they could make some use out of the ability to drive along roads to places at speed and derive some type of value out of it.

When Expert Cat was young

As a youngster Expert Cat wanted to do great things for the other cats in the world and began working on creating things. Since back then he was only really able to understand the concept of Nodes he went about and made things. Sometimes his things caught a litle bit of interest among the other cats but nothing really worked. What is wrong with my things? - he though a bit sadly.

No one really wants things it seems. Well, actually there are situations where thing are wanted but those are special cases which Expert Cat will tell us about later in this story.

Expert Cat is trained by Master Cat

After making a series of failed things Expert Cat went out looking for a Master Cat who would become a great teacher. Master Cat began training Expert Cat at using different perspectives of looking at the world and introduced Expert Cat to the idea of systems. The things Expert Cat had been making were failing because they were not part of a system which the other cats were part of. The things which had partial success had by a chance become adopted by other cats as pieces of their personal systems which are hidden in the mystery of cat souls.

Master Cat spent years and years and mentored Expert Cat to learn how to observe systems. About ten years after meeting with Master Cat our Expert Cat emerged as a fully trained systems observer.

A trained Expert Cat

To make things successfully our Expert Cat learned to create links. Sometimes links demand the creation of Nodes, and sometimes links demand other things. The core understanding of Expert Cat was to see that things without links are dead weight, waste and useless creations in general.

The next problem for Expert Cat is to describe what a link is. This is a fundamentally tricky problem because they are always different in detail but similar in the abstract. Since the details always differ we can make one detailed example and try to move onwards from there.

Details of a Link

Lets us begin describing a link. We will describe the link of Eating. We can look at something and notice that it is getting Eaten. A good start. Expert Cat wants to create the link of Eating.

To create the link of Eating our Expert Cat needs to become dependant on two nodes. One node is a Cat, who Expert Cat thinks will be interested in eating. The second node is a Fish which Expert Cat thinks has suitable fit for the occasion.

This is a deep understanding of the Expert Cat. The value in the system of "Cat eats Fish" lies within the interactive process which is commonly called eating. The value of the fish which is not eaten can be considered "inventory" which according to lean is a risk. From now on when Expert Cat make things he defines his thing as the interaction between the nodes in the system. The ten years of training which Expert Cat has contained a few weeks of trying to understand the writings of Guru Cat Chris Crawford who says the trick to creating games lies with defining the design from the verbs in the design. Expert Cat believes this argument says the same thing with different words. Expert Cat believe verbs are quite useful when defining links.

Creating systems through Links, Nodes or Systems

While training with Master Cat our Expert Cat learned a few other things about systems. Different types of systems require different approaches towards their creation. To avoid needing to write far too much text we will narrow the problem down to two extremes.

The two extremes are described by a Guru Cat called Steve Blank. He uses a model which is a useful framework to dig at this problem. He uses the words "Customer Risk" on one end and "Technology Risk" on the other. Then there are things which has both...

On one extreme end we have a system where the risk is the functionality within the nodes. At this extreme end we find things such as a Fusion Reactor. If you can get the Fusion Reactor to work as it should then you have your value right there. Infinite power for a low cost. We can define this value as its node. If it works it will also have value for the other cats in the world.

On the other extreme we have the taste of Fish. Does the taste of the fish which Expert Cat is creating have any value on the market? The risk here is not that the Fish has a taste, or even if the fish exist, but rather if the taste causes the cat to eat. How does the cat eat the fish? Oh noez! This is so complicated Expert Cat is almost trembling at the notion of describing the situation.

To describe the problematic description as hypothesis

We need to use another lens to look at the problem to get a perspective which has a better chance of succeeding. To create the correct taste when cooking the fish our Expert Cat creates a hypothesis that says: "The fish tastes good enough to make another cat eat it." This puts the link to the test!

Now our Expert Cat can cook the fish with an inventive recipe and try to make another cat eat it. He can modulate his test and see if he can make another cat pay for the cooked fish, and see which of all the cats are willing to eat it or not eat it.

By successfully describing his fish recipe as the properties of eating he has obtained the all powerful insight into how changes to the recipe influence the value of the system. Wewt!

Ok, so what does this mean in practice?

Our Expert Cat understands how to differentiate customer risk versus technology risk. When Expert Cat joins a Mean and Lean Cat Team he knows that his team will be more or less suitable to handle the different types of risks. A team full of scientist and engineering wizard cats is likely to be suitable for taking on technical risk, which a team full of marketing Fixers and emotional Artist cats is suitable for taking on customer risk. A team which has a healthy balance of both is likely to take on challenges which contain both types of risk.

Kinds of risks for game creating cat teams

This is a pretty big system which practically is rather simple to understand.

Games with some Technical Risk:

• MMORPG's
• Next Gen AAA Console Games

Games with Customer Risk (Everything else):

• Flash Games
• Web based community Games
• Facebook Games
• Marketing Campaign Games
• Indie Console Games
• Indie PC Games
• Oldskool Games
• etc

Note that no games that the Expert Cat think is a game has purely Technical Risk. Defining a game Hypothesis as Nodes is thereby almost a guaranteed failure. Alright, Expert Cat jumped over a long series of arguments to reach that conclusion, but his time is running out and he wants to get out of here rather than keep on telling us the details of this story.

Farewell Expert cat, enjoy the sunshine and the Fish cooking! *waves*

Fixer Cat comes in to finish the lesson and says:

- Lean and Mean Cat Teams that use Agile or Scrum to create games need to learn how to define their Backlogs as links. If they fail to do this and instead base their work on creating nodes they become statistical death marches. However, if you create an MMORPG or a Next Gen AAA Console Title you can use a bit of both nodes and links to deal with technical risk.

Efficiency by Forced Iteration

This is a technique I apply on almost everything I do which has any expressive ambition. I'll call it Forced Iteration, I have not previously though of it as anything which has a name. After power-reading the stuff that Eric Reis and his crew is doing I realize it lies dangerously close to the idea of Continuous Deployment.

I have been messing around with the creation of usually unambitious artistic artifacts such as pieces of music, texts or even little experimental games. What I have realized over about twenty years of doing this is that the most effective way to improve the result is to publish it and make it available to some kind of living and breathing real audience. Even if it is only one other person in the audience this is vastly better than avoiding the publishing step.

What I do is that I first fiddle with the piece, this blog post is an example of such a piece. Then after I start losing tempo on the production of it I'll publish it. This causes me to feel a slight level of panic that I just published something which is total rubbish. The understanding that the rubbish is already out there and potentially being consumed by an end user triggers a reflex which makes me fire up a burst of energy which I spend by seeking out the worst flaws and I fix them just a few minutes after the publishing. Whew... the feeling of fixing the broken bits before anyone really saw them is actually quite good!

Now after the worst bits are fixed I will have a bit of breathing room which I use to judge the resulting piece of work more closely. This tends to spit out a new series of micro-iterations which hammer away at the lesser errors that I can see.

I am not a good writer, and I know that my English skills are quite lacking compared to what you'll see from a more engaging writer. This knowledge sets my level of ambition to be relatively low when it comes to making this blog great. Maybe someone who is smack dab in my target audience finds it comprehensible enough. Well, anyway...

The act of forcefully publishing the unfinished piece of work effectively speeds up the process of going from the idea of doing something until it meets the standards set by my level of ambition. I believe that I find this behaviour to be natural from having somewhat of a background as a live performing musician, even if it was a long time ago now. The live performing musician is in a constant state of streaming art directly to the audience. The reaction from the audience is the fuel which drives the engine of creation. And the idea that my audience will see the first iteration fuels the engine that creates the second, and so on it goes.

By using Forced Iteration this way the volume of fuel is maximized, at least for my type of engine. I know there are great artists who achieve greatness by going the other way around, but I have no idea of how they really do it, or how fast they really work.

Feedback pipelines

Game production use the word ”pipeline” for the technology and behaviors needed for adding data to the end product. This is a great and useful component to the iterative process. There is a system which can be optimized for speeding up the time needed to change the data from one state to another. I have no real idea of how much of the budget for a large scale production goes into developing this pipeline. I guess it is significant and well motivated.

Now to the point of the argument, the investment made in the feedback pipeline is most likely much smaller for most game projects. This is not well motivated and I believe it surfaces as a tremendous cost, for successful project it means your staff needs to be incredibly competent and able to manage implicit iteration without excessive waste. For failed projects it is the one major cause for failure.

I knowingly use very strong words here because I want the notion challenged so I learn something useful from it.

Dangers of iterative production techniques, part 1 of n

As I mentioned in a previous post http://gamesartdesign.blogspot.com/2009/03/iterating-towards-value.html there are, from my personal perspective, several ways to cut an iterative process. This post is about general risks with any of these, and likely other, cuts.

Risk 1: Feedback conditioning

This is a risk which occurs whenever you keep getting similar feedback through a series of iterations. The shape it takes is much like what happens to us humans when we get accustomed to living close to a paper manufacturing plant which stinks of sulfurous chemicals. Our brains get used to the initially nasty experience and we get used to it. A game development team which gets familiar with some feedback and determine that it is part of reality will make a worse product than they could.

Risk 2: Reliance on implicit feedback loops

When the measurement is implicit as with the case for how the subjective components of the product are experienced it is quite likely that the team will fail to understand when they reach the top of the value s-curve and keep on investing in things which are done. For us who spend our time with the modern art form of game design this is horribly difficult to communicate. I also believe it is more difficult for the other art forms in game development as well than what it is for the older more established media. How do you know when you should make a new animation rather than keep on iterating the current one?

Risk 3: Lack of direction

This is the most often mentioned risk which comes from a combination of lacking vision or the communication of a vision which is hard for the team and business to understand. As a consequence of a poorly communicated vision the team lacks a foundation for understanding even explicit feedback loops and will progress erratically.

Risk 4: Going in circles

When the team has too little room to maneuver they rick failure to understand how to change the next experiment in the iterative process to improve the feedback state of a production. This comes appear to often come as a consequence to another failure which cascades down as an unmanageable cost of an effective cycle time. If changes to the product reach a certain volume the next change will be driven by risk aversion, risk elimination is likely to continuously surface with the same solution over and over.


Do you think this list should be extended, feel welcome to add your own risks and I’ll try bake them into part 2. ^^

No great result is really random

I was fiddling with the mix and arrangement of the latest Spaceforce song and realized that I was messing with something that one of teachers said in at Musicians Institute. It wenst something like this:

- Parts of the production might appear to be random, such as the slight out of tune guitars on a nirvana album, but in truth it never is random. It is well through out and set to be just that way conciously.

There is a lot of wisdom in this and it does apply to a whole lot more than nirvana songs. It happens to apply to Spaceforce as well, when the production is young it has a lot of randomness in it coming from the first iterations which define the structure of the work. But the process of finishing anything is partially about removing everything random and replacing it with the concious refinement of the data.

This also applies to game design and storytelling, maybe you start with randomly generated pieces, but you should be well aware of why you keep anything which was randomly created. It will rarely do more good than harm to the end result.

Feature Creep or Vision work?

Ok so now I have made this rather horrible video with the robot band. This had a series of interesting consequences. One is that I now feel the urge to refine the music so it shows off the individual talents of the robots better. If I go for this I'll have to make some major rearrangements so they all play solos, Doppz is missing his synt-bass solo, which is a quite rare beatie anyway. And Some cool synchronized solos would be interesting between the guitar robot and the keyboard robot.

After the rearrangement would be done which is an effort of several evenings of work at least I will have to make a new video which shows these new scenes.

Since I have no practical limit on the ambition of the production other than doing it for the fun of it I will categorise this type of work as product iterations. The music would be iterated farther to fit a more well defined vision.

If I did have a set of measurable ambitions and, not that it would happen but if, the work already qualified as sufficient I would call this type of further iteration as overly ambitious or feature creep.

Ideas

If design is a process which bring out the value of an idea, then what is an idea to begin with?

Wiktionary offers some definitions, among them this one: "The transcript, image, or picture of a visible object that is formed by the mind; also, a similar image of any object whatever, whether sensible or spiritual."

Which appears a bit cumbersome. I will yet again create my own definition to serve my own purpose.

- An idea is a collection of information with undetermined value.

This is a bit rough, but its enough to hook it up together with my definition of design. What yo can do with an idea is what is interesting. There are two main techniques for determining the value of an idea. You refine the idea either through creating artifacts around it which can transfer the idea to more brains, or you construct something that is the embodiment of the idea to see what happens. These two techniques are quite different.

Quality Confusion

I find the concept of "quality" to be among the more abused components of game developmen and maybe even for artistic expression in general. The problem is that different people consider different things to be quality. The main stances towards quality which I am familiar with are something like this.

Engineering: Quality is when it works technically elegant.

Business: Quality is when it has obvious sellable features.

Art: Quality is making an impression.

Perfection: Quality is meeting established goals.

The more common among ordinary people is the perspective of business quality. People talk about the obvious sellable features when they talk about a product. They often and faulty equate a small number of features as a poor quality product. This flawed reasoning is the core motivator for me writing this post.

To start correcting the problem we have to change a little bit of what describes business quality. We have to replace the word “features” with “advantages”. After we have made this correction I will easily argue the stance that to reach high quality you have to nail all four of these perspectives.

Next problem is that the reasoning above really does not talk about quality the way I consider quality. I would call the above statements for “needs” which are base requirements for reaching ROI with anything. If losing money is desired then compromise on one or more of the perspectives.

Real quality is meeting all of the product needs without the existence of defects.

Quality is not a creation or induced by adding things, rather a result of a well balanced methodology. An arch enemy of quality is the misstake of associating ambition level with quality. This leads to the common problem of people claiming that small simple products are having “low quality” which is a fundamentally flawed argument. Those simple products are, when successful, rather reducing ambition for increasing quality.

Goals

Straight from wikipedia: Goal Setting involves establishing specific, measurable and time targeted objectives.

This is a point of setting goals which which appear to be failed by teams that fail at agile development. A goal for a devlopment or I guess any business needs to be formulated in a manner which describes the state of things upon completion of the goal.

Following this principle is sometimes quite hard. Lots of the flak tossed at scrum lately can be aimed at the lack of useful goals.

Setting useful goals takes a whole lot of time and when a whole team is working on the problem together people often start showing signs of stress symptoms from feeling of wasting time. However having bad goals is worse than wasting time.

The usefulness of bugs

This is an interesting topic we had a little look at recently. In my world a bug is a subgroup of something that with a wider definition would be called a defect. Googling for a definition of the term defect gives a useful result quickly which says this:

- An imperfection that causes inadequacy or failure.

This a good place to start.

In my experience the vast majority of defects associated with professional game development or entertainment in general belong to the category of inadequacy of the user experience. I would call it ‘inadequate emotional attachment’. It happens that the experience of the player is significantly damaged by the existence of a good old fashioned bug. A program failure or crash will generally not be helpful towards the emotional state of the user but these problems are thankfully not all that common anymore and various measures are in place to remove them from software products.

A much more common problem is the lack of something intangible which makes playing the game feel just right. There are tricks or theories you can use to make better games but they are not generally accepted yet so they are hard to describe without getting into very lengthy descriptions of things. So instead of getting stuck in a theoretical black hole I can use a metaphor from the art of making music.

Most of the music that reach you will have been analyzed through a rigorous process of defect detection. During its creation the music you hear will have gone through lots of iterations, thousands to millions of tweaks and changes have been made to create the result that is emotionally attaching enough to reach your sensory system. These tweaks have had all kinds of reasons for being done. Different composers or music creating organizations have different methods towards doing things but in general we can take a reductionist look at it.

Most of the work done when creating music is inadequate on the idea level. The idea behind the work is not worth the trouble of refinement. The musician detects this quickly and iterates the idea. Eventually the idea is iterated to a point where it reaches something I will call experiential harmony. This is based on all kinds of details, such as the sounds of the instruments, the arrangement, the rhythm, melody and so on. The result may not yet be anything complete but it has reached a “proof of concept” level. To reach this point the musician has gone through defect elimination of several types. Different types of defects are eliminated based on different techniques.

Finding a good sound is the process of selecting the right instruments for the various parts of the arrangement and placing them in the sound image where they fit. This is something which typically starts with a great lot of simple defects and ends with a system of sounds which interact experientially productive.

Finding the notes played by each instrument is another type of defect prone creative process. Some parts come naturally and some are iterated heavily. It is easy to get to a state where the music is broken due to bad notes. These can be detected and changed from bad to good relatively easily.

The difference between a defect note and a defect sound is of some relevance to the topic of this article. The classical, and in modern times rare, bug is defined as a defect note. It is an explicit error in the construction which can be removed or changed to improve the emotional attachment of the product.

The more common defect in a user experience is a complex problem which arises from the interactions in the whole system. We hear player say various things to state the existence of these defects.

The game is too hard, the game is too easy, boring, slow, stressful, not interesting and so on. You might think that “he wrote boring and not interesting as different things, how stupid.” But from a game designers perspective these are different things, boring is about patterns and the interesting is about context. When you use a players feedback to iterate towards value you are well served to realize that most of the feedback you get will be describing symptoms of complex problems.

A complex problem is in this context a defect which arises from the interactions in the system. Classical examples are unrestrained positive feedback loops. These are so common that we even got a specific label and fix for them which is “restrain feedback loop” by changing how the system interact or process data in one or more points. When the player says the game is too slow it means that the player is waiting for relevant information for too long, this is matter of pacing and tuning which has different complex structure in different parts of a game application.

What makes this fundamentally interesting is that the process of creating emotional attachment is not reactive to user feedback. User feedback is useful for providing the process with relevant information at some points, especially useful for detecting complex problems in the system. The solutions to problems need to come from the system creators.

If the project which is creating this emotional attachment classify defects which are complex problems as bugs you are likely to have a much worse problem than bugs unless you already have reached primary business objectives for the whole product which means it will be ok to have a total failure anyway since everyone is happy with the already accomplished result.

From a musicians perspective it is intolerable to accept bugs. The existence of bugs in music means that you prefer making more of the bad music than making valuable music. You can use this strategy during the iterative process and for example accept bugs in the melody while refining the sound but you will never consider a piece of music to be “done” until it is free from defects and thereby flawless. This means that bugs which are reported from end users should return to the product development as the highest priority backlog, if they return as a randomly prioritized task list you are most likely far behind schedule towards reaching business goals.

Practically we can probably look at several strategies which deal with these problems. I myself believe the development team needs the mandate to stop defects from reaching the end user in the first place as they are the least likely to be having other goals than making a great product. If you reach a state where it is more important to release a defect product than it is to create a flawless product you are probably in trouble already.

Maintaining multiple options

This is a Lean productivity trick. Mary Poppendieck describes it well in this awesome movie; http://video.google.com/videoplay?docid=-5105910452864283694 - you have to watch it if you care about making good things.

When looking at game development this is done on a regular basis in some of the pipelines, most obvious when drawing art images through how the layer system operates in photoshop but also in how programmers write code with branches and classes, how storywriters use word processing software to keep dialogue and story flexible and how musicians create the music score with their software such as ProTools maintaining all the versions of the songs on the HD while edits are happening to a subset of files.

Now the question becomes how to apply this to the combined user experience which results from bringing all these pieces together. As the network theorists say something along the lines of: the whole system is the combination of the nodes and the interaction between the nodes. The game which becomes the user experience is the interaction between the nodes, the code is one node, the music is another node, the image art is a node, the interface is the tool which the user has available to navigate the network which in a good game becomes a modular agent in the network.

The game itself is a system which measure the skills of the user, these skills are deployed in context by the user to navigate the whole network. Any piece of the network which is outside the users path is roughly what lean would call "waste".

For the whole game system to maintain multiple options we'll need to expose the interaction between the nodes as readily changeable and generally transparent, human readable, data with some level of backwards compatibility. This is an utopian model, but it is not fully a pipe dream. In a not very far future we will know much more of the principles of game design. From my perspective these principles started with Chris Crawford and his model of interactivity, they have moved forward a lot with things like Raph Kosters theory of fun for game design and Daniel Cook's model of skill chains. There is a lot more stuff in this mix worth a mention, but maybe we'll go there another time.

With a better understanding of the principles for game design we will be able to identify differrent types of interactivity and thereby define the interactions between the nodes in the system with an explicit format. Music has several kinds of explicit data formats to present the patterns of sound on on various levels of abstractions, waveforms, instruments, overtones, harmonies, rythms, notes, tabulature and so on. Several of these have alternative modes, you can define a "tone" through various methods either as an "instrument (violin) playing a note (A) for an proportion of a bar timeunit" or a "fundamental frequency (440Hz) with 9 overtones (i think thats the violin if I remember it right) playing for a time in seconds". You can keep on going and define the waverofm as a function based on trigonometry if you prefer but I'll skip that example. I think the point made it through without. Note that the actual psychoacoustic representation of the violin is a lot more complex than these primitive models but on the theory side of things you go there if you want to, which is about never unless you are a bit "special". For the synthesizer manufacturer its all the way down through the trigonometry that wins the day with modulation, also trigonometry, controllers and stuff.

Game design theory is today at the stage where everyone communicate with a randomly selected spray from the whole arsenal of terminology, or rather, different people use different terminology to say the same thing and they generally don't know that there are other terminologies which define the same interactions within the system. They can barely even communicate with each other without first synchronizing the language which is a hard problem. Some very experienced designers will have a gut feeling for a lot of the principles, I myself find that I have a better grasp of some models when analysing a game play experience and some others when writing a game concept. I must admit that I also don't know the whole system, I don't think anyone does. There is no "first dan black belt" of game design and not even Sid Meyer, Will Wright, Chris Crawford or anyone else would have it if there was one.

However we can start in the pragmatic side of the equation and create a data model which best expose the whole system as nodes of data and interaction which is the transportation of other data through the system at the request of the user through the interface agent.

Now you could claim that this is already done with the programming langugage itself, and to some degree that is true. However the game systems which people play today are requiring domain experts which are not always competent programmers. (Programmers don't tend to spend most of their time doing user research as example, thats sub-otpimal to most production efforts.) Also maintaining multiple options through having several version of the whole code base is unlikely to be much help.

So what kind of data can define the interaction between nodes in the system?

If anyone wants to take a stab at it go for it, or bring up examples from various game engines. ^^ (Or I will later on and then it gets texty as usual.)

Iterating towards value

It is commonly accepted that the user value in a game and most things has a statistical probability to increase with the number of iterations it has gone through. This goes for things such as fried meatballs, your DNA, a guitar solo, a sound and so on.

This takes on different forms in different areas of life, since I am interested in writing about game development I will focus this story on some of the forms involved with making games. The process relates to cooking and music production, but since this is a big topic I’ll leave most such fun references out.

The relationship between iterations and value is often plotted as a neat little graph.

Note the S-curve type of deal going on. That says something about the existence of “no value” iterations at the beginning of the process. You got to get to some point before you have anything, then eventually it is finished and stops getting any better. To get anywhere interesting with this topic we have to take a look at the definitions of the graph. What is the value you get as a function of # iterations?

What one iteration is

One iteration is the work done from the point where there is an idea about doing something to the point where what was done has resulted in useful feedback. This is a fractal concept and it has several types of zoom levels and dimensions. The one zoom level most commonly associated with game development is on the product level. One iteration on that level have its value measured by feedback originating from the potentially shippable product through various test procedures such as user testing, technical testing and getting accepted by the product owner. These are examples of something I will call explicit feedback.

Explicit feedback gives a quantifiable measure of value which is obviously visible to the involved parties. These summarize as “done” when everything pass defined criteria often called a “definition of done”. Generally if feedback from the work shows an insufficient or negative value increase the iteration has failed.

Another zoom level for one iteration with an explicit feedback loop is the creation of code data, commonly called programming. If the code is broken it will fail to compile and all kinds of feedback is generally built into the code pipeline to prevent broken code from entering the system. The programmer makes a new iteration of the code to improve the value of the code without first shipping the code to the end user to test if it has any value. These iterations are fast, some broken code can be iterated on in a matter of seconds. This time unit defined by the time it takes to iterate is sometimes called “cycle time”, which I will also use a fractal concept. I will call this comparably low zoom level of iterating for “creative process”, the programmer creatively create the code.

Another type of creative process is the creation of images. The artist uses an image creation tool such as photoshop to create image art. This is also a fast process. The artist will generate art data on a second to second basis, undo things that go wrong and create new pixels with a stylus without thinking of it as iteration. The artist also maintains and switches between multiple options throughout the production of one image. The feedback which closes one iteration in the creation of a pixel of art data is internalized by the artist. I will call this implicit feedback.

Implicit feedback is not obviously visible to anyone. It is to the observer a subjective measurement and often leads to all kinds of emotional distress for the involved parties when it comes time to analyze the value of the output of the creative process based on implicit feedback structures. To the professional artist it is not fully subjective as some principles of the arts guide the creative process but in large it’s a relative mess compared to the efficient explicit structure of code creation. It also happens that different artists use different principles for this process which often cause communication problems, but mostly everyone has learned to live with this process quite productively.

Now that we have a model to differentiate iterations between ones based on explicit and implicit feedback it becomes much easier to understand what things you can put in through the value creation process.

A guitar solo runs its first several thousand iterations as an implicit process before it reaches the point where it gets explicit feedback in the form of praise or complaints from band members, audience or other people involved with the music production. Improvisational jazz is an example of the solo going directly from implicit practice to the end user without explicit iteration, even driven by the goal to change between each gig! The details of the solo are guided by a lot of musical theory internalized by the jazz guitarist.

The development team for a product increase its value based on explicit feedback from the producer or the team itself, as well as the implicit feedback within each team member and their feelings towards doing good or bad work. Scrum is an example of a process which forces some of this feedback to be explicit. This is good in the sense of providing direction but as all game designers or players should know we humans have the property of “gaming” systems towards maximum reward for minimum effort, hence it is likely that if not guarded well the process is used by the team to optimize towards getting the positive explicit feedback at the lowest cost possible. This is in game design terms sometimes called “bottom feeding” and needs to be guarded against by the rules of the game which define what to do to get which feedback. Another game design term which becomes relevant in explicit systems is the “mastery problem” which means that the better players will dominate the feedback structures of the game and win at a cost to the other players. Neither is good for the value of the product and digging into these details may be a topic for the future.

When explicit and implicit feedback systems fail to agree on if an iteration is successful or not there is often all kinds of problems surfacing. If these problems happen often the whole product is most likely going to fail on the market.

Implicit equals difficult

Making enjoyable and fun interactive games is primarily based on implicit iteration. Some fundamental infrastructure such as code and having a development team can be considered explicit but getting the experience to a point where the market value motivates the investment goes through tons and tons of implicit iteration.

Many of the implicit structures in a game are based on established art disciplines such as creating images, animations, music and storytelling. The product trusts itself to these domain experts effectively collaborating as a team. There are tools to rapidly iterate these components in isolation and it is part of the expertise of the experts to be masters of these tools.

The real problems surface when the implicit feedback relates to the sum of the parts of the system in the form of information design and game design. To make the game play experience have any value these also needs much iteration. Yet again there are skills and principles in the design expertise that help this process but they all fall under the implicit feedback umbrella and thereby cause communication problems.

If you look at the game industry at large you will find that there are a lot more games in certain categories, this makes the game industry appear relatively narrow. Many games resemble each other and are based on each other. The games that have been successful become so by playing on their strengths. The things that can be iterated relatively fast are strengths while things that iterate slow are weaknesses. Most are built on development techniques which upfront know where the pipeline is fast and where it is slow. This gets successful when each domain expert in the area of design owns the iterative loop for the implicit creative process. The things that have a cycle time measured in minutes or seconds reach the point of noticeable value, while anything which has a slower cycle time measured in hours or days should be considered as experience-wise valueless infrastructure. Those slower components of the experience will not be what make the user reliably have a gratifying experience. You might get lucky, but to get from lucky to certain the fundamental trick is to get the cycle time down to the range of seconds.

Techniques for effective iteration

Now that we have gone through a somewhat too big background of the problem we can take a look at a solution.

Based on concept, product vision or creative direction take a deep look at the idea which is going to result in user value then list the main components of the user experience.

• Take a through, cross-functional, look at each of these pieces. If there are more than a few dozen and in the hundred or so range you probably mistook the user experience for something too granular or technical. Or your team might not understand what the experience is supposed to be which might be a sign for future trouble.
  
  
• With the list in hand maintain the cross-functional approach and categorize the pieces in cycle time as estimates in seconds, minutes, hours, days and weeks. Include dependencies in these cycle times if needed, for example if it is relevant that some animations blend properly in the feedback system consider the pipeline for animation blending in the cycle time for animation feedback.
  

Now make three categories

1. Usability Infrastructure, everything with a cycle time of days and weeks in here
   
2. Shoehorn User experience, the things with a cycle time measured in hours
   
3. Reliable User Experience, seconds and minutes
   

With these three categories freshly composed remove everything except the third, fastest, category and see how well the things in this fast list match up against the idea about the product. It probably won’t match up very well. Don’t panic! This is quite common when making interactive entertainment and it is to your advantage. Most of your competitors will expose their weaknesses if you can figure out where their user experience is bogged down in the infrastructure category. You can also save your project by avoiding dependencies on the user experience which resides in this category. You will have some, that’s unavoidable but at this early point you have the ability to shed complexity to move the right things to the category of reliable user experience.

A general trick for getting the interactive parts of the system to the level of reliable user experience is to make all feedback elements dynamically changeable. For example if you have a storyline with a movie in it you will have the experience defined by the movie as a feedback element for an event triggered by the user. It is now important that the person who knows how to make movies can change the movie inside the experience with fast cycle time. If the cycle time is slow for movies don’t make them as parts of the system. Put the movie in the beginning instead as an intro which has no dependencies of interactive nature.

This can also be done for the verbs of the interaction circuit. You can’t expect to add a new verb with a fast cycle time so the verbs themselves go in the category of infrastructure. You can however adjust the subject the verb interact with, such as the width of a hole you need to jump over, how fast the opponent is running in a shooter. The verb can be changeable too such as the target area of the weapon or the height of the jump, expose suitable adjustable variables in an easy to change and preview data model. Beware tho, don’t mess up all the level design by changing the jump variables after production of the levels has started!

Use a recursive strategy to tune the experience. Make sure that the user is protected by efficiently tweakable variables to lean against all the way along the user journey. This will allow you headroom to react to feedback on the user experience at any point during the production. The larger percentage of what the user experience you get in a fast iterative loop the better. It’s better for the user to play a tiny game with a flawless emotionally satisfying user experience than a bigger game with a few uncomfortable moments. Note here that proper pacing is not to equal “uncomfortable” because the player will need both peaks and throughs on the emotional side to enjoy playing.

Exceptions

There are some exceptions when we look at the established game industry. Mostly they are rooted in incredibly huge budgets. Someone is spending so much money on the business that you can move the infrastructure category one notch up to the week level and just pour calendar time and man hours on making slow iterations instead. This also tends to interact with established development pipelines such as game engines for specific types of interactions. This approach is however on its last legs or already dead but it made some good stuff the last decade.

Another type of exception is the undiscovered market. You usually don’t predict these, but it happens that a new market is discovered by a product developed with a random approach towards creating value. Usually the resulting product is soon dethroned or bought by a big budget competitor. Looking at the music industry we see these random productions rather often but for each successful one we see thousands and maybe millions of failures. Reliable and consistently successful musicians iterate more or less consciously towards a safe result including all relevant disciplines in the process.

A lot of products also have no ambitions of providing a valuable experience. Instead they provide something else of value which may be relevant information or a service with pragmatic and explicit solutions to specific user needs, however games are only partially explicit and rarely pragmatic experiences.

phew, thats a long post...

Pull Schedule - everything?

So its commonly accepted that pull scheduling is efficient, quite a lot more than push scheduling. If we look at it from the information network perspective as I brought up here http://gamesartdesign.blogspot.com/2009/02/productivity-as-information-network.html previously it appears as if there is a general trick to it.

User Value and its derivates pulls development from the team. The derivates of user value is in this case stuff like product vision, strategy, audience, test feedback and such.The whole team process the User Value requests as a unit. Each team member is always ready to have information pulled on request from the team.

To avoid getting interference from management the team is likely to benefit from keeping track of progress towards the User Value derivates. Post it on a wall and update continously so this information dosnt become a burden to deliver.

Productivity as information network

There are some great descriptions of how to achieve productivity around the web. A culled list of things which the more enlightened models seem to agree on looks like this:

1. Measure productivity in market value change, often derived from user value.
   
2. Maintain a special interest in anything you did which has negative market value so you can remove it.
   
3. Avoid anything which reduces your efficiency towards creating value in the future.
   

This list can be described with other words:

1. Test the production output before attaching it to the product.
   
2. Handle all kinds of defects consciously, from bugs to bad ideas and everything in between. Lean says; stop the line quality.
   
3. Never sacrifice product integrity to increase productivity.
   

To achieve positive productivity and develop market value there are several techniques in the production toolbox which probably are more or less familiar. There are lots of bad or outdated techniques which have been tested and found to be bad such as working over time, measuring productivity in code or features and developing towards a rigid specification. These bad ideas are not that interesting, but it happens that even a modern team fall in their traps.

The interesting things happen on the good idea side.

• Team size limits
  

Team size around 7 members. If this is because our brains are good at chunking the function of each team member, because biological networks get saturated with communication if nodes recieve information from more than 7 sources, if it is for some other reason such as creating a temporary family or that this number is about where you can find all relevant disciplines for creating user value I don’t know. The good thing is that it (the reasons) does not really matter.

• Responsibility distribution
  

The team, not a boss, determines how to create value within given constraints. Scrum does this partially by defining goals which the team convert into tasks aimed at realizing the value of a time box. Lean appears to approach this through something I recall as “on demand production” via pull scheduling the development when the market demands it.

• Understanding of User Needs
  

The whole team understands what the product aims to be and how it aims to meet a demand. This often shows up as a transparent product vision and the methods used for testing user value where the whole team is involved with understanding the needs and problems of real users.

The whole business as a biological network

I will now make a very poorly grounded statement:
- The whole business is a biological network with nodes made out of humans.

This network exchange information about what needs to be done to satisfy a market with the goal to make a profit in money or sometimes a less direct unit such as “cred” or brand awareness.

Biological networks are relatively well documented on a scientific level. I am not well informed on the subject but there are some properties of biological networks which even I understand and which may be interesting when optimizing information flow towards meeting market value.

• Information degrades every time it passes a node.
  

The network specializes towards dealing with certain types of information. Common specializations are to improve short range communication at the cost of gimping long range communication. We cannot, for example, well understand the emotional stated a source node which is at some distance and emotions carry a lot of information.

• The cost of relaying information across the network affects the amount of information relayed.
  

If I understand this correctly we humans have learned that communicating information across several nodes is close to impossible. We end up either limiting the message to something insignificantly small and useless or we end up saturating the network with protocols which becomes needed for the information to survive its journey through the network. Btw, such measures does not work very often. When implemented they provide a structure for social mobility and people create links when communication is needed.

• Node failure is random
  

A node which gets swarmed with information will fail to operate properly. This causes some hesitation towards accepting information among hub nodes. The practical effect is the same as above, the network limit itself to avoid non-random node failure.

It looks like the good ideas of productivity are natural adaptations towards optimizing the biological network.

• The product vision is integrated with the team.
• Defects are seen by the whole team.
  
• The actual user value is measured directly by every team member.
  
• The decisions on how to react on feedback are determined by the team.
  

Relevant information for the team is at most 1 node distant.

• Constraints are negotiated between the team and the client.
  
• End users deliver feedback directly to the team.
  

The interesting part here is that I don’t see much other organizational structures having taken on this approach towards optimization. They probably have a slower darwinistic evolution cycle, while software development productions often mutate several times per year. Lots of large organizations appear to try strong arming their biological networks to work as digital networks and actually go ahead with implementing communication protocol rather than reorganize the network to fit naturally with our human senses. (Hello email rules and conventions.)

The infrastructure problem

If your user value is heavily dependent on large infrastructure, such as a nuclear plant and a power grid I would not guess if this type of optimization scales up. It would appear reasonable to think that you can scale the production of such a large system efficiently by optimizing the information network to fit the properties of the biological network which build these kinds of things. You will probably get a few layers of abstraction between the development team and the actual user value, defects and future velocity.

This little post should have had some nice images with it. Maybe I'll update it someday if it still makes sence in the future.