Agile Failures

There has been a lot of almost mainstream media whining about how projects fail despite using an agile development methodology. Lots of blog posts about how agile cause trouble and how scrum fails in companies.

This post aims to start a journey which looks at my perspective of this trend. The journey begins with looking at the businesses which invest in projects and we’ll begin with separating these into two major factions.

Faction 1: Profitable business.

The profitable business has found a way to reproduce a profitable sale. Steve Blank uses the phrase “repeatable sales model”. My interpretation of this is that there is a potential market value which is reliably converted into money through the business activities. The repeatability causes a reliable equation to play out where you have a high chance of staying on top of positive revenue through conducting very low risk maneuvers until the equation changes when you have the option to adjust to maintain your profits.

Faction 2: Not yet profitable business

The not yet profitable business has bigger costs than earnings. This is quite simple and includes everything which is not yet profitable. In many cases you can see a profitable business include portions of this in the profitable equation, for example by making design improvements or product line extensions. The more common scenario is however attempts at creating a profitable business out of an idea.

Now that we have these two major factions we have a tool for understanding several ways to achieve failures which will make any methodology fail. Since everyone has learned that the typical waterfall project is going to fail by default it’s not that interesting to talk about, but we’ll try and stick to general failures which can be blamed on Agile.

To predict how you will fail you can start with asking yourself: where does the money that finances the project come from? If it is financed by an already profitable business you have a particular case. If it is financed by venture capital you have another and if it is financed by yourself, often as some type of hobby, you have a third. Among these three candidates you have one in particular which spells uncomfortable failure louder than the others. The least loud failure is when you are paying yourself. No one expects success from a hobby. If you are funded by venture capital you are expected to fail about 90% of the time. No one is surprised when investments fail to find a return. The loudest failure is when an already profitable business invests in unprofitable projects. These also seem to be the ones which plays the blame game and tries to blame agile methodologies for their failures.

If you look at an already profitable business from a capitalistic perspective you would think they should not invest in a new “not yet profitable business” and rather send that money to their investors. But for various reasons they generally don’t operate this way but attempts to reproduce profitability in new innovative manners. These become complicated matters.

A profitable business has generally spent a great lot of effort to achieve their profits. As a game designer knows you will be conditioned by expending effort at achieving a goal. So the profitable business has been conditioned by its own success. This conditioning is cultural within that company. This culture tends to have an agenda which prioritizes protecting the profitable business against risks. The practical implementation of this protection is commonly called bureaucracy.

Bureaucracy causes risky product experiments to have such a great cost that they have to be cancelled before they reach the market. It is better to send the investment immediately into the trash bin than to use it to cause a potential harm to an already profitable business. This also has the benefit of creating plenty of work for bureaucrats who thereby can be hired to handle the paperwork.

Eventually this causes polarization within the company where creative forces such as marketing eventually is urged to brute force the bureaucracy to revitalize a now aged brand. They can see their market share slowly being poached by competitors. At this point in time the culture has set really well. The creative forces rally enough of a budget to overcome the cost hurdle set by the bureaucrats and begin their project.

Since times change and you learn that using “traditional methodologies” will cause failure this project must begin to work with an Agile methodology, they usually try to work with scrum.

The root of the problem here is that scrum does not remove the cultural conditioning which practically has become embodied within the bureaucratic parts of the company. There will be plenty of reasons to interpret the new project as a risk to the profitable business and thereby justifying interference.

I don’t know if there is any particular solution to this problem but I do believe that a better understanding of this structure will reduce the costs of failure for any business which is investing in any type of projects. And this is just the beginning.

To propose a practical solution to this I will recommend making investments into new business areas at a healthy distance from currently profitable businesses. Mere proximity might increase the cost of failure.

From my experience with scrum I interpret some of the rules as engineered to refuse interference from bureaucracy. These rules cause a kind of cultural power struggle within an already profitable business when scrum is introduced. Power is usually owned by the bureaucrats and scrum becomes the loser and an embodied target for the blame game.

The Symptom of Repeated Success

This is a post about something I have heard about happening from some distance. Organizations working with scrum or similar methodologies reach the conclusion that the methodology does not fit them. This is a hypothesis about why that tends to happen and a bit of reasoning around the problem.

As developers begin working with scrum or other agile methodologies it often appears as if they manage to achieve a productivity boost and gain features at a faster rate than they were used to. The teams really like this and find it to be a good practice. There is however one problem surfacing relatively often which present itself as a positive symptom of continuous and rapid success. However, behind this seemingly great facade hides a problem that seems eeringly common. The rapid increase in productivity and success is often a symptom of well... failure.

Now I'll have to explain, yay!

The problem is that the team is not achieving manageable failures. Every backlog item which comes from the product owner is successfully implemented and done within a sprint or two of work. Often each sprint contain several backlog items and the product takes on several new features in rapid succession. Now there is one of two possible states that you are in, one of these is more probable to be true and the other is likely closer to resembling a joyous pipe dream:

1: Awesome!!

• The production is going on rails and creating fully tested and reliable user value with every backlog item which is started. The Product Owner is a domain expert in the field, has incredible insights in design and production techniques and is well connected with the market.

2: Hmm, what did the user feedback say again? Does that really match our Definition of Done? Or what does it really say?

• The testing of user value does not inform the team about how much value they have been producing and they are developing features blindly.

If your team is experiencing this symptom of the repeated success you have a statistically troublesome scenario. The problem is that for the "Awesome!!" result to really occur you need to have one of the best Product Owners in the world. These guys are quite rare. You might also be part of an organization which is building an existing product within an existing market, this is the metaphorical equivalent of a Nuclear Power plant. The problem is known and the solution is known.

The user Values you develop are not known beforehand

It is quite likely that you are developing a product which aims for a niche market or a low-cost alternative within an existing market. In these cases your Product Owner will need to use the development team to run experiments against the market to find out how to handle the unknown factors. Be especially aware if you are making a new product in a new market, that means your hypothesis will be wrong almost all the time.

As any scientist will know you will not always validate all hypothesis as being true from their first experiments. This is where the failure of agile and scrum screams its loudest, at least from my perspective as Game Designer. I know that even the best of designers will only have a theoretical foundation which pre-validates a portion of the designs. A generous estimate says that you will be roughly correct maybe 70% of the time, but each feature contain several sub-hypothesis which in turn has about the same probability of success. This means that more than 20% of your Backlog will be dead wrong. And now I am being very nice about that number...

Features which are not valid user value is waste which is expensive to clean up

If you are part of an agile team which fails to invalidate the hypothesis within the Backlog be aware that you are most likely producing a huge design debt within the product you are working on. Not only are you failing to find the defects before they impact the user experience, but you are also failing to understand where the real user value actually reside and are thereby effectively torpedoing future design efforts aimed at cleaning up the debt.

Personally I would advice that the only reliable way to step out of this trap if you are sitting in it is to treat design debt just as you treat technical debt. Design Debt most commonly rears its ugly head as increased complexity within the user experience. Once there are many features of surmised value within the product which has been deployed it becomes quite difficult to separate the ones which have user value from the ones which don't. This is a type of debt which grows fast with compounding interests because the probability that any future backlog actually does anything good to the user value drops with the complexity of the already faulty design. This is well supported by how system complexity also increases exponentially with the number of components as neatly described by Metcalfe's Law.

The way out of Debt is slow and painful

You will need to slow down and begin to refactor the user value through effective measurement within the product. The more common attempt is to increase the volume of features which has the result of increasing the volume of produced waste.

I am not too sure about development teams which have had the ability to climb out of this type of mess. The most common exit strategy I hear people deploy is to package the failed product as "valuable experience" then starting over with a new vision, mostly to achieve the same experience next time. There are also other tricks of business magic which some have been able to perform where they can sell the work done to wealthy customers.

Conclusion

Beware of agile teams that do not report failures from the interpretation of relevant feedback from valid tests. These projects are quite likely to be creating delayed waste which have impact enough to be fatal to the project. When working on a particular backlog item ask yourself: "How would the statement that this feature is waste be defeated by a valid proof to the contrary?"

If the best answer you think you will get is a whole lot of opinions then you have a good reason to help improve the development process.

The complete cat strategy guide for game production

This is a story about a group of agile cats with skills and a passion for games who build a game together. They met somewhere and founded:

One Lean and Mean cat machine which will be doing something cool and have an awesomely good time together while doing it. The cat team introduces themselves and writes witty descriptions about what they do on their Google Wave.

Fixer Cat: "I am like doing all that stuff that no one else does, making sure that nothing stops the other cats from doing awesome stuff. I am obsessive about development process, cycle time, kaizen, integrity and communication. Other IT projects might call me a combination of QA, Operations and maybe Producer."

Artist Cat: "Art is like a universe of symbolic language which has emotional resonance and communicate directly into the users brain through traditional data generation. I paint, I make sounds, I write texts, I make landscapes, labyrinths, hats, mice, animations, music, particles, movies and everything that you see and hear within the game. I got quite the education to know the theory, tools and practices of all these spiffy art forms."

Wizzard Cat: "Computers speak directly to my soul. Little impulses of electricity becomes living worlds full of cats and mice through the magic I perform. You might think the world is made out of things, I know it is made out of math, logic and process. Nothing within the game reaches the player unless I make it work. All the programming languages, technical platforms, rendering engines, deployment process, automated testing and the lot is stuff I have mastered. I am quite awesome."

Expert Cat: "Sun Tzu would be pwnd by me. If he wasn't dead already, I am still alive. I know marketing and how to position products. I know how to measure success and set goals which will be even more successful. I speak directly to the end users and learn what they desire. I communicate the markets desires to the team and workshop with the team until they prove they understand what the market wants. I test the products on the market and improve our results relentlessly. A game developer would call me a designer, maybe a non-traditional artist or a biz analyst. I also do stuff like interaction design and scripting when needed."

Our four cats are maybe not using the most conventional titles as the ones you might be familiar with. But keep in mind that these guys are mean and lean cats. They don't follow conventional rules. They make their own rules and play their own game to win.

Two kinds of players here, you know them already... In the world of cats they dont know they are a market. They just want to live their lives happily and in peace.

The cats start working on a vision which causes a lot of half friendly cat fighting, including hissing and big fuzzy tails. Eventually cool cats reach agreement and begin with reality checking what they agreed upon. They keep running around in this loop until reality align with their concept. Or maybe it is the other way around, its difficult sometimes. If they fail they go back to the vision but these cats are so mean they only fail about half the time.

The cat team takes a close look at the vision and how it is positioned on the market. They don't want to run into those nasty big competitors, some smaller competitors are quite ok however. They know what they are doing so they pick a spot which looks like it might have a few lonely cats without suitable and good games to play.

Since the cat team has found its market position for a game they can start hammering out the concept. What kind of thing is this game that these other userCats are likely to want? They figure out what kind of symbolism might stimulate this audience into taking action. What kinds of goals the userCats are likely to find engaging. And they put these things into a list of priorities for production.

With priorities sitting on the wall as post-it notes they take the first one and they go out on the streets where userCats are hanging around waiting to be entertained. They talk to the userCats and carefully nudge up some information about the highest priority. Woot! This works! UserCats like this type of stuff... well sometimes at least. Quite often they have to run this loop a few times before userCats seem to provide the correct information. They might also learn new things about the competition here which needs to be taken into consideration. Sometimes they even encounter the wrong userCats and have to change them for others.

With a successfully developed list of priorities the cat team starts producing prototype products. They think this is what a Minimum Viable Product is about. They bring it to the userCats and see what happens when the user cats try it. In the beginning it does not work very good, but the mean and lean cat team knows how to repair their broken prototype and after a while the userCats start to appear rather engaged when testing it.

And there it is! The cat team has found resonance after a few attempts of prototyping. This is a key moment for the team to get a morale boost which will bring them forward towards cat nirvana.

With the successful prototype to guide them the cat team goes into production.since they are so lean and mean they don't need any more cats for this, all they need is an even tighter relationship with the end userCats. So they pull userCats really close to the team to make sure they can test their product continuously against a useful feedback pipeline.

At this point the team starts investing a significant portion of their time into making sure that they can keep up a high speed through the loop. A lean and mean cat machine knows that the most important thing is to be able to react to feedback as fast as possible and they keep on making their reactions faster and faster for every circuit through the loop. The Wizzard protests loudly when the magic might be going kaboom in the future and the Expert stops the production and does a "5 whys" if the feedback from the userCats starts sounding like strange noises.

While running through the production loop the cat team is making these userCat loops, Daniel Cook named them STARS. Each feature consist of many of these loops and each iteration the cat team add value to these loops. Stimuli is the first thing the userCats encounter. The team makes sure all the stimuli in the game makes sense and fits the expectations of the userCats and sometimes they make sure that the userCats are positively surprised when moving through the STARS loop.

Daniel Cook is a hero for lean and mean cat teams.

Eventually the cat team will be successfully stringing the userCat loops together in some awesome patterns. As the Newbie user cat successfully runs through the loops the Newbie cat gets more and more engaged. The cat team is so good at this that the loops are whole and working. UserCats experience good pleasure and they don't burn out or feel confused and annoyed. As a userCat plays the first finished parts of the game it feels showers of dopamine in its brain which are produced when the nerve system extends its associations to better handle the Skills required by the game.

When the cat team picks up a new piece of the game to build they know that they are only in the beginning of a journey which will need a few round trips with the same piece. They are quite competent so they understand that trying to add another piece before the current one reaches the "market demand per feature threshold" is going to lead to a ton of waste in future production.

While working on making one function in the game good they methodically monitor the feedback from userCats for each iteration. They keep on iterating until there is statistically reliable proof that the purring is caused by the product and not something else.

Oh noez! The cat team has run into a road block!

Regardless of how many times they iterate on a piece of the concept which aim towards the vision they just can't make the userCat purr from it. This is a tough spot to be in. Now they have to do a pivot. Such a lean and mean cat team is not very scared of this problem. They know that they have a solid core product developed which causes userCats to purr satisfyingly. To keep on progressing their product so they can get even more purring all they need to do is some analysis and observation.

The cat team steps back a bit and takes a close look at the market again. "Ok" they say, lets figure out where to move now. Should they abandon the project? Switch to some totally brand spanking new vision? Or ignore the feedback and release it anyway? No, instead they keep one paw firmly planted in their current vision, and prepare to take a step towards a new direction.

They adjust the aim of the vision a bit, keep the successfully developed userCat value, kill the remainder of the old backlog and define a new set of concept pieces which will lead to the new visionary point they find interesting.

Eventually the game does all of the things it should be doing and the userCats are excitedly running along its dopamine infused feedback loops, having a blast and feeling the luurve all around. The userCats realize that the lean and mean cat team made something that was just perfect for them and they send love letters and even a bit of money to the cat team.

The Cat Team are heroes in the world of the userCats and they know it. What the cat team will be doing next is a secret hidden within the future.