Efficiency by Forced Iteration

This is a technique I apply on almost everything I do which has any expressive ambition. I'll call it Forced Iteration, I have not previously though of it as anything which has a name. After power-reading the stuff that Eric Reis and his crew is doing I realize it lies dangerously close to the idea of Continuous Deployment.

I have been messing around with the creation of usually unambitious artistic artifacts such as pieces of music, texts or even little experimental games. What I have realized over about twenty years of doing this is that the most effective way to improve the result is to publish it and make it available to some kind of living and breathing real audience. Even if it is only one other person in the audience this is vastly better than avoiding the publishing step.

What I do is that I first fiddle with the piece, this blog post is an example of such a piece. Then after I start losing tempo on the production of it I'll publish it. This causes me to feel a slight level of panic that I just published something which is total rubbish. The understanding that the rubbish is already out there and potentially being consumed by an end user triggers a reflex which makes me fire up a burst of energy which I spend by seeking out the worst flaws and I fix them just a few minutes after the publishing. Whew... the feeling of fixing the broken bits before anyone really saw them is actually quite good!

Now after the worst bits are fixed I will have a bit of breathing room which I use to judge the resulting piece of work more closely. This tends to spit out a new series of micro-iterations which hammer away at the lesser errors that I can see.

I am not a good writer, and I know that my English skills are quite lacking compared to what you'll see from a more engaging writer. This knowledge sets my level of ambition to be relatively low when it comes to making this blog great. Maybe someone who is smack dab in my target audience finds it comprehensible enough. Well, anyway...

The act of forcefully publishing the unfinished piece of work effectively speeds up the process of going from the idea of doing something until it meets the standards set by my level of ambition. I believe that I find this behaviour to be natural from having somewhat of a background as a live performing musician, even if it was a long time ago now. The live performing musician is in a constant state of streaming art directly to the audience. The reaction from the audience is the fuel which drives the engine of creation. And the idea that my audience will see the first iteration fuels the engine that creates the second, and so on it goes.

By using Forced Iteration this way the volume of fuel is maximized, at least for my type of engine. I know there are great artists who achieve greatness by going the other way around, but I have no idea of how they really do it, or how fast they really work.

The complete cat strategy guide for game production

This is a story about a group of agile cats with skills and a passion for games who build a game together. They met somewhere and founded:

One Lean and Mean cat machine which will be doing something cool and have an awesomely good time together while doing it. The cat team introduces themselves and writes witty descriptions about what they do on their Google Wave.

Fixer Cat: "I am like doing all that stuff that no one else does, making sure that nothing stops the other cats from doing awesome stuff. I am obsessive about development process, cycle time, kaizen, integrity and communication. Other IT projects might call me a combination of QA, Operations and maybe Producer."

Artist Cat: "Art is like a universe of symbolic language which has emotional resonance and communicate directly into the users brain through traditional data generation. I paint, I make sounds, I write texts, I make landscapes, labyrinths, hats, mice, animations, music, particles, movies and everything that you see and hear within the game. I got quite the education to know the theory, tools and practices of all these spiffy art forms."

Wizzard Cat: "Computers speak directly to my soul. Little impulses of electricity becomes living worlds full of cats and mice through the magic I perform. You might think the world is made out of things, I know it is made out of math, logic and process. Nothing within the game reaches the player unless I make it work. All the programming languages, technical platforms, rendering engines, deployment process, automated testing and the lot is stuff I have mastered. I am quite awesome."

Expert Cat: "Sun Tzu would be pwnd by me. If he wasn't dead already, I am still alive. I know marketing and how to position products. I know how to measure success and set goals which will be even more successful. I speak directly to the end users and learn what they desire. I communicate the markets desires to the team and workshop with the team until they prove they understand what the market wants. I test the products on the market and improve our results relentlessly. A game developer would call me a designer, maybe a non-traditional artist or a biz analyst. I also do stuff like interaction design and scripting when needed."

Our four cats are maybe not using the most conventional titles as the ones you might be familiar with. But keep in mind that these guys are mean and lean cats. They don't follow conventional rules. They make their own rules and play their own game to win.

Two kinds of players here, you know them already... In the world of cats they dont know they are a market. They just want to live their lives happily and in peace.

The cats start working on a vision which causes a lot of half friendly cat fighting, including hissing and big fuzzy tails. Eventually cool cats reach agreement and begin with reality checking what they agreed upon. They keep running around in this loop until reality align with their concept. Or maybe it is the other way around, its difficult sometimes. If they fail they go back to the vision but these cats are so mean they only fail about half the time.

The cat team takes a close look at the vision and how it is positioned on the market. They don't want to run into those nasty big competitors, some smaller competitors are quite ok however. They know what they are doing so they pick a spot which looks like it might have a few lonely cats without suitable and good games to play.

Since the cat team has found its market position for a game they can start hammering out the concept. What kind of thing is this game that these other userCats are likely to want? They figure out what kind of symbolism might stimulate this audience into taking action. What kinds of goals the userCats are likely to find engaging. And they put these things into a list of priorities for production.

With priorities sitting on the wall as post-it notes they take the first one and they go out on the streets where userCats are hanging around waiting to be entertained. They talk to the userCats and carefully nudge up some information about the highest priority. Woot! This works! UserCats like this type of stuff... well sometimes at least. Quite often they have to run this loop a few times before userCats seem to provide the correct information. They might also learn new things about the competition here which needs to be taken into consideration. Sometimes they even encounter the wrong userCats and have to change them for others.

With a successfully developed list of priorities the cat team starts producing prototype products. They think this is what a Minimum Viable Product is about. They bring it to the userCats and see what happens when the user cats try it. In the beginning it does not work very good, but the mean and lean cat team knows how to repair their broken prototype and after a while the userCats start to appear rather engaged when testing it.

And there it is! The cat team has found resonance after a few attempts of prototyping. This is a key moment for the team to get a morale boost which will bring them forward towards cat nirvana.

With the successful prototype to guide them the cat team goes into production.since they are so lean and mean they don't need any more cats for this, all they need is an even tighter relationship with the end userCats. So they pull userCats really close to the team to make sure they can test their product continuously against a useful feedback pipeline.

At this point the team starts investing a significant portion of their time into making sure that they can keep up a high speed through the loop. A lean and mean cat machine knows that the most important thing is to be able to react to feedback as fast as possible and they keep on making their reactions faster and faster for every circuit through the loop. The Wizzard protests loudly when the magic might be going kaboom in the future and the Expert stops the production and does a "5 whys" if the feedback from the userCats starts sounding like strange noises.

While running through the production loop the cat team is making these userCat loops, Daniel Cook named them STARS. Each feature consist of many of these loops and each iteration the cat team add value to these loops. Stimuli is the first thing the userCats encounter. The team makes sure all the stimuli in the game makes sense and fits the expectations of the userCats and sometimes they make sure that the userCats are positively surprised when moving through the STARS loop.

Daniel Cook is a hero for lean and mean cat teams.

Eventually the cat team will be successfully stringing the userCat loops together in some awesome patterns. As the Newbie user cat successfully runs through the loops the Newbie cat gets more and more engaged. The cat team is so good at this that the loops are whole and working. UserCats experience good pleasure and they don't burn out or feel confused and annoyed. As a userCat plays the first finished parts of the game it feels showers of dopamine in its brain which are produced when the nerve system extends its associations to better handle the Skills required by the game.

When the cat team picks up a new piece of the game to build they know that they are only in the beginning of a journey which will need a few round trips with the same piece. They are quite competent so they understand that trying to add another piece before the current one reaches the "market demand per feature threshold" is going to lead to a ton of waste in future production.

While working on making one function in the game good they methodically monitor the feedback from userCats for each iteration. They keep on iterating until there is statistically reliable proof that the purring is caused by the product and not something else.

Oh noez! The cat team has run into a road block!

Regardless of how many times they iterate on a piece of the concept which aim towards the vision they just can't make the userCat purr from it. This is a tough spot to be in. Now they have to do a pivot. Such a lean and mean cat team is not very scared of this problem. They know that they have a solid core product developed which causes userCats to purr satisfyingly. To keep on progressing their product so they can get even more purring all they need to do is some analysis and observation.

The cat team steps back a bit and takes a close look at the market again. "Ok" they say, lets figure out where to move now. Should they abandon the project? Switch to some totally brand spanking new vision? Or ignore the feedback and release it anyway? No, instead they keep one paw firmly planted in their current vision, and prepare to take a step towards a new direction.

They adjust the aim of the vision a bit, keep the successfully developed userCat value, kill the remainder of the old backlog and define a new set of concept pieces which will lead to the new visionary point they find interesting.

Eventually the game does all of the things it should be doing and the userCats are excitedly running along its dopamine infused feedback loops, having a blast and feeling the luurve all around. The userCats realize that the lean and mean cat team made something that was just perfect for them and they send love letters and even a bit of money to the cat team.

The Cat Team are heroes in the world of the userCats and they know it. What the cat team will be doing next is a secret hidden within the future.

Feedback pipelines

Game production use the word ”pipeline” for the technology and behaviors needed for adding data to the end product. This is a great and useful component to the iterative process. There is a system which can be optimized for speeding up the time needed to change the data from one state to another. I have no real idea of how much of the budget for a large scale production goes into developing this pipeline. I guess it is significant and well motivated.

Now to the point of the argument, the investment made in the feedback pipeline is most likely much smaller for most game projects. This is not well motivated and I believe it surfaces as a tremendous cost, for successful project it means your staff needs to be incredibly competent and able to manage implicit iteration without excessive waste. For failed projects it is the one major cause for failure.

I knowingly use very strong words here because I want the notion challenged so I learn something useful from it.

Dangers of iterative production techniques, part 1 of n

As I mentioned in a previous post http://gamesartdesign.blogspot.com/2009/03/iterating-towards-value.html there are, from my personal perspective, several ways to cut an iterative process. This post is about general risks with any of these, and likely other, cuts.

Risk 1: Feedback conditioning

This is a risk which occurs whenever you keep getting similar feedback through a series of iterations. The shape it takes is much like what happens to us humans when we get accustomed to living close to a paper manufacturing plant which stinks of sulfurous chemicals. Our brains get used to the initially nasty experience and we get used to it. A game development team which gets familiar with some feedback and determine that it is part of reality will make a worse product than they could.

Risk 2: Reliance on implicit feedback loops

When the measurement is implicit as with the case for how the subjective components of the product are experienced it is quite likely that the team will fail to understand when they reach the top of the value s-curve and keep on investing in things which are done. For us who spend our time with the modern art form of game design this is horribly difficult to communicate. I also believe it is more difficult for the other art forms in game development as well than what it is for the older more established media. How do you know when you should make a new animation rather than keep on iterating the current one?

Risk 3: Lack of direction

This is the most often mentioned risk which comes from a combination of lacking vision or the communication of a vision which is hard for the team and business to understand. As a consequence of a poorly communicated vision the team lacks a foundation for understanding even explicit feedback loops and will progress erratically.

Risk 4: Going in circles

When the team has too little room to maneuver they rick failure to understand how to change the next experiment in the iterative process to improve the feedback state of a production. This comes appear to often come as a consequence to another failure which cascades down as an unmanageable cost of an effective cycle time. If changes to the product reach a certain volume the next change will be driven by risk aversion, risk elimination is likely to continuously surface with the same solution over and over.


Do you think this list should be extended, feel welcome to add your own risks and I’ll try bake them into part 2. ^^

Feature Creep or Vision work?

Ok so now I have made this rather horrible video with the robot band. This had a series of interesting consequences. One is that I now feel the urge to refine the music so it shows off the individual talents of the robots better. If I go for this I'll have to make some major rearrangements so they all play solos, Doppz is missing his synt-bass solo, which is a quite rare beatie anyway. And Some cool synchronized solos would be interesting between the guitar robot and the keyboard robot.

After the rearrangement would be done which is an effort of several evenings of work at least I will have to make a new video which shows these new scenes.

Since I have no practical limit on the ambition of the production other than doing it for the fun of it I will categorise this type of work as product iterations. The music would be iterated farther to fit a more well defined vision.

If I did have a set of measurable ambitions and, not that it would happen but if, the work already qualified as sufficient I would call this type of further iteration as overly ambitious or feature creep.

Layers of iterative development process

All the layers in this image contain iterative process, of varying formal structure.

Note that each level also tend to be branched to different territories, at the bottom you have different types of experts or artists producing different types of work often through implicit iteration.

Maintaining multiple options

This is a Lean productivity trick. Mary Poppendieck describes it well in this awesome movie; http://video.google.com/videoplay?docid=-5105910452864283694 - you have to watch it if you care about making good things.

When looking at game development this is done on a regular basis in some of the pipelines, most obvious when drawing art images through how the layer system operates in photoshop but also in how programmers write code with branches and classes, how storywriters use word processing software to keep dialogue and story flexible and how musicians create the music score with their software such as ProTools maintaining all the versions of the songs on the HD while edits are happening to a subset of files.

Now the question becomes how to apply this to the combined user experience which results from bringing all these pieces together. As the network theorists say something along the lines of: the whole system is the combination of the nodes and the interaction between the nodes. The game which becomes the user experience is the interaction between the nodes, the code is one node, the music is another node, the image art is a node, the interface is the tool which the user has available to navigate the network which in a good game becomes a modular agent in the network.

The game itself is a system which measure the skills of the user, these skills are deployed in context by the user to navigate the whole network. Any piece of the network which is outside the users path is roughly what lean would call "waste".

For the whole game system to maintain multiple options we'll need to expose the interaction between the nodes as readily changeable and generally transparent, human readable, data with some level of backwards compatibility. This is an utopian model, but it is not fully a pipe dream. In a not very far future we will know much more of the principles of game design. From my perspective these principles started with Chris Crawford and his model of interactivity, they have moved forward a lot with things like Raph Kosters theory of fun for game design and Daniel Cook's model of skill chains. There is a lot more stuff in this mix worth a mention, but maybe we'll go there another time.

With a better understanding of the principles for game design we will be able to identify differrent types of interactivity and thereby define the interactions between the nodes in the system with an explicit format. Music has several kinds of explicit data formats to present the patterns of sound on on various levels of abstractions, waveforms, instruments, overtones, harmonies, rythms, notes, tabulature and so on. Several of these have alternative modes, you can define a "tone" through various methods either as an "instrument (violin) playing a note (A) for an proportion of a bar timeunit" or a "fundamental frequency (440Hz) with 9 overtones (i think thats the violin if I remember it right) playing for a time in seconds". You can keep on going and define the waverofm as a function based on trigonometry if you prefer but I'll skip that example. I think the point made it through without. Note that the actual psychoacoustic representation of the violin is a lot more complex than these primitive models but on the theory side of things you go there if you want to, which is about never unless you are a bit "special". For the synthesizer manufacturer its all the way down through the trigonometry that wins the day with modulation, also trigonometry, controllers and stuff.

Game design theory is today at the stage where everyone communicate with a randomly selected spray from the whole arsenal of terminology, or rather, different people use different terminology to say the same thing and they generally don't know that there are other terminologies which define the same interactions within the system. They can barely even communicate with each other without first synchronizing the language which is a hard problem. Some very experienced designers will have a gut feeling for a lot of the principles, I myself find that I have a better grasp of some models when analysing a game play experience and some others when writing a game concept. I must admit that I also don't know the whole system, I don't think anyone does. There is no "first dan black belt" of game design and not even Sid Meyer, Will Wright, Chris Crawford or anyone else would have it if there was one.

However we can start in the pragmatic side of the equation and create a data model which best expose the whole system as nodes of data and interaction which is the transportation of other data through the system at the request of the user through the interface agent.

Now you could claim that this is already done with the programming langugage itself, and to some degree that is true. However the game systems which people play today are requiring domain experts which are not always competent programmers. (Programmers don't tend to spend most of their time doing user research as example, thats sub-otpimal to most production efforts.) Also maintaining multiple options through having several version of the whole code base is unlikely to be much help.

So what kind of data can define the interaction between nodes in the system?

If anyone wants to take a stab at it go for it, or bring up examples from various game engines. ^^ (Or I will later on and then it gets texty as usual.)

Iterating towards value

It is commonly accepted that the user value in a game and most things has a statistical probability to increase with the number of iterations it has gone through. This goes for things such as fried meatballs, your DNA, a guitar solo, a sound and so on.

This takes on different forms in different areas of life, since I am interested in writing about game development I will focus this story on some of the forms involved with making games. The process relates to cooking and music production, but since this is a big topic I’ll leave most such fun references out.

The relationship between iterations and value is often plotted as a neat little graph.

Note the S-curve type of deal going on. That says something about the existence of “no value” iterations at the beginning of the process. You got to get to some point before you have anything, then eventually it is finished and stops getting any better. To get anywhere interesting with this topic we have to take a look at the definitions of the graph. What is the value you get as a function of # iterations?

What one iteration is

One iteration is the work done from the point where there is an idea about doing something to the point where what was done has resulted in useful feedback. This is a fractal concept and it has several types of zoom levels and dimensions. The one zoom level most commonly associated with game development is on the product level. One iteration on that level have its value measured by feedback originating from the potentially shippable product through various test procedures such as user testing, technical testing and getting accepted by the product owner. These are examples of something I will call explicit feedback.

Explicit feedback gives a quantifiable measure of value which is obviously visible to the involved parties. These summarize as “done” when everything pass defined criteria often called a “definition of done”. Generally if feedback from the work shows an insufficient or negative value increase the iteration has failed.

Another zoom level for one iteration with an explicit feedback loop is the creation of code data, commonly called programming. If the code is broken it will fail to compile and all kinds of feedback is generally built into the code pipeline to prevent broken code from entering the system. The programmer makes a new iteration of the code to improve the value of the code without first shipping the code to the end user to test if it has any value. These iterations are fast, some broken code can be iterated on in a matter of seconds. This time unit defined by the time it takes to iterate is sometimes called “cycle time”, which I will also use a fractal concept. I will call this comparably low zoom level of iterating for “creative process”, the programmer creatively create the code.

Another type of creative process is the creation of images. The artist uses an image creation tool such as photoshop to create image art. This is also a fast process. The artist will generate art data on a second to second basis, undo things that go wrong and create new pixels with a stylus without thinking of it as iteration. The artist also maintains and switches between multiple options throughout the production of one image. The feedback which closes one iteration in the creation of a pixel of art data is internalized by the artist. I will call this implicit feedback.

Implicit feedback is not obviously visible to anyone. It is to the observer a subjective measurement and often leads to all kinds of emotional distress for the involved parties when it comes time to analyze the value of the output of the creative process based on implicit feedback structures. To the professional artist it is not fully subjective as some principles of the arts guide the creative process but in large it’s a relative mess compared to the efficient explicit structure of code creation. It also happens that different artists use different principles for this process which often cause communication problems, but mostly everyone has learned to live with this process quite productively.

Now that we have a model to differentiate iterations between ones based on explicit and implicit feedback it becomes much easier to understand what things you can put in through the value creation process.

A guitar solo runs its first several thousand iterations as an implicit process before it reaches the point where it gets explicit feedback in the form of praise or complaints from band members, audience or other people involved with the music production. Improvisational jazz is an example of the solo going directly from implicit practice to the end user without explicit iteration, even driven by the goal to change between each gig! The details of the solo are guided by a lot of musical theory internalized by the jazz guitarist.

The development team for a product increase its value based on explicit feedback from the producer or the team itself, as well as the implicit feedback within each team member and their feelings towards doing good or bad work. Scrum is an example of a process which forces some of this feedback to be explicit. This is good in the sense of providing direction but as all game designers or players should know we humans have the property of “gaming” systems towards maximum reward for minimum effort, hence it is likely that if not guarded well the process is used by the team to optimize towards getting the positive explicit feedback at the lowest cost possible. This is in game design terms sometimes called “bottom feeding” and needs to be guarded against by the rules of the game which define what to do to get which feedback. Another game design term which becomes relevant in explicit systems is the “mastery problem” which means that the better players will dominate the feedback structures of the game and win at a cost to the other players. Neither is good for the value of the product and digging into these details may be a topic for the future.

When explicit and implicit feedback systems fail to agree on if an iteration is successful or not there is often all kinds of problems surfacing. If these problems happen often the whole product is most likely going to fail on the market.

Implicit equals difficult

Making enjoyable and fun interactive games is primarily based on implicit iteration. Some fundamental infrastructure such as code and having a development team can be considered explicit but getting the experience to a point where the market value motivates the investment goes through tons and tons of implicit iteration.

Many of the implicit structures in a game are based on established art disciplines such as creating images, animations, music and storytelling. The product trusts itself to these domain experts effectively collaborating as a team. There are tools to rapidly iterate these components in isolation and it is part of the expertise of the experts to be masters of these tools.

The real problems surface when the implicit feedback relates to the sum of the parts of the system in the form of information design and game design. To make the game play experience have any value these also needs much iteration. Yet again there are skills and principles in the design expertise that help this process but they all fall under the implicit feedback umbrella and thereby cause communication problems.

If you look at the game industry at large you will find that there are a lot more games in certain categories, this makes the game industry appear relatively narrow. Many games resemble each other and are based on each other. The games that have been successful become so by playing on their strengths. The things that can be iterated relatively fast are strengths while things that iterate slow are weaknesses. Most are built on development techniques which upfront know where the pipeline is fast and where it is slow. This gets successful when each domain expert in the area of design owns the iterative loop for the implicit creative process. The things that have a cycle time measured in minutes or seconds reach the point of noticeable value, while anything which has a slower cycle time measured in hours or days should be considered as experience-wise valueless infrastructure. Those slower components of the experience will not be what make the user reliably have a gratifying experience. You might get lucky, but to get from lucky to certain the fundamental trick is to get the cycle time down to the range of seconds.

Techniques for effective iteration

Now that we have gone through a somewhat too big background of the problem we can take a look at a solution.

Based on concept, product vision or creative direction take a deep look at the idea which is going to result in user value then list the main components of the user experience.

• Take a through, cross-functional, look at each of these pieces. If there are more than a few dozen and in the hundred or so range you probably mistook the user experience for something too granular or technical. Or your team might not understand what the experience is supposed to be which might be a sign for future trouble.
  
  
• With the list in hand maintain the cross-functional approach and categorize the pieces in cycle time as estimates in seconds, minutes, hours, days and weeks. Include dependencies in these cycle times if needed, for example if it is relevant that some animations blend properly in the feedback system consider the pipeline for animation blending in the cycle time for animation feedback.
  

Now make three categories

1. Usability Infrastructure, everything with a cycle time of days and weeks in here
   
2. Shoehorn User experience, the things with a cycle time measured in hours
   
3. Reliable User Experience, seconds and minutes
   

With these three categories freshly composed remove everything except the third, fastest, category and see how well the things in this fast list match up against the idea about the product. It probably won’t match up very well. Don’t panic! This is quite common when making interactive entertainment and it is to your advantage. Most of your competitors will expose their weaknesses if you can figure out where their user experience is bogged down in the infrastructure category. You can also save your project by avoiding dependencies on the user experience which resides in this category. You will have some, that’s unavoidable but at this early point you have the ability to shed complexity to move the right things to the category of reliable user experience.

A general trick for getting the interactive parts of the system to the level of reliable user experience is to make all feedback elements dynamically changeable. For example if you have a storyline with a movie in it you will have the experience defined by the movie as a feedback element for an event triggered by the user. It is now important that the person who knows how to make movies can change the movie inside the experience with fast cycle time. If the cycle time is slow for movies don’t make them as parts of the system. Put the movie in the beginning instead as an intro which has no dependencies of interactive nature.

This can also be done for the verbs of the interaction circuit. You can’t expect to add a new verb with a fast cycle time so the verbs themselves go in the category of infrastructure. You can however adjust the subject the verb interact with, such as the width of a hole you need to jump over, how fast the opponent is running in a shooter. The verb can be changeable too such as the target area of the weapon or the height of the jump, expose suitable adjustable variables in an easy to change and preview data model. Beware tho, don’t mess up all the level design by changing the jump variables after production of the levels has started!

Use a recursive strategy to tune the experience. Make sure that the user is protected by efficiently tweakable variables to lean against all the way along the user journey. This will allow you headroom to react to feedback on the user experience at any point during the production. The larger percentage of what the user experience you get in a fast iterative loop the better. It’s better for the user to play a tiny game with a flawless emotionally satisfying user experience than a bigger game with a few uncomfortable moments. Note here that proper pacing is not to equal “uncomfortable” because the player will need both peaks and throughs on the emotional side to enjoy playing.

Exceptions

There are some exceptions when we look at the established game industry. Mostly they are rooted in incredibly huge budgets. Someone is spending so much money on the business that you can move the infrastructure category one notch up to the week level and just pour calendar time and man hours on making slow iterations instead. This also tends to interact with established development pipelines such as game engines for specific types of interactions. This approach is however on its last legs or already dead but it made some good stuff the last decade.

Another type of exception is the undiscovered market. You usually don’t predict these, but it happens that a new market is discovered by a product developed with a random approach towards creating value. Usually the resulting product is soon dethroned or bought by a big budget competitor. Looking at the music industry we see these random productions rather often but for each successful one we see thousands and maybe millions of failures. Reliable and consistently successful musicians iterate more or less consciously towards a safe result including all relevant disciplines in the process.

A lot of products also have no ambitions of providing a valuable experience. Instead they provide something else of value which may be relevant information or a service with pragmatic and explicit solutions to specific user needs, however games are only partially explicit and rarely pragmatic experiences.

phew, thats a long post...

User Value, by looking at users

With a definition of design fully focused on creating user value it appears important to have a decent understanding of what user value is. The most important aspect is that it needs to be something you can measure.

The first and most useful method comes through looking at and listening to users, or testers who reasonably well represent users. This method is a bit ugly because it will not give you much hard data. The lack of hard data will inhibit the ability to communicate the result to anyone without insight into the actual test itself. People you need to communicate the result with will have to trust your interpretation of the behavior exhibited by the user during the test. You are already fully loaded with test analysis hardware for this particular type of test, use it. We’ll look at methods for scaling up this type of test later but for now think small.

You can start measuring the value of any design as soon as you can find a test subject. A good start may be to talk about the idea with someone. This is extremely cheap, but the test output is very unreliable. However if your design gets shot down by such a primitive test then go back and start over. Or adjust accordingly, possibly by replacing the test subject with another tester.

If your cheapest test is successful it might be time to invest a bit of effort into the design process. What kind of investment you will benefit from depends a lot on the idea itself and your own skills. At this point I tend to find myself needing to develop my own understanding of the idea which is getting developed so I sketch up a series of takes on the idea. If the idea is associated with a game type of product or if the idea is the whole game itself I tend to find a primitive sketch of interface components to be a decent start. This helps me get a better grip on what actions the user should be taking. The end result does not inherit any particular components from this simple sketch, but it gives me a better fundament for testing the idea on more test users.

If the idea survives this far and if it is simple enough it is time for a functional prototype, some paper prototyping is a good idea to start with. If the idea has a close relative implemented in an existing product you can look at altering that product to fit the idea or if you got plenty of time and access to the right skills develop a prototype in code. However most ideas are critically dependant on the value of other ideas, after you got the first idea to the point of where you can invest some time in it you are likely better off measuring the value of the other ideas which it depends on. Take your time and go through each idea until you have collected a firm picture of all the relevant ideas and their individual value before promising any end result to anyone.

Keep in mind that ideas are cheap, plentiful and most often bad. Avoid promising that an idea has value until after you have verified its value against a decent representative of users. By continously measuring the things you do against test users you increase your chances of designing successfully. When you fail to get a successful test from an effort you have produced waste.