Sunday, March 29, 2009

The usefulness of bugs

This is an interesting topic we had a little look at recently. In my world a bug is a subgroup of something that with a wider definition would be called a defect. Googling for a definition of the term defect gives a useful result quickly which says this:

- An imperfection that causes inadequacy or failure.

This a good place to start.

In my experience the vast majority of defects associated with professional game development or entertainment in general belong to the category of inadequacy of the user experience. I would call it ‘inadequate emotional attachment’. It happens that the experience of the player is significantly damaged by the existence of a good old fashioned bug. A program failure or crash will generally not be helpful towards the emotional state of the user but these problems are thankfully not all that common anymore and various measures are in place to remove them from software products.

A much more common problem is the lack of something intangible which makes playing the game feel just right. There are tricks or theories you can use to make better games but they are not generally accepted yet so they are hard to describe without getting into very lengthy descriptions of things. So instead of getting stuck in a theoretical black hole I can use a metaphor from the art of making music.

Most of the music that reach you will have been analyzed through a rigorous process of defect detection. During its creation the music you hear will have gone through lots of iterations, thousands to millions of tweaks and changes have been made to create the result that is emotionally attaching enough to reach your sensory system. These tweaks have had all kinds of reasons for being done. Different composers or music creating organizations have different methods towards doing things but in general we can take a reductionist look at it.

Most of the work done when creating music is inadequate on the idea level. The idea behind the work is not worth the trouble of refinement. The musician detects this quickly and iterates the idea. Eventually the idea is iterated to a point where it reaches something I will call experiential harmony. This is based on all kinds of details, such as the sounds of the instruments, the arrangement, the rhythm, melody and so on. The result may not yet be anything complete but it has reached a “proof of concept” level. To reach this point the musician has gone through defect elimination of several types. Different types of defects are eliminated based on different techniques.

Finding a good sound is the process of selecting the right instruments for the various parts of the arrangement and placing them in the sound image where they fit. This is something which typically starts with a great lot of simple defects and ends with a system of sounds which interact experientially productive.

Finding the notes played by each instrument is another type of defect prone creative process. Some parts come naturally and some are iterated heavily. It is easy to get to a state where the music is broken due to bad notes. These can be detected and changed from bad to good relatively easily.

The difference between a defect note and a defect sound is of some relevance to the topic of this article. The classical, and in modern times rare, bug is defined as a defect note. It is an explicit error in the construction which can be removed or changed to improve the emotional attachment of the product.

The more common defect in a user experience is a complex problem which arises from the interactions in the whole system. We hear player say various things to state the existence of these defects.

The game is too hard, the game is too easy, boring, slow, stressful, not interesting and so on. You might think that “he wrote boring and not interesting as different things, how stupid.” But from a game designers perspective these are different things, boring is about patterns and the interesting is about context. When you use a players feedback to iterate towards value you are well served to realize that most of the feedback you get will be describing symptoms of complex problems.

A complex problem is in this context a defect which arises from the interactions in the system. Classical examples are unrestrained positive feedback loops. These are so common that we even got a specific label and fix for them which is “restrain feedback loop” by changing how the system interact or process data in one or more points. When the player says the game is too slow it means that the player is waiting for relevant information for too long, this is matter of pacing and tuning which has different complex structure in different parts of a game application.

What makes this fundamentally interesting is that the process of creating emotional attachment is not reactive to user feedback. User feedback is useful for providing the process with relevant information at some points, especially useful for detecting complex problems in the system. The solutions to problems need to come from the system creators.

If the project which is creating this emotional attachment classify defects which are complex problems as bugs you are likely to have a much worse problem than bugs unless you already have reached primary business objectives for the whole product which means it will be ok to have a total failure anyway since everyone is happy with the already accomplished result.

From a musicians perspective it is intolerable to accept bugs. The existence of bugs in music means that you prefer making more of the bad music than making valuable music. You can use this strategy during the iterative process and for example accept bugs in the melody while refining the sound but you will never consider a piece of music to be “done” until it is free from defects and thereby flawless. This means that bugs which are reported from end users should return to the product development as the highest priority backlog, if they return as a randomly prioritized task list you are most likely far behind schedule towards reaching business goals.

Practically we can probably look at several strategies which deal with these problems. I myself believe the development team needs the mandate to stop defects from reaching the end user in the first place as they are the least likely to be having other goals than making a great product. If you reach a state where it is more important to release a defect product than it is to create a flawless product you are probably in trouble already.

No comments:

Post a Comment