It cannot be said often enough: Agile methods were designed for complex problems, not chaotic ones.
Just as a reminder – what is the difference between complicated, complex and chaotic problems?
- In complicated problems, there is a direct and predictable relationship between cause and effect. It is difficult to predict the effect from the cause, but with a certain skill set, it is nonetheless possible.
- In complex problems, there is a relationship between cause and effect, but the effect is generally not fully predictable, and the relationship between cause and effect can only be fully understood in hindsight.
- In chaotic problems, there is no discernible relationship between cause and effect.
Agile methods like Scrum are most effectively applied to complex problems, where short feedback cycles allow us to iteratively gain a better understanding of the complex cause-and-effect relationship.
It is important to consider not only the problem itself but also the environment. Not just the task itself but also the environment can be complex or chaotic.
In a complex environment, outside influences are not necessarily predictable, but there is still a meaningful cause-and-effect relationship. Stakeholders follow the work progress and derive new ideas and requirements primarily from the results of the work. Agile is well-suited for integrating these new requirements into the workflow. The result is a controlled creative process. Looking at the resulting progress, we usually see a „x steps forward and y steps back“, where x is greater than y. This means that in a complex environment, we have to be prepared to rework certain parts of our previous work, but we will see a steady progress, and short feedback loops are meant to ensure that x remains not just greater but significantly greater than y.
In a chaotic environment, outside influences are not just completely unpredictable – they also are in no way related to the work results. The stakeholders constantly introduce new requirements which are not related to the previously done work. New features or whole new projects appear out of nowhere and supplant previously high-priority work items. In this kind of environment, we also take „x steps forward and y steps back“, but here, x might be greater than y, or y might be greater than x, which leads not just to a lack of progress from the stakeholder‘s point of view, but which also creates a lot of frustration for the developers. This is not a creative process. It is a destructive process. Some people seem to believe that creativity arises from chaos. This is not true. Creativity arises only from complexity. Chaos is a purely destructive force.
When confronted with the loss of efficiency created by this chaos, stakeholders usually reply: „But we are agile!“ often followed by an „Aren‘t we?“
The big issue here is that stakeholders often do not realize that they are creating a chaotic environment, or they do not realize that agile methods are not going to magically solve the problems caused by arbitrarily changing requirements. They assume that agile methods will fix these problems, because after all, agile methods are designed to deal with changing requirements. And while this is true, agile methods only work efficiently when we are dealing with complex changes, not chaotic ones.
Obviously, we can easily keep integrating arbitrarily changing requirements into a product backlog. We can change priorities on a daily basis. But one should not expect any kind of efficient output from this chaotic environment only because „We are agile!“ In a chaotic environment, even perfectly built agile frameworks can NOT ensure that x is greater than y in the „x steps forward and y steps back“ equation.
It is critically important to understand this, because when a project eventually fails after months or years of chaos, it is easy to blame the agile methods for not having magically turned chaotically changing requirements into a valuable product.
AgileEvangelist 