AgileEvangelist

Has anyone else ever had crisis meetings with project managers after a first Sprint? It seems to be a very common thing. After all, first Sprints often go wrong. Of course, already the use of the word “wrong” is highly debatable, but let’s say they go “wrong” from an outside point of view. The team is not yet settled in, they have not even started their storming phase, they have no idea how much work they can get done in a Sprint, they are working with new tools and toolchains. So of course, many problems surface during the first Sprint. And that is great. More problems -> more learnings -> more improvements.

So why have I had to sit through so many first Sprint crisis meetings with project managers? There’s the project manager asking me why the team underestimated the work, why they didn’t see early enough that they would not manage to reach the Sprint Goal, why they did not raise impediments earlier, and so on. And of course, the project manager wants to know why we don’t have a burndown chart, a popcorn board and why subtasks are not being estimated in person hours. Because all of that would have ensured a “successful” Sprint. And that is just the tip of the iceberg.

I think one major reason for this crisis mode is that project managers in a classic project world are used to projects that go very smoothly at the beginning. Nothing goes wrong for many many months. Everything is being reported as “on track”. Everything goes according to plan. But then, three months before the fixed delivery date, people start admitting that things are not going so well, and the test teams are finding more bugs than the developers can fix. So experienced project managers go into crisis mode the very moment these indicators start popping up – because after all, they only have a few months left. Experienced project managers are trained to panic the moment they see a problem. That is a survival strategy in classic project management.

It is also a part of this project management experience not to trust anyone. After all, everyone has been telling you for many months that everything is fine, while in reality everything was a complete mess. So as a project manager, you don’t trust those people to fix things, but instead you go in and micromanage everything.

How does that translate to an agile project? Like I said, there are many good reasons why a first Sprint may not go so smoothly. And in an agile project, you see this immediately. You get a lot of indicators for things that need to be improved.

Now remember the project manager’s survival strategy: You see problem indicators, you go into crisis mode, stop trusting people and start micromanaging. So there are the Scrum Masters, sitting in a crisis meeting with the project manager after the first Sprint, being told what to do to fix all those terrible problems.

How can this be avoided? I guess one important method for you as Scrum Master or Agile Coach is to manage a project manager’s expectations – to let them know in advance that the first few Sprints will not go so smoothly and to explain to them that this is an important part of the learning and improvement process. Build trust by being honest with the project manager. You have to to remember though that honesty and transparency is something that many project managers are not used to, so you have to understand that this is a learning journey for them.

If you are the project manager in this scenario, I won’t even tell you not to get involved, because I know that it might be your natural survival instinct. Transparency is there for a reason, and it’s completely fine that you ask some questions. But I’d ask you to resist the temptation to micromanage. Don’t tell people what to do. You hired Scrum Masters for a reason. Ask them what the teams have learned, and you’ll probably get plenty of answers. Give yourself the chance to be surprised by what a team can learn within a few Sprints.

Disclaimer: I know, “project managers” are not considered a part of agile projects by many, but most organizations still use project managers, especially when they are at the beginning of an agile transition, so I am speaking from a realistic and pragmatic point of view here. If crossing your arms and saying “This project manager shouldn’t even be here” works for you as a Scrum Master, then that is great of course.

Have you ever felt that you are not getting as much out of your retrospectives as you should? Perhaps the problem is already in the term “retrospective”. What is our main goal in an inspect&adapt cycle? Is it the “inspect” part: looking back and analyzing problems? Or is our main goal the “adapt” part: generating creative ideas and building an action plan for improvements in the future? I believe it is the latter. So why do we call the event a “retrospective” and not a “prospective”?

Looking back at what problems we have been facing (the “inspect” part) is necessary to build a platform for further discussion. The big question is how far we want to go in analyzing the past problems. A deep analysis would be especially useful in a complicated, engineering-focused environment, where a “Good Practice” can be applied to a recurring problem, because we know that if a solution (would have) worked in the past, it will work in the future. In complex environments, we cannot be so certain. And most of the time, we deal with complex problems in our retrospectives.

Look at the “5 Whys” method for example. That is a great method for quickly finding an underlying cause for a problem. So let’s play through a common example:

Problem: We couldn’t deploy anything in the last Sprint, so nothing got “Done”.

Why?
Cause 1: The toolchain was completely broken. Nothing worked.

Why?
Cause 2: We only have one person working on the toolchain.

Why?
Cause 3: Because we don’t have a budget for a second person to work on the toolchain.

Well, that looks good so far. But from here on out, the Whys become increasingly pointless:

Why?
Cause 4: Because we did not plan for a bigger budget in the initial planning.

Why?
Cause 5: Because the project manager doesn’t have sufficient experience with large-scale projects like ours.

So what have we achieved with the last two “Whys”? We have found somone to blame (the project manager), and we know that if we had a time machine, we could go back in time and plan a bigger budget. That leads nowhere. Have you ever been in a retrospective where you found out who to blame and that having a time machine would be really great? Did you find these insights to be particularly useful?

Well, the exercise was not completely pointless. At least it looks like we have one important insight: The budget is too low. So now the common approach is to climb back up the “chain of causes”: Get a bigger budget -> Hire an additional person to work on the toolchain -> Fix the toolchain -> Get stuff “done”. Simple enough. But what if we find out that we will not get more money?

In my opinion, we could have stopped after the first “Why”. We know why we couldn’t get anything “Done”. Getting more people to work on the toolchain is an obvious fix. Climbing down a ladder of causes is pointless and only limits us to the solution of “We need more money”. What if there are other solutions? Can team members help out with fixing the toolchain for a few days? Can we reuse something from another project instead of maintaining our own toolchain? Is it time to rethink the toolchain and for example throw out steps or quality gates that nobody actually needs?

I am not saying that these other solutions are better. But working in a wider solution space is definitely better. Drilling down into the problem space unnecessarily can keep us from looking at the wider solution space though. That can lead to discussions like: “Why use up precious developer resources to help with fixing the toolchain if we know that it’s all the project managers fault?” Have you ever been in a situation where you got stuck in that kind of thinking? Well, that’s what you get for wasting most of your retrospective drilling into the problem space.

Even the most common retrospective structure (from the book “Agile Retrospectives” by Esther Derby and Diana Larsen) leads you down that path. After setting the stage, you gather data, and then you generate insights, and these are not insights about what to do better in the future, but about why things went wrong. The “5 Whys” are actually one method for gathering insights proposed in that book. And the authors actually discourage you from looking at solutions too quickly, but they prefer thinking about causes analytically. This obviously caters to an engineering mindset, but we are usually not dealing with engineering problems but with complex problems in a complex adaptive environment. In one model for a two hour retrospective in the “Agile Retrospectives” book, 50-80% of the time are assigned to talk about the past in “Gather data” and “Generate insights” and only 15-20% of the time are assigned to talking about the future in the “Decide what to do” segment.

Doesn’t that strike anyone as completely bizarre?

What kind of creative ideas are we going to come up with in 20 minutes?

I propose to turn this around: We should spend 20% of the time to build our platform – to figure out what went wrong and  to determine the basic root causes. And then we should spend 80% of the time on talking about what we can do better in the future: Action plans, creative ideas, outside-of-the-box thinking, experiments. We don’t need to spend 80% of the time on becoming “problem experts”. We should spend 80% of the time on becoming “solution experts”.

This kind of event would not be a forum for crying over spilled milk, for nostalgically dreaming about water that was already under the bridge two weeks ago, for assigning blame or for discussing about how to build a time machine to fix past mistakes. Instead, it would be a generator of ideas, a catalyst for change, a foreward-looking creative workshop. The key question of the event would not be “What went wrong?” but “What are we going to do better?”.

And this event would no longer be a retrospective but a prospective.

Transparency is an important cornerstone of agile methods. However, transparency purely for the sake of transparency – or even worse – to facilitate control or to alleviate irrational management fears, only creates pointless overhead.

No user story has ever been completed faster by being split into sub tasks. No sprint goal has ever been completed by a burndown chart. No progress has ever been made by a progress report.

The purpose of these tools and of transparency in general is to foster team work and to discover areas of improvement for the team. Improvements in performance are achieved not by putting people under a microscope but by creating an ideal work environment for them and by allowing them to focus on their tasks.

A common complaint by software developers about Scrum is that it supposedly creates overhead by forcing them into a sprint structure with many meetings. This is somewhat paradoxical, considering that Scrum is basically an extension of how probably all software developers choose to work when left to their own devices.

When a software developer works on a piece of code, they compile it to see if there are any compilation errors. Then they either run their test suite or, if possible, run their code to see if it works as expected and check debug or log output to determine if anything went wrong. After writing some code, they once again compile the code and check if it works.

Now, how long, on average, is the interval between these compile/test/run/check cycles? Is it months? Weeks? Days?

My assumption is that in most cases, the interval is just a few minutes long and rarely longer. If for any reason, developers are forced into longer intervals, I believe that they become more worried and impatient, eager to finally see if their latest additions to the code are correct.

So why do developers do this? After all, every compile/test/run/check cycle causes a lot of overhead. In fact, it might take longer to compile and test the code than it took to write it. Why introduce such an immense overhead? Would it not be much more efficient to just write all the code and then compile and test it? Coding efficiency could easily double, couldn’t it?

Obviously, that suggestion is complete nonsense. No software developer would write 10,000 lines of code without even compiling it once to see if it is correct. So, developers naturally accept a lot of “overhead” in their daily work. And why? Because they intuitively understand that they are working on a complex problem that requires a lot of intermediate checking, fixing of unavoidable bugs or unforeseeable problems, and potentially replanning.

Scrum is only an extension of this natural way of working from a single-developer methodology to a framework that works in the same way for teams of several developers working together on the same product. And the “overhead” seen by many developers is only a reproduction of the intermediate checking, fixing of unavoidable bugs or unforeseeable problems, and potentially replanning. Any Scrum event serves only this purpose.

I guess the usual problem is that Scrum events are often added on top of other meetings that add little or no value for the software developers, so they feel that the Scrum events are adding overhead to their work. But I believe that all software developers can easily understand that an iterative incremental approach is the most natural approach to software development.

I collected some terms that do not appear in the Scrum Guide – and I have a feeling that to some, these might be quite surprising, as many of these terms have become a part of the folklore surrounding Scrum. Note that this is not meant as any kind of judgement on whether or not these terms or the concepts behind them are useful. Most of them are definitely useful. It is mostly meant as a reminder of how basic and lightweight the definition of Scrum actually is and how much flexibility we have in applying it.

These are some terms not included in the Scrum Guide:

• “User Story” – The Scrum Guide only refers to “Product Backlog Items”.
• “Definition of Ready” or “DoR” – The Scrum Guide states “Product Backlog items that can be Done by the Scrum Team within one Sprint are deemed ready for selection in a Sprint Planning event.” but does not say anything about the need for further rules to determine PBI readiness.
• “Story Points”
• “Estimates”
• “Velocity”
• “Burndown Charts” – While these are mentioned (along with burnup charts and cumulative flow diagrams, they are only mentioned as examples, not as a prescriptive part of the Scrum framework.
• “Ceremonies” – The Scrum Guide only refers to “Scrum Events”, not “Scrum Ceremonies”.
• “Task Board”, “Sprint Board” – Scrum does not prescribe a task board. It is not even mandatory to decompose PBIs into tasks.
• “Three Questions” (for the Daily) – In previous versions, the three questions were meant as an example for a typical Daily. In the 2020 version, the three questions are gone.
• “Stand-up”
• “Set the stage”, “Gather data”, “Generate insights”, “Decide what to do”, “Close the retrospective” – These retrospective phases are actually derived from the book “Agile Retrospectives: Making Good Teams Great” by Esther Derby and Diana Larsen.
• “Approval”, “Acceptance” – There is no process for the Product Owner to approve or accept PBIs as “Done”. PBIs are “Done” when they meet the Definition of “Done” (which could include an approval by the Product Owner, but that is not mandatory).

It cannot be said often enough: Agile methods were designed for complex problems, not chaotic ones.

Just as a reminder – what is the difference between complicated, complex and chaotic problems?

• In complicated problems, there is a direct and predictable relationship between cause and effect. It is difficult to predict the effect from the cause, but with a certain skill set, it is nonetheless possible.
• In complex problems, there is a relationship between cause and effect, but the effect is generally not fully predictable, and the relationship between cause and effect can only be fully understood in hindsight.
• In chaotic problems, there is no discernible relationship between cause and effect. 

Agile methods like Scrum are most effectively applied to complex problems, where short feedback cycles allow us to iteratively gain a better understanding of the complex cause-and-effect relationship.

It is important to consider not only the problem itself but also the environment. Not just the task itself but also the environment can be complex or chaotic.

In a complex environment, outside influences are not necessarily predictable, but there is still a meaningful cause-and-effect relationship. Stakeholders follow the work progress and derive new ideas and requirements primarily from the results of the work. Agile is well-suited for integrating these new requirements into the workflow. The result is a controlled creative process. Looking at the resulting progress, we usually see a „x steps forward and y steps back“, where x is greater than y. This means that in a complex environment, we have to be prepared to rework certain parts of our previous work, but we will see a steady progress, and short feedback loops are meant to ensure that x remains not just greater but significantly greater than y. 

In a chaotic environment, outside influences are not just completely unpredictable – they also are in no way related to the work results. The stakeholders constantly introduce new requirements which are not related to the previously done work. New features or whole new projects appear out of nowhere and supplant previously high-priority work items. In this kind of environment, we also take „x steps forward and y steps back“, but here, x might be greater than y, or y might be greater than x, which leads not just to a lack of progress from the stakeholder‘s point of view, but which also creates a lot of frustration for the developers. This is not a creative process. It is a destructive process. Some people seem to believe that creativity arises from chaos. This is not true. Creativity arises only from complexity. Chaos is a purely destructive force. 

When confronted with the loss of efficiency created by this chaos, stakeholders usually reply: „But we are agile!“ often followed by an „Aren‘t we?“

The big issue here is that stakeholders often do not realize that they are creating a chaotic environment, or they do not realize that agile methods are not going to magically solve the problems caused by arbitrarily changing requirements. They assume that agile methods will fix these problems, because after all, agile methods are designed to deal with changing requirements. And while this is true, agile methods only work efficiently when we are dealing with complex changes, not chaotic ones. 

Obviously, we can easily keep integrating arbitrarily changing requirements into a product backlog. We can change priorities on a daily basis. But one should not expect any kind of efficient output from this chaotic environment only because „We are agile!“ In a chaotic environment, even perfectly built agile frameworks can NOT ensure that x is greater than y in the „x steps forward and y steps back“ equation.

It is critically important to understand this, because when a project eventually fails after months or years of chaos, it is easy to blame the agile methods for not having magically turned chaotically changing requirements into a valuable product. 

Type 1: “Someone in my leadership sphere made a mistake. They are obviously stupid.”

Type 2: “Someone in my leadership sphere made a mistake. What could I have done to support them better? Which tools could I have given them to get the job done? How could I have motivated them better? How could I have communicated expectations more effectively? How could we have collaborated to recognize earlier that something was going wrong? And was that „mistake“ even a mistake, or is that a problem with my own perception? How can I collect honest and open feedback about my actions as a leader from them? How can I help them to see this as an opportunity to improve? Which lessons do I take away from this, and how will I improve as a leader in the future? Which tools do I use to improve and whose advice and coaching support can help me?”

The behaviour of type 2 is often referred to as „Agile Management“, „Management 3.0“ or „Modern Management“. I personally like to call it „Successful Management“.

(Sidenote: In my opinion, the same comment also applies to parenting.)