No Agility Without Retrospectives

A popular interview question for Scrum Masters is: “Which one is the most important Scrum event?.”

Obviously, the purpose of the question is not to actually mark one event as the “most important” event, but to see the candidate’s thought processes. There is no right answer, and there is no good answer.

I have nevertheless thought about this question quite often. And if I were forced to pick one event, it would be the retrospective.

The reason is that the retrospective is in my opinion the least likely event to happen “by accident” and it is not easily replaced by other tools or methods. It is a human impulse to ignore the need for improvement when the pressure is high to deliver results. A formalized retrospective gives a team the space to think about improvements, even if the customer is asking for everything to be delivered “tomorrow”.

Also, if I had to develop a plan for a staged approach to an agile transition (instead of the “big bang” strategy many organizations seem to prefer), I would probably start by introducing regular retrospectives for all teams. So in this approach, retrospectives would exist even before the introduction of full-fledged Scrum.

I actually believe that the omission of retrospectives in supposedly agile organizations is an underestimated but highly critical antipattern. Often, retrospectives are not the last but the first meetings to be cancelled or postponed when other meetings seem more important. However, this is one important sign that an organization is missing basic aspects of agility, that the agile mindset is not well-developed and that the organization follows agile methodologies as a cargo cult.

Process Diversity

When people speak about diversity in the workplace, they usually refer to building teams from people of diverse backgrounds. However, I would like to write about a different kind of diversity that can be highly beneficial: Having a diverse set of approaches for how to organize the work within an organization – or to use a shorter term: process diversity.

To illustrate the meaning of this term, I would like to describe two different fictional companies going through an agile transition:

Company A took a very structured approach to their agile transition. They spent several months thinking about how to organize the teams, how to distribute the work, how the teams should interact with each other and with the various stakeholders. Their project handbook clearly describes how teams are expected to conduct their Scrum ceremonies and what kind of output is desired from them. It also contains a template for a task board that the teams should use and explains how to use the chosen software tool to make the task board available in digital form.

Company B did not care quite that much about coming up with such a detailed process description. There is great variation between how the teams organize their work in Company B. Some use Scrum, while some use Kanban. The task boards in each team look completely different. Some teams use burndown charts, others use burnup charts, while others use neither. Some decided to transfer their task board 1:1 into the software tool, while others track only the higher-level user stories there. Some teams do their dailies in the usual “Everyone answers three questions” fashion. Other teams decided to use other less presciptive formats. One team decided that they want their Product Owner to take an active role in their daily, but other teams have clearly decided against that.

So which approach do I prefer? Personally, I strongly lean towards Company B’s approach.

I like to explain my reasoning by pointing to biological evolution. The only way how species could develop new useful traits is through a diverse gene pool which is caused by mutations. If all members of a species were identical, there would not be any kind of evolution, and the whole species would either succeed in its given environment, or the whole species would fail.

I believe it is the same for companies. Evolution of positive ideas, practices and frameworks is more likely to come from a diverse process landscape. For example if every team structures their task board differently, then the teams can learn from each other, adapt good ideas from other teams and avoid making the same mistakes as other teams. I strongly feel that Company B will have huge advantages over Company A in the long run, even if the differences between the teams might lead to problems every now and then.

Obviously, wanting to benefit from process diversity implies that teams are willing to learn from each other, and you will have phases where teams converge towards common approaches. But the teams should always be encouraged to diverge again by running various experiments to enable another round of process diversity followed by collective learning and process convergence.

Why do Agile Coaches Love Post-Its?

Often, people joke about an agile coach’s affinity for post-it notes. And it’s true – it is difficult for me to imagine how I would do my job without post-its. It hardly ever happens that I do not have a pack of post-it notes within an arm’s reach, and it hardly ever happens that I moderate a meeting without using any post-its.

However, I believe there is a deeper meaning behind this:

Once, I was told that the meeting moderator’s pen is like a modern scepter. Whoever holds the pen in a meeting also holds the power over the meeting. Whoever holds the pen is the primary decider when it comes to what will be written down and how it will be written down. There are many ways in which a moderator can (more or less) subtly manipulate a meeting to elicit a preferred result, simply by presenting the results in a certain way.

This changes completely when the moderator hands post-it notes and pens to the meeting’s participants. This is effectively a handover of power, and the moderator fully retreats into a facilitator role. And this is a deeply agile concept – to not hold on to power but to give it to the team.

So I actually believe that the heavy use of post-its is just another expression of an agile mindset.

The Agile Burden of Proof

In my opinion, the worst thing agile coaches can do is to introduce practices they cannot motivate. And by “motivate” I do not mean that they can recite the relevant passage from the Scrum Guide or from some book. By “motivate” I mean that they actually reach and convince the people they are dealing with.

The burden of proof that a specific practice is useful is always on the agile coach.

Saying “It’s done like this, because we do Agile/Scrum/Kanban/Whatever” is only marginally better than saying “It’s done like this, because I say so.” If we accept these as valid arguments, then we also have to accept “It’s done like this, because we always did it this way” as a valid argument – and it is a common argument of people who oppose Agile, because they want to hold on to their old processes.

If someone asks “What is a retrospective good for”, we should have a meaningful answer and not just something pre-packaged from the Scrum Guide, such as “It’s an important inspect-and-adapt meeting”. That is like saying “It’s important, because we do Agile”, because after all, Agile is the reason why we run frequent inspect-and-adapt meetings. But why do we run frequent inspect-and-adapt meetings in Agile? We need to be willing and able to dig quite a bit deeper here. I would recommend using the 5 Whys method here before you even go into the discussion with your team.

Yes, that makes life more difficult for agile coaches, because there will often be some resistance, and often, we cannot even conclusively prove that something works until it has been tried. But the important word here is “try”. I do not see a problem with introducing a practice explicitly as an experiment, stating right from the start that it is a time-limited experiment while encouraging feedback and making sure that the experiment is properly documented and monitored. That is the agile way for handling this: If we emphasize the importance of empirical product development, then we also need to practice empirical process development by running our “great” ideas as experiments and not as decrees that are not open for discussion.

Burn Those Burndown Charts!

Transparency is one of the key ingredients of Agile, and burndown charts are widely seen as one of the most important tools for achieving this transparency.

My problem is: I hate burndown charts.

Everyone is completely hung up on those charts now. Scrum Masters end up having conversations like:

  • „Let’s talk about your team’s progress. Where is your burndown chart?“
  • „I had a look at your burndown chart, and it didn’t look good!“
  • „You DON’T have a burndown chart? Uh-oh!“

The first two are things you often hear from project managers. The third one came from a Scrum Master, and I could immediately tell that it made him judge me: „No burndown chart? Such a bad Scrum Master!“

I would go as far as proposing an „Agile Mindset Test“: When someone enters a team room, do they walk over to the burndown chart, or do they walk over to the task board? I think their choice says a lot about their agile mindset.

I admit, it is of course a somewhat natural reaction to walk over to the burndown chart first. People prefer an image over a wall full of text. But does that image truly represent what the team is doing and how it is doing its work? No, of course not! And that is one big problem with burndown charts. Of course, a flat burndown chart indicates that there could be some kind of problem, but that is where it has already reached the limit of its usefulness. There could be some kind of problem. That’s it.

My other problem with burndown charts is that they are simply non-agile waterfallish artifacts to me. One of my former companies used „race charts“ towards the end of the projects (i.e. in the test phase, as these were waterfall projects). These race charts (you also could have called them “burndown charts”) showed how many bugs of a certain type were still open and by when that number must reach zero. There were two issues with this: First: New bugs were constantly being discovered, but that was not reflected in the race chart. They simply made the curve look flat, or often the curve even went up. Second: The completion date was arbitrary and had nothing to do with what we COULD do – it was only related to what we HAD to do. We were in a situation with fixed cost, scope, quality and time.

Many people would assume now that in such a situation, teams would give up quality first. But actually, the first victim of this scenario was not quality but common sense. As it was usually impossible to reach the goal, a lot of effort was put into making the statistics look nicer, and a lot of items were simply pushed out of the race chart, resulting in a „not my problem“ attitude. In general, the chart did nothing to create a positive atmosphere, it did nothing to make people work better, it did nothing to motivate people and it certainly did not create any kind of transparency. It only created pressure and killed common sense. I believe that burndown charts in general tend to kill common sense.

I know, burndown charts are seen as a useful tool for retrospectives. „Why was our burndown chart flat for several days during the Sprint?“ is a valid question to ask. But to ask that question, does the chart have to hang visibly in the team room during the Sprint? Oooh, heresy, I know! We are supposed to be transparent. But there are other, better tools to be solve the same issues a burndown chart is supposed to solve and to create the necessary transparency for that.

For example, if tasks tend to spend several days on the task board, this would result in a flat burndown chart. But if you commonly have that kind of problem, I feel it is much better to put some kind of marker on the tasks after every Daily. That way, you can see which tasks are not moving forward, and you can address it immediately. When you speak about a flat burndown chart in a retrospective, a lot of people will not remember anymore why exactly it was flat. In fact, some tasks might be moving quite quickly, while others are stuck in „In Progress“ for several days, so the chart looks fine, but the real progress is perhaps not fine at all.

Another counter proposal to burndown charts are burnup charts. This is not intuitively clear to many people, as burnup charts seem to be upside-down burndown charts and nothing else, but in my opinion, the difference is huge!

The major difference is that burnup charts always represent the work that was done. If you completed a task, the burnup chart „rewards“ you. In a burndown chart, if you complete a task, while another team member adds an additional task that was missed in the Sprint Planning, then the chart remains flat. Is that supposed to be motivating? If anything, it motivates people not to add new tasks, even if they would be needed. I have heard the statement „This makes our burndown chart look bad“ quite a few times. Is that what we want?

Burnup charts always go upwards, towards the goal. They very rarely go down (only if you have to put a task back from „Done“ to „In Progress“, which is hopefully a rare occurrence). And what is at least as important, they not only visualize the progress towards the goal, they also visualize the goal itself. Of course, the general idea is that the scope of a Sprint should not change, but let’s face it: It often does, and the burnup chart visualizes this. What’s more, if the team misses the Sprint goal because it was forced to add additional work to the Sprint, then the burnup chart can immediately prove that the team did as much work as it had predicted, even if it did not reach the original goal.

Take the following burndown chart as an example:

burndown

Here, the team realized twice that new tasks have to be added to existing user stories. This causes the burndown chart to go up or to remain flat. This is a situation where teams are often reluctant to add new tasks because it “makes the burndown look bad”.

In the middle of the Sprint, the team took another story into the Sprint, because someone claimed that it was terribly urgent. And of course, this should not happen. But let’s be realistic: It does happen. And thanks to these additions to the Sprint Backlog, the burndown looks very bad for the first eight days. That is the point where nervous project managers drag the Scrum Master and Product Owner into their office or, even worse, start pestering team members about their bad progress.

Here is the same scenario presented in a burnup chart:

burnup

The red line at the top visualizes the total number of tasks, while the blue line visualizes the progress. The first noticeable point is that the blue line always goes up and that the team actually worked at a reasonably constant pace. If you only have the burndown chart, then the first question in a retrospective would be: “Why was our burndown chart so flat?” If you have the burnup chart, you can ask the much more meaningful question: “Why did we end up adding tasks? Do we need to improve something about our Sprint Planning?” You get to the point much quicker and you do not waste so much time on trying to interpret a chart.

Also, you can see immediately that the blue line would have reached the red line if a new story had not been added. So the team’s original plan for the Sprint was definitely feasible. What caused the Sprint Goal to be missed was the new story that was forced upon the team.

So I will continue to argue for the eradication of burndown charts. Sadly, all software tools used for agile planning (such as JIRA, TFS) spit out burndown charts, and people will continue to look at these charts. We cannot influence that. But at least our teams and we as Scrum Masters still have the power to decide how we want to visualize our progress in our team rooms!