Agile Journal - The Trouble With Retrospectives

Home

Agile Journal

Challenges with Distributed Agile - May 08

The Trouble With Retrospectives

Written by Allan Kelly

Tuesday, 08 April 2008

Within the Agile community retrospectives are widely seen as the mechanism for promoting learning and change. But many teams fail to hold retrospectives, or fail to act on the findings, thus they fail to learn and improve. If we are going to fix this we need to change our approach to retrospectives, and find new ways to learn and create change.

On the face of it the idea is simple: periodically take some time to reflect on and discuss recent work, then agree improvements. With so much agreement over a relatively simple idea why do so few retrospectives happen?

An Advanced Technique
In reality retrospectives are one of the more advanced techniques in the Agile toolbox. One we can only use when we have some experience and success with other techniques, and when our organizations are mature.

The most common reason given for not doing a retrospective is lack of time. True, we are all busy people, but if we really value retrospectives the way we say we do, surely can we make time? Finding the time is really a question of prioritization, deciding that the retrospective is more important than other things.

The ‘I don't have the time' excuse is hides deeper reasons. It is a convenient explanation for something most individuals and organizations just don't want to do. The underlying problems are a fear of what might be found and a lack of trust between colleagues.

As individuals, we fear retrospectives for what might be uncovered, or what might be whitewashed. We intuitively have a good idea of what worked and what didn't work but we fear exposing these problems. Perhaps we skimped on testing, perhaps someone wasn't pulling their weight, or perhaps the business never really knew what it wanted. Facing up to problems requires courage.

Our companies fear retrospectives too, largely for the same reason: they fear what might be exposed. The business may well know it had an ill-defined idea when work started, but once the project had been approved nobody wanted to point out the Emperor had no clothes. Admitting a problem exists implies it should be fixed, which means finding time, energy and money - in other words more work.

All too often trust is missing. Individuals don't trust the corporation, the corporation doesn't trust the teams and managers are stuck in the middle being pulled in all directions. Without trust it is hard to be open and face up to problems.

Change
Perhaps the only thing worse than not having a retrospective is having one and not being able to follow through to implement the suggestions. Teams that are brave enough to hold a retrospective and face up to problems quickly become frustrated and disillusioned when they are blocked from fixing the problems identified.

Which brings us to the second big problem with retrospectives: they don't bring about change. Organizations that have tried retrospectives feel good, the exercise is cathartic. Once we have exposed the problem and think we know how to solve it, we have renewed hope and energy. But if nothing happens frustration results, and the next time someone suggests a retrospective we think ‘Not again!'

One team at a London based software company used to conducted regular retrospectives but the team was not improving. When asked the team manager said: ‘I've stopped going. They find the same things every time and nothing happens.'

Making change happen is difficult. There may be technical problems, time problems, or the need for support from outside the team. Perhaps we need money for training, an extra person to fix a specific problem, or some other expenditure that is unlikely to get approved.

Promote Learning
Tackling these problems requires a two-pronged approach. Firstly we need to find new ways of learning and improving apart from retrospectives. Secondly we need to adjust our approach to retrospectives.

Change starts with managers who are responsible for improving things. They are given a little authority to help get things changed but authority is not the answer to all problems. Before managers can change anything they need to know what to change and how to change it.

Rather than diagnose problems and suggest solutions managers should ask the people who actually do the work and face the problems everyday. Team members normally know the solutions already. All it requires is the time to ask and the ability to listen. Unfortunately, it seems, too many managers only talk to one another and do not spend enough time listening to those who actually do the work and face the problems.

One way of understanding problems is to try and explain them to another person. Even explaining it to yourself can help and maintaining a journal can help here. You don't need to write in it every day, or record everything that happens. Just taking some time each week, to record major events and explain in words what has been happening will help improve understanding and inform decision making.

There are times when problems and solutions are better discussed in groups rather than on one's own or in one-to-one conversations. Such forums do not need to be retrospectives. It helps if the group meets regularly and if there is a hook to base the discussion around.

One manager used to convene monthly improvement meetings. She would take a specific subject, say code quality, and base a discussion around that. At the end of the meeting, the group would have a collective agreement on what it wanted to achieve and how.

Another group at the same company formed a book study group. The group would meet every second week and work their way through a relevant book. Over time participants increasingly applied ideas from the books to the company. The book acted as a seed to help the group collectively identify problems and solutions. In time the book became less important and the discussion more important.

Back to Retrospectives
Techniques such as these have both direct and indirect benefits. Teams learn directly from these excises and their collective learning leads to change. These exercise also help build an environment where learning is valued, people can talk freely and trust one another. When individuals can talk openly about serious issues, they will feel confident raising issues in public and they will understand when such issues are better kept private. Thus some of the barriers to effective retrospectives are removed.

There is no sure fire way to ensure that recommendations for change are acted on after a retrospective. It gets easier if we have already identified and fixed some problems, and over time we gain confidence in our ability to do it again.

Although it may sound counter-intuitive, one technique is to deliberately limit the scope of the retrospective. Placing issues we cannot address out-of-bounds allows attention to be focused on issues that can be fixed. So we might accept that requirements come in large documents not on cards, or that the Test Manager wants a two-week code freeze before release. When a team is confident with retrospectives and has a track record of improvement these issues can be tackled.

We can also limit the outcome of the retrospective. Trying to change many things at once dilutes our ability to change anything. So rather than try and implement 10 changes, we can focus our attention on the top one or two.

Once you have chosen your thing to change talk about it in the retrospective, discuss what you mean by this change, talk about how things will be different, what obstacles you might encounter. Get agreement on exactly what you mean, how you will do it and what the outcome will look like.

In one retrospective a team agreed that the number one thing that needed to change was unit testing. There were no unit tests and everyone thought automated unit testing would help, so they agreed to start doing it. But when they talked about specifics it transpired that everyone had different ideas of what ‘automated unit testing' meant and how to do it.

Broader Change
While the team needs to focus on a few specifics and changes at a time the team manager needs to take a broader view. They need to be conscious of what issues need to be beyond scope and which are can now be tackled. Mangers need to work a step ahead of the team, preparing the ground for future.

When a team holds a retrospective, and when it implements changes and improvements, it should be encouraged to present its findings to a wider audience. Not only does this spread the learning but it builds trust, helps reduce the fear of retrospectives and demonstrates they can, and do, work.

The hallmark of a true Agile team is that it is learning and changing but retrospectives are not the only tool for change. Retrospectives are an advanced tool and like all advanced tools you can't just pick them up and expect instant results. Other tools can help learning and build towards the goal of effective retrospectives.

About the Author
Allan Kelly is a London-based consultant and interim manager specializing in Agile adoption. His first book, "Changing Software Development: Learning to be Agile" was recently published by John Wiley & Sons. He is a qualified Product Manager and Project Manager, and holds a BSc in computing and an MBA in management.

Comments (0)