Second of a three part series...

In this second part of our article you'll find the next three challenges we identified. The experiences shared in this series of articles come from working in large waterfall oriented enterprises and governmental departments. In our function as hired Scrum coaches from Xebia we gathered data from projects having between thirty and sixty people. The systems all comprised multiple platforms and technologies and often had a service oriented architecture.

Click here for part one.

#4 Defective Product Owners
We see that it's hard to find capable product owners. When a product owner is eventually found, we frequently see that this person doesn't have the mandate or knowledge to do the job properly.

A product owner's main responsibility is getting the most valued functionality at a certain date within a certain budget. The product owner therefore must be able to prioritize work and decide what functionality to put into a release, maybe removing obsolete functionality and adding new functionality. The product owner also has to answer questions about functionality that arise during the course of the project, signing off sprint results, and reporting to upper management. In order to do his job properly the product owner needs the right authority and knowledge about the business, the problem domain, and the envisioned product and Scrum process.

A defective product owner can heavily decrease the delivered value at each release and the team performance. We often see that the product owner is not the budget holder and cannot decide what functionality to put in a release. As a result decisions cannot be made quickly enough and throughput stalls as teams must wait for a decision to be made. Another thing we see is that product owners do not have the right understanding about the product requirements to be able to make the "right" decisions on requirements prioritization. The question "what is the business value of the product backlog items?" often can not be answered in any meaningful way.

To cope with this make we found that having a customer or product owner team consisting of people with the right knowledge and authority is fundamental. From the business you'll need to have marketing and business analyses to clarify requirements and provide input of market research activities so that the right prioritization can be made. If you do not have the right authority on budgeting then you have to get the budget holder in the customer team so that decisions can be made quickly.

From IT you'll probably need Architects and requirement engineers to help prioritize and prepare the product backlog. And of course actually having a user representative is not a bad idea. Note that even for small projects a (small) customer or product owner team can be very useful.

Starting product owner teams are faced with lots of new things and often need to change their view and way of working. A different way of planning and handling requirements is needed, new ways of measuring progress and a different way of managing. Because of this and because the product owner is such a critical role in the Scrum process, the adoption process will greatly benefit from extensive coaching the product owner team and development teams.

#5 Doing agile strictly and only by thebook
Now here's a strange one. When adopting Scrum one of the things we teach is: Scrum is a simple process with not all that much obligated steps in it: don't cherry-pick and stick to the rules. How is it then that we indicated "Doing Agile strictly and only by the book" as a possible implementation pitfall? The catch of course, is in the wordonly.

One other thing Scrum literature will often tell you is that Scrum is a simple process with complex behaviour, meaning that the process itself is indeed simple. There are just a few roles and there are simple process steps and practices to follow.

We have seen numerous problems occurring when people started doing things exactly as specified in the books. The book says: "estimation should be in gummy bears", "we can have only one product owner", "burn downs should be in hours", "everything should be on the product backlog and prioritized according to business value", etc. We see a lot of dogma. People act according to the book, see that things do not work in their specific situation but do not adapt their behavior because the book tells them to do it in that specific manner.

Please note that in the end it is all about achieving the right results. The behaviour we would like to see from team members, product owners, ScrumMasters and all stakeholders should lead up to that result and is much more complicated than you'd first expect when studying the process. This complex behaviour is excellently captured by a series of pattern languages by James O. Coplien and Neil B. Harrison in their book "Organizational Patterns of Agile Software Development". A fair number of patterns they've described in this book can directly be "mapped" onto Scrum. The point is; you can read the book and follow the mechanics of the process, but that will not really get you where you would like to be.

We experienced that you should do what best suits your particular situation even if this means not following all the details of the books. If you literally follow the book you could, for example have potential shippable product at the end of every sprint. If you are exploring solutions you know you are not going to ship your product at the end of a sprint. If you know you are building a rough version, validating it and slowly build up quality, there is no need to have a potential shippable product meeting all company standards at the end of each sprint. So, if you know all this, then don't build it like the book says!

Another thing you could infer from "the book" is that sprints should start before specifications are complete. What if agreeing and clarifying on specifications takes a couple of weeks thanks to the multitude of stakeholders and system complexity? And what if finishing them during the sprint will not give you enough time to prepare the specifications for the next sprint? Well.... complete them as much as possible before the sprint starts or maybe build up a small buffer of specifications.

You will need books to help teach the mechanics of the process and to strengthen you're overall knowledge about Lean amp; Agile in general and Scrum more particular. This is especially the way to go when you are in the process of learning and adopting Scrum. But also realise that at some point following the books are not going to help you take the really important steps. You'll have to understand the underlying principles and get to the upper stages of the Dreyfus model of skill acquisition; the true meaning of Do, Check, Inspect and Adapt does not fit into a book. So start out using the books to get yourself going and once you are competent forget about the books and adapt according to the principles of Agile and Lean.

#6 Not preparing the organization around a Scrum project
When introducing Scrum into an organization you cannot look at one department or one team and discard the rest of the organization. It doesn't matter the team at hand is enthusiastic; you will need to cooperate with other parts of the organization to really make a difference. You will need to cooperate with sales and marketing to handle product ownership, test, design and maintenance departments to involve their people in the team and management to help them in this new way of tracking and managing projects.

Lack of understanding of how Scrum provides benefits can get you in lots of trouble. New insights in requirements may be interpreted as lack of upfront requirement engineering. Re-architecting may be interpreted as improper architecture or incapable architect and so on. In case the surroundings of the project do not understand Scrum and how it can benefit them and the project you'll encounter lots of resistance.

Overall you will have to make sure people get this Scrum thing and how it concentrates more on working software than on proving how far the project is along by showing lots of plans and other documentation. They will need to get used to the fact that if the date and the budget is fixed, the functionality must be variable.

When introducing Scrum you quickly should start working on some success enablers. You need to have an Agile foundation setup. Scrum does not give you a capable product owner, customer engagement, teams practicing suitable engineering practices or good requirements management. These should already be there.

It would be best to get active support and commitment not only from company level management but also from various department managers and team leaders you're project or department needs to interact with. The best way to get this support is by showing them the advantages of introducing Scrum will bring not only for your team or department, but also for theirs. In most cases you have to tell them several times, or better yet; show them. Showing people some quick improvements should gain enough trust to be able to do the next steps.

 In time, failing to prepare the organization around the project will result in local optimization at the most. Even worse, it will probably result in another project that fails and Scrum will take the blame for it.

 Wrap-up
In this second part we discussed some big challenges that have to do with various aspects of adopting Scrum. A capable product owner team is fundamental to achieving success but not enough on its own. If you want to achieve global improvement you'll need to align with other departments. In doing so you'll have to learn and really understand the principles behind agile and Scrum so you can inspect and adapt to what best suites your specific situation.

Click here for part 3.

About the Authors

Eelco Gravendeel is an consultant for Xebia with the unit Xebia Agile Consulting. Eelco has started out his career as a software developer. Later moved on to become a team leader and then became the manager of a Software Development department. During this period he implemented Scrum the first time. Having tasted the Agile fruits he then moved on to become a Consultant and Project Manager. In this position he coaches departments and organizations to implement Scrum and Agile practices or manages Agile projects.

Cesário Ramos is a Senior Consultant in Agile methodologies and Enterprise Java at Xebia. Since the late nineties he has worked in various roles ranging from developer to agile coach. He has coached various projects at telecoms, financial and governmental institutions using popular methods including Scrum, XP, FDD and UP.