Many software companies are adopting Agile methods at an exponential rate - whether the adopting company is a software service or a product development firm or whether the work is on retail domain or avionics. This article covers the challenges that organizations may face and also recommends possible solutions that could help in quickly adopting Agile. The article is based on my experiences while adopting Agile at Valtech India. I've also had the privilege of working with one of the important thought leaders in this field, Craig Larman, Chief scientist at Valtech. Without his constant guidance and coaching, we could never have come so far on the pathway of Agile.
Introducing Agile in an Organization
A typical software services company can be viewed to be made of three layers of people with a varying degree of authority (see Figure 1).
Figure 1: Stakeholders in a typical organization
Over the past five years, I have personally conducted more than fifteen training programs about Agile - both at Valtech and other companies where I have been invited. Developers and project managers form the majority of attendees. During my interactions with them I've realized that it is these people who:
Subscribe to Agile blogs;
Actively pore through many popular Agile mailing lists; and
Have the most passion in implementing Agile.
These people are
Advertisement
eager to learn Agile and look forward to implementing it in their projects. This kind of strategy is known as "bottom-up" Agile adoption.
While this strategy may work in the short run, this is a non-sustainable model of Agile implementation. I am not saying that the developers do a lousy job in implementation - it is simply that they lack active support from the sponsors.
In a top-down Agile adoption strategy, the sponsors take the initiative and encourage (not push) the teams to adopt Agile methods. Some of the Agile practices need investment for procuring tools, changing the physical environment, organizing training for the developers, having a coach, etc. Such decisions are typically taken by the sponsors.
Some of the advantages of top-down adoption include:
Higher visibility - everybody observe the changes happening in the work place and culture.
Confidence - teams can experiment with new practices and tools since they have the firm backing of the sponsors.
Transparency - teams have the opportunity to correct mistakes during initial stages.
Consistency - the practices have a higher chance of being consistent with the company vision.
The Need for Top-down Agile Adoption
Agile is all about people, trust, communication, flexibility, and feedback. These values need a lot of nurturing at all levels in the organization. First and foremost, the sponsors must believe in these values for effective Agile adoption.
In a bottom-up Agile adoption strategy, many of these values cannot bubble up easily when compared to the top-down strategy. The challenges are:
Lack of buy-in when ideas come internally (versus given by an external neutral party).
Generally more difficult and time consuming to convince sponsors by the developers.
Requires lots of sponsor time and good communication skills by the developer passionate about implementing Agile.
Categories of Changes
Figure 2 shows the changes needed during Agile adoption. Each type of change is discussed in detail below.
Changes needed
Requirement from sponsors
Change in thinking patterns and culture
Encouragement, motivation
Investment in new tools
Capital
Sustained support for Agility
Encouragement, motivation and commitment
Figure 2: Agile Adoption Changes
Change in Thinking Patterns and Culture
This is one of the most difficult things to achieve. We, as human beings, resist change and hold on to our old beliefs and thoughts that are close to our heart.
The following thinking/culture changes are necessary for Agile adoption
a. Building trust
b. Getting rid of command and control
c. Improving communication and collaboration
Building Trust
In an Agile training program, a team was discussing about the usage of Post-Its to display project information. A project manager posed this question:
"What should I do if my team members remove post-its without my knowledge?"
This is a clear sign of lack of trust within the project team. In this case, the manager was sent to attend Agile training and implement his experiences on project. But, such managers who lack trust -- a fundamental value of Agile -- will be ineffective in implementing Agile practices. Management should keep observing the teams, identify issues that crop up (they will!), and coach them accordingly. In fact, it is healthy to see lots of issues in this stage - this implies that the system is gearing up for the new philosophy.
The sponsors should encourage an environment for learning for continued Agile adoption. Valtech has taken the following steps in this regard:
Getting help from experts (e.g., Craig Larman spent five months coaching the India team).
Forming an Agile Mentoring team (we have four members at Valtech).
Sponsoring CSM certifications.
The Agile principle "Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done" clearly emphasizes the importance of trust.[i]
Prime Directive and Trust
Here is the "Prime Directive"[ii] which is commonly used during retrospective sessions:
Regardless of what we discover, we must understand and truly believe that everyone did the best job he or she could, given what was known at the time, his or her skills and abilities, the resources available, and the situation at hand.
The Prime Directive just redefines what trust is all about.
A team practicing Agile methods might work collaboratively, trusting each other, but if the customer or sponsors don't trust the development team or vice versa, the foundation of Agile becomes weak.
Getting Rid of a Command and Control Culture
A command and control system is created due to fear and lack of trust. Employees in the traditional organizations are "controlled" through a set of processes. For example, a developer working in a service organization is expected to gather all requirements upfront. The stakeholders create a plan and the developers are expected to stick with the plan and deliver the product as per the deadline. The developers' lives are controlled through a predefined plan written on Gantt charts.
Traditional organizations also create and track various types of metrics. Even though many metrics may be useful, some of them are misused for controlling individuals and teams. Some metrics are also used as a weapon during performance appraisals. (Does this ring a bell to you?)
In a command and control environment:
The Project manager creates all estimates and assigns tasks to the development team.
An Architect handing off his plans to designers and developers.
Developers are restricted from communicating directly with the customers.
Agile organizations, on the other hand, discourage usage of such metrics and encourage only those metrics that adds value to the customers. These include burn down charts and velocity metrics.
In an Agile environment, self organizing teams make the necessary decisions on a project, as opposed to one person making all decisions (as in traditional projects). A single person in control leads to lot of subjectivity and personal bias.
The culture of one person in charge and making decisions for project needs to be dismantled. This can be done only with the support from the sponsors- otherwise self organizing teams cannot emerge.
Improving Communication and Collaboration (C & C)
In traditional systems, the customer's role begins by validating and signing off on the requirements. Later, the customer may participate in periodic reviews done during milestones and monitor project progress. We can equate this behavior to a fire and forget culture, and one that leads to inadequate communication and collaboration.
Software development is inherently a complex process which requires frequent inspect and adapt cycles. This demands strong communication and collaboration -between the customer and the development team and within the team itself.
Developers and customers working together with a commitment to achieve common goals is real collaboration.
The Agile principle, "Business people and developers must work together daily throughout the project" brings out the importance of collaboration.[iii]
Some practices in an Agile method like Scrum that exemplify this principle are:
Release Planning
Requirement workshops during each iteration
Iteration planning meetings
Retrospective session
Iteration Review, etc
At Valtech, we implemented a number of changes to improve communication amongst the team members including:
Removing cubicle boundaries, thus allowing the team to sit together.
Encouraging the Project Managers to sit with the team.
Using of simple tools.
Investing in New Tools; Willing to Get Rid of Old Tools
Agile adoption definitely leads to a position where old tools will have to be discarded and new ones should be introduced. At Valtech, we discarded usage of Microsoft Project Plan and significantly reduced usage of Microsoft Word.
We started using tools like Web Cameras, Confluence Wiki, Digital Cameras, Audio/Video conferencing tools like Interwise, and WebEx, and encouraged dual monitor usage. We also took advantage of whiteboards, cling-on sheets, flip charts, Post-Its, markers, and Red-Green lamps to serve as information radiators (see Figures 3 and 4).
Figure 3: Team with no cubicles and many whiteboards.
Figure 4: Dual monitor usage with information radiators.
Conclusion
After considering the various challenges and solutions during Agile adoption, we have found that Agile adoption is much smoother with continued support from the sponsors. I firmly believe that any organization will succeed in Agile adoption with the right mixture of both top-down and bottom-up strategies.
About the author
Venkatesh Krishnamurthy is an Agile mentor and technical architect at Valtech India, a global software development company. In this role, he mentors and coaches project teams in implementing Scrum and XP practices. Extensive Agile mentoring experience over the past six years has given him the ability to identify the bottlenecks in the company, and suggest appropriate Agile adoption strategies. His expertise in working with offshore Agile teams keeps him busy identifying new and creative practices to scale Agile.
This is really an informative article. We are in the proces of setting up a new Agile team and your article helped us to know the risks and effort needed to build the new team. Thanks Again
1
April 24, 2008
Donarona: ...
I am being too harsh.. but I think it makes sense. I don't think there is matter in the article that conveys what needs to be done for an organization to become agile ready. There is enough matter to market Valteh and the author himself.
The point should be on why to go and implement Agile. Not the reasons like customer and development team collaboration etc. which actually harps everything possible in development.
There are more failure projects/ products than that are successful. We need to address following issues in elaborate fashion: (1) How Agile (scrum etc.) helps you realize ROI faster and cheaper? (2) How Agile (scrum etc.) gives visibility to project/ product stake holders? Here emphasize on what is new and what is not present in the existing system (3) What is that extra thing stake holders get when Agile process is implemented? Emphasize on what they are losing here without Agile (4) Does making changes to seating arrangements really mean success of project/ product? (5) More Importantly Emphasize on how change request management is most well managed in Agile and not in traditional approaches. Thee are many many more items from the developer's perspective but shall take a back bench as developers are the least heard. (Otherwise they don't remain developers)
2
April 22, 2008
VenkataRao Ratakonda: ...
Thanks a lot Mr.Venkatesh Krishnamurthy for this fantastic article about agile. It's very informative.
3
September 10, 2007
Arvind: ...
Sitting plan is a killer for the team, cant even have a chat with wife. article is just another management jargon speller, nothing concrete it has got. appraisers above seems to be working under Mr Venky.
Thnx Arvind
4
May 25, 2007
Narayana.D: ...
Venky,
This is an eye-opener for early agile adoptors.The article has been evenly paced and serves as a stepping stone for individual managers and organizations who are keen and want to explore project implementation using agile methods.
True to your mentioning & like always, I also second your pointer that it is very much neccessary to get rid of the much in vogue, command control environment prevalent in most of the IT companies today :)
Cheers!! Narayan
5
May 14, 2007
Athresh: ...
Venkatesh,
This is a very informative article for organizations contemplating adoption of Agile practices. It is an eye-opener...Yes it has been my experience too at Siemens Health care division that without buy in by sponsors and their continued support it is next to impossible to mobilise teams and move the agile way...
Thank you.
6
May 10, 2007
Ramesh R: ...
Thanks Venkatesh,
This is a good article giving a good insight into Agile adoption need and benefits.
Articles 
