Many IT managers assume that once they've convinced management and their developers to adopt an Agile process, the hard work is done. Unfortunately, this is far from the truth. I find companies spending as much, if not more, time determining which process (or processes) to pick as they do training their staff in how to use them. The Agile market today is flooded with options - mostly good ones - but inexperienced shops may be overwhelmed. We've reached the point where consensus and consolidation are necessary for this market to scale. Agile teams must pressure their vendors, consultants, and the industry thought leaders to help move the Agile market forward and make it accessible and useful to mainstream IT shops. In the meantime, they must seek out means to work together, sharing their best practices and helping the industry to coalesce.
There is consensus in the industry that to be considered an Agile process, an approach must comply with the core values specified by the Agile Manifesto:
Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
Appropriately, the Agile Manifesto emphasizes the values on the left, but does not deny the value of the items on the right. As we know, there are many ways to support these values and thus Agile teams are faced with the challenges of (1) determining
Advertisement
which processes they should adopt and then (2) determining how to adopt them vis-à-vis legacy development environments. Hybrids are the reality; purist implementations are few and far between. As I wrote in Being Agile in 2007, we must consider "best of breed Agile practice adoption as an option, not a rebellion."
This is the ultimate conundrum: Agile processes are supposed to make development easier, yet the challenges in adoption can easily outweigh the benefits. First, there are way too many processes that fall under the Agile umbrella. Second, it's impossible to compare different processes, as some only address a portion of the software development lifecycle whereas others attempt to be all-inclusive of development tasks. Third, there are few resources to help tie Agile practices to the legacy processes and tools that are so deeply entrenched in IT shops. And finally, many of these processes set out goals or values for teams to follow, but are not so prescriptive as to tell teams how they should do their work. Thus, in every company, you'll find teams implementing different practices from different processes in different ways. And they will all say that they're Agile.
How Many Agile Processes Are There, Anyway?
The answer is: more than you'd think! You can find lists of Agile processes on numerous sites, including Wikipedia and the Agile Alliance, but none of the lists are identical. Interestingly, neither of these sites include Microsoft's recent Agile process "guidance." The processes that most frequently discussed, including Microsoft, are:
Adaptive Software Development (ASD)
Agile Modeling
Agile Unified Process (AUP)
Crystal Family of Methods
DSDM
Evolutionary Project Management (Evo)
Extreme Programming (XP)
Feature-Driven Development (FDD)
Lean Development
Microsoft Solutions Framework (MSF) for Agile Software Development
OpenUP
Rational Unified Process (RUP)
Scrum
Not all agree with this list. Many in the industry argue that RUP, while a strong iterative approach to software development, doesn't belong in the Agile world. The problem here is that there are as many ways to implement RUP as there are companies using it. Those who began using RUP years ago may be whetted to a documentation-centric approach. Recent adopters use as little as possible to say they're compliant without incurring too much overhead. Recent RUP variants, such as OpenUP and AUP, seek to address these challenges and justify RUP's place in the Agile community. In his book "Agile & Iterative Development: A Manager's Guide," Craig Larman does a good job of explaining how to integrate RUP (or the generic Unified Process, UP) with Scrum and XP.
Some of these approaches, such as Evo, predate the Agile community by many years. Tom Gilb published the term and approach in 1976 in his book "Software Metrics." While not widely used by the Agile community, Evo incorporates many Agile techniques including frequent delivery to stakeholders and frequent feedback, so as to adjust requirements for subsequent delivery.[i] In fact, Evo's strong emphasis on business value and measurement would be a strong addition to many Agile development processes.
Where Is All This Going?
In the short term, things aren't going to get much easier. This hearkens back to the days of the early object-oriented methodologies: too many approaches, too many variants, and not much in the way of integration with legacy processes that live on in large organizations. There are, however, some longer-term ideas that may make a real difference.
Industry discussions regarding consolidation among Agile processes continue. As early as 2002, Agile leaders like Ken Schwaber and Martin Fowler explained the benefits of implementing XP and Scrum together, and many predicted that the two popular processes would become one. Many XP shops have adopted Scrum, and many Scrum users have adopted XP or one of the other development-oriented processes, to fit with Scrum's management approaches. We're not there yet, but the conversations continue. If we're lucky, the Agile processes will follow their object-oriented predecessors and we'll wind up with a smaller number of comprehensive approaches.
In the open source community, the Eclipse Process Framework (EPF) team is working towards providing a wide range of process resources to the development community.[ii] Initially, many in the industry argued that EPF did not address the Agile community and was too skewed towards RUP. Actually, OpenUP, the subset of RUP that IBM donated to the open source community, also includes concepts from Scrum, XP, and DSDM. Now that XP and Scrum plug-ins are available, and non-IBM contributors including Xansa (big DSDM users), Telelogic, and Covansys have greater involvement, there is optimism that EPF will truly be able to support the Agile community.
Independent consulting firms and tool vendors have a vested interest in helping to simplify Agile environments and support multi-process organizations. For example, UK-based consultancy Conchango worked with Ken Schwaber and the Microsoft UK Technology Centre and developed Scrum for Team System, a free add-in for MS Visual Studio Team System users.
Help Is On The Way
In the near term, however, Agile teams must forge ahead and deal with the process chaos. Some practical suggestions:
Buy books. This is a great industry for writers and I see new books every week that address some dimension of Agile development. Fortunately, many are written by experienced practitioners who share what they've actually done. A number of them have been featured in Agile Journal issues and we'll certainly keep going.
Build a network. At last year's Agile 2006 conference, I was thrilled to see the level of IT participation (rather than the talking-head focus of years past). The Agile 2007 conference coming in August in Washington, DC promises to have an even stronger practitioner focus and opportunities for peer-to-peer conversations. Agile shops should send a team to the conference so they can absorb as much as possible from others' experiences. The many newsgroups, blogs, and other Websites are also helpful, although participants should beware of personalities and bias' that can hamper a discussion.
Bring in outside help. This is also a great time to be an Agile consultant - there is still a dearth of expertise in the industry. It's amazing to me that with so many large organizations using Agile processes, the "big" consulting firms really haven't demonstrated their credentials. Sure, you can find Agile experience within the global firms, but I still maintain that the best experience can be found in the small and mid-sized boutique Agile consultancies. (Not surprisingly, I approach many of these consulting firms and ask them to submit articles for the Agile Journal.) And, while I've been disappointed in the expertise found in many large Indian offshore firms, the emerging Russian and Eastern European firms are doing some exciting work.
Just try it out. Nowhere are pilot projects more important than when changing the way people work. Piloting is critical for testing out new practices, new tools, new ways of collaborating, etc. Otherwise, how will you learn from your mistakes?
Give back. Through the advent of blogs, newsgroups, and online journals like this, it's never been easier to share and leverage best practices. This can be from a technology perspective, as we see in the increasingly sophisticated open source tools; through best practices, potentially shared through EPF or less formally through newsgroup discussions; and, most importantly, through personal connections throughout this growing industry.
About the author
Liz Barnett is the Editor in Chief of the Agile Journal and Principal Analyst at EZ Insight Inc. Previously Liz spent 10 years as a Vice President and Research Analyst at Forrester Research, joining Forrester as a result of its acquisition of Giga Information Group. Liz held management positions at Accenture, PepsiCo, and Atelier Research. She also was the Research Director for the advanced software development and advanced network computing research services at New Science Associates, prior to its acquisition by Gartner Group. Liz holds a patent for developing a distributed application development/CASE tool. Liz earned her B.S. in operations research and industrial engineering at Cornell University.
In my Doctoral thesis (http://alistair.cockburn.us/index.php/People_and_methodologies_in_software_development) I addressed this question head-on:
"Question 1. Do we need yet another software development methodology, or can we expect a convergence and reduction at some point in time?
Question 2. If convergence, what must be the characteristics of the converged methodology? If no convergence, how can project teams deal with the growing number of methodologies?"
The considered answer I carefully developed was (http://alistair.cockburn.us/index.php/People_and_methodologies_in_software_development#4._Consolidated_Results_and_Reflection):
"The answers to those questions appear directly from the papers and the discussion of the papers in the previous chapter. In short form, they are:
Answer 1: No, there can be no convergence; yes, we will always need another one. There are, however, principles of methodology design that can be applied.
Answer 2: Develop and evolve a project-specific methodology within the project, in time to serve the project, paying attention to the methodology design principles. A technique for doing this was presented."
Yes, people will always be bothered by the confusion of multiple authors describing what has worked for them.
No, it is not possible to eliminate that confusion by converging methods.
Yes, it is required that every person on every software project be wide awake, thinking and learning.
Sorry for the bad news.
(related: I odten don't like the effects of gravity, but I've long since given up dreaming of anti-grav and now spend my time more profitably learning to deal with the existence of gravity, friction, aging, etc etc. As I think about it, the need for multiple methodologies falls into the same category.).
all the best, Alistair Cockburn
1
October 29, 2007
Madhusudhanan A R: ...
Many of the offshore IT services firms, use process frameworks like CMMi, CMM and project management frameworks like PM-BOK with the following objectives in mind.
- To improve predictability of the services. These firms are scaling up capacity and services very rapidly and hence need to keep a close watch on predictability of services and hence the use of formal frameworks, which emphasizing on improving standardization and reducing variability.
- One of the major challenges for firms in the offshore IT services arena is size and depth of the talent pool (specifically in India). Consequently, one of the major risks is exodus of people. It would be next to impossible to adopt an attitude of "working software over comprehensive documentation", when people in a team keep on changing.
There is too much emphasis on agile methodologies in the software industry, but there is no parallel in any other services industry for this kind of approach. There would be very few service providers (even in the non-IT services arena) willing to sign-up - where the service consumer is unable to crystallize his requirements or agree to a fulfillment criteria / acceptance criteria.
The bottomline, is if the Agile approach is not planned well by taking into account all the variables (including the contextual), it is very easy for a development exercise to degenerate into chaos. Agile approach should not be used an excuse for not possessing clarity regarding outcomes and poor planning.
2
June 05, 2007
Steven Gordon: ...
Sorry, there are no silver bullets in software development. The expectations set in this article could only be fulfilled by a silver bullet.
Agile is not a process. It is a mindset/approach based on the principles set forth in the Agile Manifesto. Every project is a different context with different goals and tradeoffs. This will necessarily lead to somewhat different implementations of agile being the most appropriate for each context. Common patterns among the successes have lead to codifying what worked in particular contexts into more tightly defined processes, hence so many different processes. An agile implementation can learn from these common patterns, but merely cookie-cuttering one of them will often fail.
There are lots of good agile coaches available, but engaging one is expensive. Furthermore, the process of fitting agile to a particular context is iterative. It takes time to try ideas, get feedback, leverage that feedback to improve, and repeat. That time multiplies the expense. Also adding to the expense is the culture changes required to become agile.
One cannot cookie-cutter what worked in one context and really expect it to work in another culture. If that were the case, General Motors, Ford, and Chrysler could become profitable by just copying what works for Toyota, yet their attempts at doing this have not worked. What they would have to do is to apply Toyota's underlying principles to their own contexts, but that would take much more understanding, effort, and time, not to mention a lot of change to their culture.
Home 
Many IT managers assume that once they've convinced management and their developers to adopt an Agile process, the hard work is done. Unfortunately, this is far from the truth. I find companies spending as much, if not more, time determining which process (or processes) to pick as they do training their staff in how to use them. The Agile market today is flooded with options - mostly good ones - but inexperienced shops may be overwhelmed. We've reached the point where consensus and consolidation are necessary for this market to scale. Agile teams must pressure their vendors, consultants, and the industry thought leaders to help move the Agile market forward and make it accessible and useful to mainstream IT shops. In the meantime, they must seek out means to work together, sharing their best practices and helping the industry to coalesce.
