Agile development has been used effectively for a number of years, but its
adoption has primarily been limited to pockets within organizations. Lately,
adoption has begun to grow in industries such as insurance, telecom and
financial services; but for agile development to become mainstream, it needs
backing by major industry players, a unified rather than scattered agile
landscape, and evolutionary rather than revolutionary agile transformations. We
need improved support for large-scale agile transformations, including knowledge
transfer on a large scale, necessary HR policies, and scalable agile tooling
architectures. Finally, we need to address development issues that large
organizations deal with, including large-scale development projects,
geographically distributed development, and compliance. Let's take a look at
what promising work is happening in each of these areas.
Listen to the Podcast
In a three part interview with Agile Journal Editor in Chief, Liz Barnett, Per
Kroll discusses how Agile development has crossed the chasm to become
mainstream.
Mainstream Companies Are Looking For Safe Bets
In Crossing the Chasm,[1] Geoffrey Moore
points out that for a product to make the transition from "bleeding edge,"
early-adopter technology to wider marketplace acceptance, it needs to appeal to
a growing majority of consumers. Mainstream organizations will go for what they
deem are safe bets rather than venture into unknown territories. They will go
with the market leader rather than hottest technology and seek continuous
improvement rather than radical paradigm shifts.
Consider how Moore's observations relate to making agile development mainstream:
Backing by major industry players
A cottage industry of players have entered the agile market, including services
companies such as Object Mentor, Mountain Goat Software, Net Objectives, Quadrus,
and Industrial Logic, and product companies such as Rally Software Development
and VersionOne. Most of these companies have small staffs of less than one
hundred people. Until now, the presence of major services and product companies
has been lacking. This is changing as we speak.
For example, the world's largest system integrators have recently put more focus
on agile development. IBM Global Business Services just launched an Accelerated
Solution Delivery Practice focusing on agile development and agile
transformations, Capgemini is investing in an accelerated delivery platform for
agile development, and even a few Indian system integrators, such as Cognizant
and ITC Infotech India, are moving towards agile development.
Major vendors are investing in agile tools and processes. IBM has recently
previewed Jazz, a next generation technology platform for, among other things,
agile and collaborative development. Microsoft recently launched MSF for Agile,
an agile process, and IBM, Telelogic, Capgemini, and 15 other large and small
companies are co-developing the Eclipse Process Framework (EPF),[2] an open source process
initiative focusing on agile processes.
Unifying a scattered agile process landscape
As agile development is growing in popularity, so is the number of agile
processes. These include XP, Scrum, OpenUP, AUP, DSDM, Lean Software
Development, Adaptive Software Development, Rational Unified Process (RUP), MSF,
FDD, Crystal Clear, EssUP, and Agile Modeling.
Advertisement
Many of these processes cover only certain aspects of the software development
lifecycle. As an example, Scrum only covers project and requirements management
aspects and needs to be integrated with other processes such as XP and Agile
Modeling to address the full lifecycle. Integrating different processes is
difficult and time consuming, and mainstream companies do not want to take on
that investment.
The EPF project, however, addresses the process integration issue. EPF is an
open source initiative providing an authoring, configuring, and deployment
platform for software best practices. The project was launched earlier this
year, and many leading agile processes are, or will soon be, available within
EPF, including OpenUP, XP, Scrum, DSDM, and Agile Modeling.
Within EPF, we hope to leverage all of these agile methods to eventually develop
reusable agile process components, which can be combined to produce different
agile processes, such as OpenUP, XP, and Scrum, or to produce your own agile
process. Organizations should be able to add or modify process components, use
them internally, or make them available for free or sell them.
Evolutionary or revolutionary?
Agile development is often presented as an abrupt paradigm shift-a scary
proposition for many companies. Agile transformations can be a continuous
journey. You may transform additional projects every quarter, and each time, you
learn and improve your transformation process further. You may introduce
practices a few at a time. For example, you may start with iterative and
test-driven development, and then follow up with pair programming and collocated
teams.
Certain agile practices, such as self-organized teams, do represent true
paradigm shifts. There is a big difference between allowing team members to
influence decision making, and having team members truly control their own work
and destiny. When a team is faced with a difficult choice, and team members turn
to their manager for a decision and the manager replies, "I'm going out for a
coffee, let me know what you decide," it creates a radical shift in people's
understanding of who is in the driver's seat. This can lead to a new level of
collaboration and motivation, generating an often radical productivity boost.
We also need to articulate what organizational factors are impacted by an agile
transformation, and what commitments are required. If you are not prepared to
revisit, for instance, how you approach testing, how you motivate and reward
people, how your IT organization is collaborating with your line-of-businesses,
and what tooling infrastructure you have, then full-scale agile transformations
are not for you.
Dealing with large-scale organizational transformation
You can transform the way ten people work by having a skilled agile coach
spoon-feed information to the right people, but transforming an organization of
hundreds of people will also require the availability of a reliable
infrastructure. Let's have a look at some of the issues that need to be
addressed.
Knowledge transfer on a large scale
As consulting firm Industrial Logic was tasked to do an agile transformation of
a 600-person development team at Kronos, it realized that just relying on a
handful of coaches would not cut it. To help transfer basic knowledge about XP,
the team built Web-based training courses, which allowed its coaches to focus on
the higher-value activity of coaching teams on the practical application of XP.
Large-scale adoption requires complementing traditional coaching models with
electronically transferred knowledge, whether it is through Web-based training,
process guidance, tutorials, or other means.
Changing HR policies
Agile development holds implications for traditional HR concerns, such as
employee incentives and rewards, career paths, and responsibilities. We need
early support from HR departments and managers in carrying out the necessary
changes.
Agile development requires close collaboration among a set of people who regard
each others as peers, which can change the dynamics associated with team
leadership. Typically, a job promotion (from developer to architect, for
example) is seen in larger organizations as a means to escape the dirty job of
writing code. But these organizations need specialized and experienced people
who serve as mentors in their team-lead roles, not as supervisors who have
earned the right to sit on a pedestal.
A focus on collaboration also changes how you reward and measure people. Rather
than measuring only individual productivity, you need to measure how much value
an individual provides to the team. The most valuable team members often have
lower individual productivity, since they spend most of their time mentoring
other team members.
These HR-related notions reflect agile values, and it is crucial that an
organization articulates the values it wants to live by in rewarding and
promoting its people.[3] For agile development to
become mainstream, we need to better articulate required HR changes and how to
accomplish them.
Providing a scalable tool infrastructure
A key agile principle is to focus on individuals and collaboration over tools,
and possibly as a result, many agile coaches are tool adverse. However, tool
infrastructures can greatly facilitate collaboration and are crucial to making
agile mainstream. True, some traditional tools may not be conducive to agile
development and may inhibit meaningful collaboration. But there is a promise on
the horizon-a next generation of tools focused on the collaborative aspects of
software development is being developed.
VersionOne and Rally Software Development have launched agile project management
tools, providing support for core agile concepts such as velocity, iteration
planning, size and effort estimates, as well as breakdown of work into small
units that can be accomplished in a couple of days or less. The IBM Rational
organization, in collaboration with IBM Research, has recently previewed
technology code-named Jazz, which goes beyond current agile project management
solutions to include also agile development support, adding agile concepts such
as team collaboration, transparent development, continuous integration, and
test-driven development as first-class citizens. Jazz is also process-aware and
can be tailored to support different processes.
Large-Scale and Corporate Development Issues
For it to become mainstream, agile needs to address the problems facing large
organizations, such as support for geographically distributed development,
large-scale development, and compliance. Current agile solutions do not clearly
articulate solutions to these issues, and this needs to change.
Large teams and larger organizations
Many agile processes explicitly target smaller teams of a dozen or fewer highly
skilled developers, but more experience and guidance needs to be provided for
dealing with large teams. Support for larger teams is in development for
processes like XP, Scrum, OpenUP, RUP, and Agile Modeling[4]. For example, IBM is currently
refactoring RUP to be built as a set of extensions to OpenUP. This will add
guidance for domains such as SOA, compliance management, geographically
distributed development, and packaged application development to an agile core
process.
Geographically distributed development
Geographically distributed development has become a matter of life in most
larger development organizations. But face-to-face communication is a core
principle of agile development. Organizations can compensate for the lack of
shared physical project room through a combination of modified practices,[5] such as more effective
distribution of work across different teams, and improved supporting
infrastructure. Jazz technology provides team awareness, allowing team members
to see who is present and what they are working on. Various technologies such as
instant messaging allow collaboration in the context of the work done. The goal
is to leverage available technology to create a cohesive team, even if team
members are miles apart.
Compliance
Compliance means Say what you do, Do what you say, and Show you
did it. This is not inconsistent with agile development, but there are a few
hurdles to overcome.
First, the agile community needs to overcome the inherent fear of documenting
processes. Agile teams should continuously evolve the process to ensure it works
for them, but the fear is that when you document the process, it becomes hard to
change. EPF and RUP address this concern by separating a rich reference library
of practices from a succinct description of how the team chooses to work in
their specific project. There is no need for massive changes to elaborate
process guidance; instead, teams change their short process description and
associated pointers to reference material. In 2007, EPF will add Wiki technology
to make these changes even easier to ensure team ownership of their process.
Second, teams need to avoid non-productive work to ‘show you did it.' IBM
Rational's experience shows that automation can allow necessary book keeping to
be done so audits are passed with no or limited extra work.
Conclusions
Agile development will have to overcome many hurdles to become mainstream, but I
expect tremendous progress in 2007. Leading systems integrators and vendors are
investing in agile development, consolidation may happen in a scattered agile
process landscape enabled by EPF, and agile transformations are hopefully
increasingly presented as a journey rather than an ‘all-or-nothing' strategy.
Tools specifically built for agile development are emerging. Much work is left
to do in the areas of compliance, support for large-scale agile development, and
geographically distributed development, but here also we see emerging practices
and supporting infrastructures that will help making agile development
mainstream. All in all, there is good hope that 2007 will prove to be a year
where agile development is crossing the chasm and reaching mainstream
development organizations.
About the Author Per Kroll is development manager for RUP and the IBM Rational Method
Composer, and the project leader on the Eclipse Process Framework Project, an
open source project centered on software practices. He is responsible for IBM
Rational's strategy in the process area. Per has twenty years of software
development experience in supply chain management, telecom, communications, and
software product development. He is author of the books The Rational Unified
Process Made Easy - A Practitioner's Guide, Kroll and Kruchten, and
Agility and Discipline Made Easy - Practices from OpenUP and RUP, Kroll and
MacIsaac. Per is a frequent speaker at conferences and author of numerous
articles on software engineering.
Error, missing joomlaboard config file!
[1] See Crossing the Chasm,
Geoffrey A. Moore, Collins, 2002.
It seems that IBM Rational is afraid on falling behind MS. I wonder why Scrum inventer Ken Schwaber prefers to colaborate with MS VSTS instaed of Eclipse PF?
Articles 
