When the Douglas
County CO Government IT Department was asked to create a custom application for
the Sheriff's Office to track and manage the County's resident convicted sex
offenders, the project estimates using the traditional waterfall-based
methodology proved too long and too costly to gain approval from its IT
Steering Committee. Rather than cancel
the project or purchase a less optimal off-the-shelf product, we implemented it
as our first Agile/Scrum project. The
outcome was a very successful system implementation delivered in four months -
less than half the original schedule estimate.
But the journey was equally rewarding, and taught us that "Agile" meant
more than just a change to the project management methodology. For us, it meant changing almost everything.
To give ourselves
the best opportunity for success on this first agile project, we explored and
changed almost every facet of how we developed software: which project we chose
to first attempt agile development, how we estimated project size, how we
staffed the project, how interactions between team members should occur, which
technologies we used, and how we sold the project to our customer, our team, and
our IT Steering Committee. Most parts of
the first agile plan went as designed; other aspects could have gone better. Key lessons learned on the project were
centered around simplifying process and design approaches, as well as improving
project communication and team dynamics.
About the Application
The
Douglas County Government IT Department developed an application to (1)
automate law enforcement processes to manage the county's convicted sex
offender registry, and (2) publish the registry to the public. The application provides powerful management
and search tools to all of the police jurisdictions within the county to sex offenders'
files, and transfers them between jurisdictions (see Figure 1). County citizens can also view the most accurate
sex offender registry available (see Figure 2), and sign up to receive email
notifications if new convicted offenders move within a designated radius of
their home, children's schools, or other places of interest. The public portion of the application is viewable
at: http://www.douglas.co.us/apps/soso/initPublicIndex.do
Figure 1 - Law Enforcement Agencies have a tool to unify the
management of convicted sex offenders between jurisdictions. The search capabilities provide investigators
with multiple ways to search for suspects among previously convicted sex
offenders.
Figure 2 - Citizens can search for previously convicted sex offenders
within a designated radius of any address in the county.
Planning for Success Choice of Project. We took great care in choosing
this particular project as our first attempt at Agile. The project was near ideal to us for a few
key reasons. One, the system did not
require integration with many other existing systems. This was beneficial in that we were not
injecting changes to other systems that would require re-coding or re-testing. Two, the business rules were not
complex. Unlike other systems in the
County, the business rules for the application were easy to understand, and
there was a degree of latitude in
Advertisement
how we fulfilled the user's need. (By comparison,
the County's system that calculates residents' Certificate of Taxes Due, has
very complex business rules where one and only one calculation is correct.) This reduced the risk that developed
functionality would have to be reworked upon customer review. Three, the size of the application was
smaller compared to other County systems, but large enough to uncover any
issues we would face if we chose to apply this methodology to future projects.
Overcoming Resistance. The IT Steering Committee was
initially (and rightfully) wary of our pitch that we could reduce the project
timeline in half with the implementation of a new approach. Their experience taught them that IT groups
such as ours would promise "big changes" but produce little in tangible
results. To overcome their reluctance to
our approach, the IT management team took an educated gamble - we offered a
guarantee. If, after two months of our
four month project, the customer was not happy with our progress, we would recommend to the Steering
Committee that the project be cancelled.
While we knew this would increase the pressure on our project team for
results, this tactic helped us gain approval from the Steering Committee. And after receiving a round of applause from
our customer at the month two demo, it was clear that we had begun to win some
trust in our approach.
Staffing. Determining who would work on the project turned out
to be as important as any process or technology decision we made. We estimated that the size of the project
would require two developers for design and coding, a QA analyst for testing,
and a Scrum Master, who managed the project.
The team members were hand-picked not just because they were technically
astute, but also because we believed they could absorb change quickly, be
comfortable working under some ambiguous circumstances, and had some
flexibility to work extra hours, if needed, to recover from mistakes along the
way. Perhaps the most important decision
was staffing the project's Scrum Master.
We chose the Software Engineering Lead for this role, rather than a
traditional project manager, as he had previous agile experience and could
adapt the process on the fly, was well suited to provide technical guidance,
and could relate to the developer's concerns, who were implementing new
processes and technologies under the gun of the project schedule.
Team Dynamics. One of our best decisions was to
co-locate the team for the duration of the project. We temporarily moved the developers and QA
analyst into a workroom. While seemingly
a small detail, the project team is united in its belief that the resulting increase
in communication directly reduced the schedule.
Before the project, we expected that the bigger change would be that the
Scrum process allowed us to "build a little, test a little," and give us a big
schedule lift from our waterfall-based project methodology. Although hard to quantify, we are now certain
that co-location was as big a key to success, especially on our first agile
project where the constant communication cut down on confusion and wasted time.
Estimation. Two techniques new to us were
particularly useful to estimate the project.
The first was the creation an HTML mockup of the core application
screens, developed by the team that investigated the project scope. The mockup confirmed high-level functionality
and was a useful analogy for the Scrum team to use for the database schema and
UI designs. The second technique was
using story points and nonlinear estimation techniques, like those elaborated
by Mike Cohn in his book Agile Estimating
and Planning (Prentice Hall, 2005).
An estimation scale where the gaps between values doubled each time (1,
2, 4, 8, 16 hours, etc.) proved valuable, and gave a realistic level of "accuracy"
for complex or ill-defined story points.
Design Simplicity. No matter the methodology used,
there are upper limits to a developer's productivity. Studies have shown that for Java/J2EE
development, a very productive developer can hand-code about 2,000 source lines
of code (SLOC) per month[i]. A core strategy for this effort was to drive out
unneeded complexity from the software architecture, and reduce the amount of
hand-generated code. For example, we excluded
the use of EJBs, as the cost in developer time outweighed the benefits of such
a heavy approach. In addition, we used
Hibernate for the persistence layer of the architecture. Hibernate is an open
source java-based tool that maps database relationships to an object-oriented
domain model, and can significantly reduce development time otherwise
spent with manual data handling in SQL. Previous project
strategies required our developers to hand-code this tedious layer of the
software architecture. The Scrum Master
also chose to implement Google Map APIs for the application's mapping
functionality. This required less code
than using the county's traditional mapping solutions, and further reduced the
overall application SLOC count. The final
application was completed with approximately 25,000 SLOC, with over 8,000 SLOC generated
through Hibernate. Given the
two-developer, four month project, it is clear that the project would have
extended by months without these choices being made.
Project Management. Tracking progress on an agile
project would not have been possible using the county's traditional earned
value management and waterfall-based work breakdown structures. The Scrum Master reported progress in the IT PMO's
weekly project status meeting using a visual graph of the team's burn-down
chart and progress towards that sprint's story points (see Figure 3). The approach simplified the data that needed
to be maintained to accurately gauge the project's progress, while being as (or
perhaps more) informative, than the traditional project management
measurements.
Figure 3: A simple
backlog and Burndown chart was maintained in MS Excel, and summarized in one MS
PowerPoint slide to communicate the project's weekly status
Retrospective
For all
the success of the project, we learned from a couple of missteps as well. While neither of these issues was a huge
detractor, lessons learned included the following:
Organizational Change. We underestimated the anxiety
that this move to Agile would produce within the IT organization. We communicated that the management team was
driving this first attempt at agile development as a pilot only, and created
and presented a "Scrum 101" presentation to share the approach. But staff members read between the lines of our communications and were apprehensive on the
implications of agile development.
Business Analysts were concerned that the closer relationship in an
Agile project between developers and the customer would eliminate a need for
their positions. Project Managers were
concerned that they would need to be technical experts, as was our pilot's
Scrum Master. IT Operations was concerned
that creating constant builds would introduce configuration management issues. We could have spent more time one-on-one with
each of these groups to better manage these concerns.
Personal Change. The move to Agile required more
personal change for the project team members than we anticipated. Developing software in a more cyclic,
feature-driven, less structured approach was a huge paradigm shift. Moving away from the comfortable patterns with
which the team was most familiar, towards new approaches - even if we thought
they were better - was initially viewed by them as risk, not opportunity. In hindsight, we could have prepared more to
overcome their initial resistance to these changes.
Summary
Our first
attempt at an Agile project was driven top-down by management, and sold to our
IT Steering Committee and project team members. By all measures, it met or exceeded our expectations. Our path to faster delivery was not a revolutionary move to a new methodology,
but rather an evolution of many of our
software development processes. Each
change we made chipped away at the schedule, and fit together to cut the
overall project timeline in half. Our
advice to those driving change from the top: spend time evaluating all aspects
of your project management and development practices for efficiencies, but do
not underestimate the effect of those many changes on the project team members. The extra effort is sure to result in
improving your project teams' speed and quality.
About the Author
Chuck
Fredrick, CTO of Douglas County, CO. Chuck has 14+ years experience in Software
Engineering, leading all functions of the software development process using a
variety of development methodologies. Chuck joined Douglas County, CO
government in January 2006, and serves as the Chief Technology Officer, where
he has responsibility for Enterprise Architecture, Software Engineering, Quality
Assurance, and technical strategy. Prior to joining Douglas County, CO,
Chuck held various management positions at a Fortune 20 telecom company, and
was a Captain in the United States Air Force, serving as an Acquisition
Officer. Chuck has a B.S. in computer science and an MBA with a Marketing
emphasis. He is a certified Project Management Professional (PMP) and a
certified Six Sigma Greenbelt.
