It is apparent that agility works, whatever that may mean. However, many software projects remain artifact-driven and waterfallish. Why is this? The most common excuses are that agility is too developer-centric, that it is too lightweight, and that feedback to business is hard to understand. In particular, many managers in larger companies miss their metrics. In this paper we address this last problem by defining a new metric, Earned Business Value (EBV), that replaces standard Earned Value Analysis (EVA) metrics, and can be calculated in an agile project. Using EBV, teams can gain better understanding of a project's progress and determine when and where to reallocate resources or change course.
Earned Value Analysis is a powerful method often used in "standard" project management, and there are three basic metrics in EVA[1]. The first is Budgeted Cost of Work Scheduled (BCWS), which is the baseline cost up to the status date you choose. The second primary metric is Actual Cost of Work Performed (ACWP), which is the actual cost required to complete all or some portion of the tasks, up to the status date. And the third basic metric is the Budgeted Cost of Work Performed (BCWP), which is the value of the work performed by the status date, measured in currency.
Earned Value Analysis mostly consists of combining and manipulating these three metrics in various ways. Unfortunately, both BCWP and BCWS are not metrics that exist in agile projects, as they require a detailed, up-front plan to compare against. So, what are we to do?
The Goals
In order to solve this problem, we need
Advertisement
to know why Earned Value Analysis exists, so that we can figure out how to replace it. When the business asks about EVA it is usually asking about progress on our project: how we're doing and when we'll be done.
In other words, the business is are asking for information about the value of the product - how much value is being provided? So we need a metric that measures how "done" we are from a business perspective. We'll call our new metric Earned Business Value (EBV).
Features and Stories
In order to discuss EBV, we need to have the notions of feature and story.
Feature
A feature is something about our project and/or product that we advertise or sell, and that we must do work on to provide. Typically, our product (or release) is described as a list of features, such as:
Allows Users to Update Customer Information
Supports the XXX Claims form
Added "Updated Customer Info" Reporting
Improved User Interface
Faster Rendering of Reports
Improved Security and Protection against Hackers
and so on...
Features are implemented in the product by working on stories, and the stories are said to belong to the feature.
Stories
A story is the fundamental unit of value, requirements, and work that is visible from both the inside and the outside of the project. Stories are used as the interface between the business and development sides of the project.
Stories are important since they are the smallest units of value that are managed. Stories have been described as "promises for conversation" by Ron Jeffries[2], and I agree wholeheartedly. Basically, a story consists of four things:
1. Description: a short description of the goal to be achieved.
2. Size: for rough estimation purposes. Units such as Story Points (SPs), T-Shirt sizes (S, M, L, XL), Days, and even gummi-bears, have been used.
3. Validation Criteria: what it means to be "done" with this story. This is probably the most important attribute, as being "done" is what allows the team to claim the value of the story.
4. Business Value: not all stories have business value, but the ones that drive development do. This is what we are trying to calculate.
Stories are of various types, and the following are example stories for a "Update Customer Information" Feature:
Determine what the stakeholders want to happen (analysis story, done when we have a validated user scenario, screen mockups, and success post-conditions);
Develop the main execution flow (development story, done when we have verified tests that pass);
Determine alternative flows, and what we want to do about them (analysis story, usually time-boxed);
Develop alternative flow X (development story, one for each alternative flow); and
Stress-test the feature for 500 simultaneous users (test story, done when we have a validated testing regimen that passes).
Work Breakdown Structure
Management needs a view of the work that shows the "big picture." Luckily, standard project management has such a structure already invented for us - the Work Breakdown Structure (WBS) - which is a "deliverable-oriented grouping of project elements which organizes and defines the total scope of the project.[3]" Many agilists cringe at the mention of a WBS because of its ‘high ceremony' connotations, but I use it merely as a way to categorize or organize my stories into areas. The WBS that I use organizes the work into three main areas:
Product - work that is directed to actually producing the software product;
Team - work that allows/enables the team to be able to produce the product; and
Business - work that allows the business to market, sell, install, or deliver the product to users.
Each project is different and the actual structure of the WBS (what rollup buckets there are) is a business matter, because the WBS is primarily used as a tool for organizing the work in ways the business can understand. The stories that live at "the bottom" of these buckets define the work being done. A WBS for a fictional development project appears at Figure 1.
Figure 1: Sample WBS
Let's use this WBS to calculate business value.
Calculating Business Value
We do this in a straightforward way and provide the business value of any WBS bucket, including stories (the ones at the bottom). We note that the Business Value metric we are calculating is actually a percentage, not a dollar value.
First, we assign additive weights to the buckets of the WBS, which represent features and other organizations of stories. In the simple view in this article, we only get business value in our sample WBS by either putting features in the code or by doing something that lets the business market or sell the product - there is no business value in other parts of the WBS.
I'm using relative weights on the WBS here - each WBS bucket is compared to its siblings - and the following WBS with weights is telling the following story:
Our Product Leg is worth 3 times as much as our Business Leg, and our Team leg provides no business value;
Adding features to the product supplies business value, but modifying or cleaning up code that already works does not;
Improving the User Interface is the most important feature, worth 1.5 times as much as being able to Update Customer Info; and
User Documentation is the most important business bucket, but is still not worth very much compared to any of the features.
It is up to the business, or customer, to assign the weights to the WBS legs and stories - this is not a technical matter. There is no guaranteed formula; there is only the requirement that the weights be additive. For example, is "Team Training" something with business value or not? I don't know, it's up to your project. Typically it doesn't provide value if paid for out of project funds, but might if paid for out of overhead (training) funds. Our WBS (with weights) could look something like Figure 2.
Figure 2: WBS with Benefit Weights
After we've assigned weights to the WBS legs and buckets, we further assign additive weights to the Stories within each WBS "bucket." For example, for the Stories we find in the "Update Customer Information" feature we could have the weights as in Table 1.
Table 1: Stories and Their Weights
Note that the analysis stories have no weight; they only exist because they provide the development stories that do have business value.
Given this assignment of weights, we can now calculate the Business Value (BV) of any WBS bucket, including Stories. The idea is simple and has two parts. First, the value of the whole tree is 100% (or 1), which means that doing all the work gives all the value. This value of 1 is assigned to the top of the tree. Second, as we move down the tree, each bucket's value is the appropriate percentage of its parent's value, as compared to its ‘siblings' - the other children of its parent. This is a multiplicative formula, because it's based on taking percentages of percentages as we move down the tree. The calculation is recursive, and is given by the following formula
Let's do an example, and calculate how much the Main Execution Flow of the "Update Customer Information" feature is worth. According to the last table and the marked up WBS, the feature leg is worth 3/4 of the total value of the project; the "Update Customer Information" feature is worth 10/40 of the feature leg; and the Main Execution Flow is worth 10/20 of the total value of the feature.
Therefore, the Business Value of the Main Execution Flow of the "Update Customer Information" feature is (1)x(3/4)x(1/1) x(10/40)x(10/20) ≈ 9.4% of the project. Remember that this is a multiplicative formula, taking percentages of percentages as we move down the tree. If we were to add another feature with a weight of 10 (say "Account Report"), this would drop to 7.5%, as it changes the "(10/40)" piece of the formula to "(10/50)".
Earned Business Value
Once I know how to calculate Business Value (BV), I can calculate an Earned Business Value (EBV), which I define as "the percentage of the known business value that is coded up and running." In other words, we add up the business values for all those stories that have been completed - that have earned their business value.
There are two parts to this calculation:
A subjective part that determines the weights allowing us to determine the BV, and
An objective part that determines whether or not a story is "complete," which is purely a "yes" or "no" (1 or 0). Since stories are small, it usually makes no sense to have a % complete at this level, so we go with a simple "done or not" measure.
We already know about the first part of the calculation (we use the additive weights), and the second is straightforward. Once we have our story defined, we have validation criteria for it. A story is complete (has earned its BV) once it meets its validation criteria (its tests pass, for example).
Summary
A functional Work Breakdown Structure (WBS) provides structure to an agile project, as well as a framework for calculating business metrics (BV, EBV). These metrics are valuable to the business for a number of reasons, including:
The combination of standard burndown charts and EBV charts provides a good, composite understanding of the progress of a project.
The business can see when the project is providing diminishing returns - when the additional business value provided is not worth the cost of the work being done. This could indicate that the project needs to move into a stabilization phase and release.
If the business knows a project's value to the business, these metrics provide a way to compare projects to see if resources need to be reallocated.
Dan Rawsthorne is a CST (Certified ScrumMaster Trainer) and a Use Case expert who lives at the "process end" of things at Net Objectives (http://www.netobjectives.com/). His focus is on helping organizations get products "out the door" using agility, and he has been doing so for over 20 years. He has a PhD in mathematics from the University of Illinois, and is currently writing a book on how to be a ProductOwner. He concentrates his training and coaching in Use Cases and Scrum.
Error, missing joomlaboard config file!
Comments (4)
JD from ScrumWorks Pro: ...
Danube’s Agile tooling Solution, ScrumWorks Pro implements this theory using "Business Weight". Business Weights represent a Product Backlog Item's business value index, a relative value used to compare the business value of PBIs. You can learn more about it here: http://www.danube.com/sw_flash/bpg/BPG_Business_Value.html
Dan Rawsthorne is a Certified Scrum Trainer and one of Danube Technologies’ transformation coaches, helping organizations transition to Agile practices. Rawsthorne is a 27-year veteran of the software industry, having worked in a wide range of capacities, from coder to project manager, for companies large and small. In addition to possessing deep knowledge of many software processes, procedures, and techniques, he holds a Ph.D. in mathematics. More of Rawsthorne’s thoughts on Agile development and implementation can be found at his blog: www.danube.com/blog/danrawsthorne.
1
February 13, 2008
johnnyfromcanada: ...
Great model, and well presented. Understandable, explainable to laypeople. Reasonably straightforward to implement. Fills the big gap between upper management and Scrum team management.
Is this built into any of the Scrum tools?
2
February 07, 2008
bolarotibi: ...
I totally agree that this is an extremely useful metric to calculate. What would be very interesting is to find if organisations implementing agile processes have calculated EBVs and if so whether they found them useful and accurate.
There does not seem to be enough sharing of metrics within the industry and would like to see more done in this area. Perhaps the agile journal could start a database of metrics that subscribers could add to.
3
June 05, 2007
Athresh: ...
A very viable model for project tracking in the agile world
Blogs 
