Measuring Agile Success

Video Transcript

from Agile Journal, this is AgileTalk. I'm Patrick Egan, publisher of the Agile Journal. I'm here today with Isaac Montgomery, an Agile coach with Rally. How're you doing today Isaac?

Really good, how're you doing Patrick?

We're doing well. Well, you know, when we're talking about Agile. What do we mean by 'Agile Success'?

Well, that's a great question because so often people talk about Agile and being successful with Agile, what they're really talking about is how well are we doing Agile? How well are we performing according to Agile best practices? Whatever that might mean. And I find that that's not really the question that I would prefer that people ask.

Rather, I would say, is the question is, "how well is Agile working for you? What is business success? "And that's the question we really want to ask ourselves, is how do we define business success for our organization and if Agile is being successful then it should be driving business success.

 Right, so then, what specifically should we be measuring for to, measuring success? in Agile.

Right. So every organization's a little bit different, right? Everybody's got their own definition of success. their own mission, but what we find is that probably ninety five percent of the success criteria, the thats that organizations are trying to achieve fall into the categories of improved productivity, right?

Doing more with less.

Right.

Increased quality, you know? Delivering a better solution, a more robust solution, a less defect solution improves responsiveness being able to be more nimble and Agile in how we are able to get things to market and respond to changing situations and changing market conditions. Driving customers' satisfaction, right?

We're delivering lots of stuff. It's really high quality. But is it actually solving customer problems, are we delighting our customers? And then, finally, employee satisfaction, right? Are we doing things in a way that help us to build our organization and and breed success and happiness and joy from our people.

Which will then yield a greater solution overall. All right, so, let's say we are able to gather these metrics, and we don't like the results. What should we do then?

Well that, so that is the essence of being able to inspect and adapt. To be able to run plan-do-check-act cycles, to be able to do Kaizen. So what I recommend is that once you have a ability to measure and understand your productivity, your quality, your customer satisfaction if you don't like those, those metrics, what you should be doing is you should be running some experiments, right?

Because those outcomes are your dependent variables, right? And we need to be able from a hypothesis. And if you have a hypothesis that by increasing the level of testing the development or test coverage that your organization has will yield higher quality, then let's design an experiment around that.

How can I isolate that independent variable in test driven development and create an experimental group and a control group and let me run that experiment for a while and see whether or not I get the expected results. And if I do, then that's great, that's really interesting and that should yield further experiments to expand that experiment and if I don't, well that's really interesting too, because part of the nature of experimentation is there's no such thing as a failed experiment because the result of disproving your hypothesis are often times even more interesting and more valuable than proving your hypothesis.

You're right.

So if you're running really good experiments, and you're really taking a learner's spirit in what you're doing, then you're constantly going to be improving and asking yourself what do I know about my organization? What do I believe about my organization? And how can I test that? Right, so, is there anything else we should be measuring?

So,that's really question for those experiments that your going to run and so we've identified some possible dependent variables for you, those outcome metrics.

What are the levers that you're going to be moving to try an, try and adjust those dials, right? And based upon the experiments that you choose to run, you'll need to be able to measure those dials, so like I said before, are we running test driven development? What's our test coverage look like? What's our iteration links look like?

What's our mate meet? Lot of these standard Agile, metrics that we talk about measuring adherence to good Agile practices. What practices that are outside of the Agile space? What is the average length of a Agile project? How often are we able to release? What is the debt of our backlog? How How far out do we have our backlogs sized?All of those things are interesting.

I wouldn't suggest that an organization measure all of them but they would measure the ones that are interesting to them based upon the experiments that they're looking to run. What questions do you have about your organization?

Right, so, on our measurement program, what should we guard against, what can we look out for really.

So, the important thing with measurement, measurement can be incredibly powerful, but as as they say, you know, with great power comes great responsibility and so when you're measuring you need to be making sure that your measuring those things to test, to understand, to experiment, right? We don't want to use those as, as ways of governing the organization and measuring for process adherence, or measuring in order to create a a, a target or a performance evaluation where we, where we simply defer our leadership and abdicate our our ability to, to manage our organization.

By being more lazy. And saying I want velocity to increase by 20% iteration over iteration. Because what we know is that our organization and our people will perform to our measures, not necessarily to our goals, and so if we use those measures that way, I can guarantee you you will see twenty percent velocity growth.

It won't necessarily mean what you want it to mean. Right? So that's what I strive for organizations. Why I think that measurement takes a certain level of maturity and leadership to be able to understand that measurement is a way of answering questions and to feed additional questions and to continuously improve.

It's not really a way to assess whether we're good or bad it's just a lens into your organization.

Alright. Well great, this is some great information for our viewers and thanks again for joining me today.

Well thank you very much for having me, I appreciate it.

Sure, well if you'd like to see more video like this just visit AgileJournal.com and I look forward to seeing you at our next talk. Thanks a lot Isaac.

Thanks Patrick. Bye bye.