Continuous improvement is at the heart of Agile. Retrospectives provide opportunities to the Teams to inspect their process, and continuously improve it. Many Teams fail to take advantage of this and thus fail to improve.

Common Retrospective mistakes are

Scrum is one of the biggest process invention but it does not guarantee success. Even creators of Scrum accept that more than half of the Scrum implementation does not go well. Why is that? Scrum is critical but what is more critical than Scrum? Moving from waterfall model to Scrum is a welcoming change that makes people think in the right direction, however, this does not guarantee success.

If a person with ongoing back pain visits the doctor then doctor certainly suggests to do exercises or yoga to make body agile enough to get rid of pains. If that person is wise then he starts doing exercises and pain disappears. Exercises don't cure pain overnight. It requires a regular routine. Isn't it common to see pain appear again when exercises are stopped?

Ian Mitchell is Chief Scientist at proAgile Ltd. He can be followed on Twitter: dr_ian_mitchell

Ian Mitchell is Chief Scientist at proAgile Ltd. He can be followed on Twitter: dr_ian_mitchell

Ian Mitchell is Chief Scientist at proAgile Ltd. He can be followed on Twitter: dr_ian_mitchell

"Constant development is the law of life, and a man who always tries to maintain his dogmas in order to appear consistent drives himself into a false position".  

- Mahatma Gandhi

Contracts, and why you can't avoid them

The Agile Manifesto places customer collaboration over contract negotiation. Yet it must be admitted that it would take a hardline ascetic to work without a contract at all. Look at it this way. Without a contract of some form to bind parties into an agreement, the prospects of being paid start to look bleak. 

This might not faze a yogi who eschews material wealth, but to employees of the tech sector an unreliable pay check presents a serious barrier to productivity. Consequently, without a contract, the client's prospects of receiving a deliverable that is fit for purpose can be expected to diminish as well. No matter how uncool and unhip the drafting of legal agreements might be when compared to the visceral joy of crafting code, agile developers need something in place upon which the expectations for delivery can be predicated and managed.

In many ways Gandhi was a hardline ascetic, and an examplar of the sort of intellectual who can live without written contractual guarantees from others. Yet he was also a canny lawyer, and in retrospect we can see he was right to assert that constant development is the law of life. It can be argued that the agile revolution has been founded upon this very principle.

When good contracts turn bad

There is a definite tension between the need to express contractual obligations, and the need to embrace change.

On the one hand, a contract needs to be precisely formulated and exacting. A well-written one will leave no room for ambiguity in interpretation, and the authoring of such documents has rightly become the domain of legal experts. We may enjoy painting lawyers as grasping types who relish discord and the opportunity to harvest fees, such as when the parties to an agreement fall out and seek redress through the courts. However, the measure of a good contract is the extent to which it satisfies the stakeholders. In fact a well-written contract will help to keep people out of the courtroom. It should also be remembered that legal agents can incur negative publicity and professional liabilities from a poorly drafted agreement. So in truth, contract lawyers are motivated to produce terms that are defined as tightly as possible, and which are unlikely to result in acrimonious wrangling. On the other hand this drive towards unambiguous contractual specification makes change hard to accommodate. This is a problem. If market conditions change during a project - perhaps something which can lead to the business case being questioned or even invalidated - then the terms of the contract still hold. If the requirements are only poorly understood at the beginning of a project - but are subsequently clarified - then the original scope in the contract also still holds. The requirements and conditions which were enumerated within the contract are everything. Subsequent clarification and changing conditions count for nothing. Where is the client's best interest now?

What this shows is that although we need contracts in order to deliver products and services, it is horribly impractical to apply them in extremis. We need flexibility, and the traditional approach to squaring this circle is the Change Control Mechanism. 

Welcome to Hell

The Change Control Mechanism started with the best of intentions. Since contracts become out of date as requirements and the business environment change, why not formalise a process of amendment? The original contract doesn't have to be set in stone. It's perfectly admissible to replace one contract with another one. If that's too cumbersome, then it is equally possible to replace outdated portions of a contract with an up-to-date addendum, as long as it is signed and understood by the impacted parties. 

Unfortunately, the devil is in the details:

1. Changes must be recosted. The project budget will need to be revised accordingly. Since changes usually involve new requirements or rework, the price (referred to as the "commercials") will normally be revised upwards rather than downwards. It will, of course, usually fall to the supplier to drive through any such amendments. It should be noted that the time a supplier spends drafting and finalising contractual changes is also potentially billable.
2. Before a contract can be amended, a proposal must be drafted which outlines the recommended changes along with their justification and an indication of what the revised commercials will be. This proposal is known as the Change Request (CR) and there will typically be some sort of pro forma for this. Clients and suppliers can each have their own pro forma. However it is most often the client's pro forma which is adopted since they must review and approve the changes and associated costs.
3. In theory, it is the client who should initiate Change Requests since they ultimately own the product. But in practice a client is likely to communicate new requirements informally with the expectation that they will "ride" on the original budget. The onus is then on the supplier to initiate (and take ownership of) a Change Request.
4. From the client's point of view, there is a huge opportunity cost associated with Change Requests, or more specifically with the refusing of them. This is because the only alternative to approving essential changes with revised commercials might well be to choose another supplier. This may be far too expensive to be practical.
5. From the supplier's point of view, there is little opportunity cost associated with Change Requests. Very small changes may be absorbed by the supplier, but since the time spent raising and progressing CR's is itself billable, the threshold that must be reached before a CR is triggered by a supplier is low. That threshold is largely determined by political considerations, in that a surfeit of CR's may be viewed unfavourably at board level.
6. From the client's point of view Change Requests are "distasteful". This is because of the cost increases that typically accompany them. A client would instinctively prefer to progress changes informally so no costs are incurred, or perhaps - that old chestnut - under the guise of bug reports. However this does not serve the client well since a deviation from contractual conditions would leave them exposed, especially if they can be seen to have initiated it.
7. From the supplier's point of view Change Requests are great! Any change to the original contract can, in principle, be subject to a CR and to cost increases which the client may have little option but to pay. Suppliers also know, through experience, that change is inevitable no matter how sure a client thinks they are about requirements. In practice it is quite possible for a seasoned IT supplier to double or even triple the original contract size through Change Requests. What was that about lawyers being grasping types who relish the opportunity to harvest fees?

We clearly need a better way to "square the circle". To be more specific: it is clear that industry needs to replace the established Change Control Mechanism with a contractual model that supports Agile practice and a more evolutionary take on requirements.

Lifting the Curse

A few months ago Gabrielle Benefield (Scrum Training Institute) & Susan Atkinson (Gallen Alliance Solicitors) wrote a paper titled The Curse of the Change Control Mechanism. I reckon they're right. That's exactly what Change Control has become - a curse. The alternative is to look at the problem afresh, and to promote an "evolutionary" contract model which better supports change. 

I and other proAgile staffers have been applying such contractual models since 2009, originally as part of the Codeworks DEV programme. A key plank of that initiative was the incorporation of an Agile-friendly contract model which redresses the balance between stakeholders, and which holds clients and suppliers to be equal partners in successful ongoing delivery.

Evolving better requirements

Technically, the core of the problem that needs to be solved can be reduced to two observations:

1. Once defined, the value of requirements deteriorates over time. In fact requirements demonstrate the pathology of a "half-life" similar to that of radioactive decay. According to a Kansas University study, the half-life of a requirements set (i.e. the time before it loses 50% of its value) is typically about six months.
2. The ability to define project requirements improves over time. In other words, the uncertainty associated with a project diminishes as the project progresses - a pathology sometimes referred to as "the Cone of Uncertainty". This is because project members will learn things and grasp the problem domain better. However, this improvement is subject to the law of diminishing returns. The first few days may reduce uncertainty by 80%, but it could take several weeks to reduce it by 90%. 

In short we need to capture requirements quickly enough so they don't lose value...but not so quickly that uncertainty remains a problem, and not so slowly that "paralysis by analysis" becomes an issue. 

If we use an iterative approach then we can constantly track the changing requirements. Assuming each iteration lasts only a few weeks, we won't get even close to reaching that six month half-life, and the requirements set will be allowed to evolve. At some point, requirements will have been clarified to the extent that the additional return on investment in clarifying them further will not be justifiable. But if we are also releasing incrementally, by the time that happens we'll have delivered a viable system. And as we know, that "proof through ongoing delivery" is a key part of the Agile sell.

The Evolutionary Contract as a baseline for Agility

Seen in this light, the duties of an Agile-friendly "Evolutionary Contract Model" are threefold.

Firstly, the contract must define the Agile process that will be adhered to during the project:

1. It needs to stipulate that the process will be timeboxed. The precise terminology can vary - "Sprint" or "Iteration" perhaps - but the fact that it is time-constrained must be declared. 
2. The length of each iteration must be stipulated, such as 2, 3, or 4 weeks. 
3. The mechanism to be used for raising and prioritising requirements must be described, along with the roles and responsibilities of those involved. The fact that the initial first-cut of requirements (scope) is subject to ongoing amendment needs to be stipulated.
4. If requirements can be traded in and out of timeboxes, then the mechanism for estimating their difficulty or complexity should be indicated, so that like can be traded for like. Again, the roles and responsibilities of those performing the estimates needs to be elucidated. Will bugs be fixed gratis, or will they be raised as new requirements tickets?
5. It should be made clear whether or not the client (or product owner) can reprioritise timeboxed requirements once the timebox has started, or introduce new ones, and if so what the process for accommodating this will be. Some agile methods do not permit such interference mid-iteration.
6. Client responsibilities for making business representatives available must be specified, including domain experts and/or product owners.
7. The protocols of issue escalation need to be tabulated, and any special reporting requirements (above and beyond incremental evaluation) must be clarified, as do any special mechanisms for risk management.
8. The protocols of incremental evaluation and release must be described. Who will evaluate the releases? Will demonstrations be held in client or supplier environments? Is Continuous Delivery required? How will QA and UAT be done for each release? How much formality is required for sign-off? What documentation will be supplied?

Secondly, the relationship between cost, time, and scope has to be nailed down. There are two common Agile approaches to this:

1. Variable time, cost, and scope. This is sometimes referred to as "time and materials" contracting. The client does not know how much the project will cost, but will pay the supplier on an ongoing basis until a satisfactory product is delivered. 
2. Fixed time and cost, with variable scope. This is becoming more common since clients are increasingly price-sensitive and wish to know their financial liabilities up-front. Although time and cost can vary independently, it is usual for a client to expect full-time resources and for the product to be delivered by a certain date. On the principle that "time is money", the correlation between time and cost is typically invariant. In practice, this means that a client pays X which will fund Y iterations, and a product backlog of Z requirements will be reworked, prioritised, actioned, and delivered until the project terminates. 

Thirdly, the initial "first cut" of requirements needs to be specified:

1. Unlike traditional contracts, it does not represent the feature set that must actually be delivered at the end of the project. The first-cut is the starting point from which a product backlog will be populated.
2. It provides an indication to the supplier of the type of expertise that will be required in order to deliver a satisfactory product
3. The first cut can be mapped against a provisional list of milestone deliverables. It should be noted that this project schedule can be revised iteratively without the need for any change requests, since the mechanism of change is incorporated into the contract.

Special case contracts

There are two special cases of Agile contract for which the heads-of-terms will differ from those outlined above.

Firstly, Lean projects:

1.  A Lean contract will need to be modified to eliminate timeboxing. Instead of negotiating which requirements will be implemented in particular iterations, the product backlog will be constantly topped up and reprioritised. It may be possible to stipulate the expected Kanban velocity in the contract if the number of team members can be fixed along with the time per day that is to be spent on progressing tickets. 
2. Don't assume that an 8 hour day amounts to 8 hours spent on driving velocity. There are always ancillary tasks (such as the provision of assistance) which do not show up on Kanban. In some cases these tasks can be exposed as tickets, and it is recommended that this be done as much as possible in order to promote transparency. However, this is only practical in situations where a product owner can reasonably prioritise the ancillary task. It may be more practical to base any contractual assertions regarding velocity on (say) a 6 hour day.
3. One other thing. It can be argued that Lean projects are not projects at all, but rather operational activities, since it is common for them to function without a business case sensu stricto and without clear rules for termination. The termination clause of the contract will therefore need to be given particular thought.

Secondly, there are public sector projects:

1. These mandate special consideration for two reasons: they typically carry a greater burden of compliance, and they often involve public-private partnerships.
2. The requirements for compliance vary by nation, international agreements, and in some cases by state. For example, bodies in receipt of European Regional Development Funding will need to ensure that their contracts and tendering process are compliant with OJEU requirements, unless the project is very small. Contractually, the best way to manage compliance may be for public sector clients to pre-select a panel of suppliers who can then bid for work. The burden of compliance is then largely reduced to a one-off activity (formation of the panel) and the tendering process for contracts can be made comparatively light-weight.
3. Public-Private Partnerships can require contracts that encompass multiple parties. For example, if an incubator spin-off project is being actioned then we can expect a private start-up seeking matched funding, as well as the public body and the supplier. A tripartite contractual agreement may seem the logical choice but this can become messy. Clearer arrangements can be made if two separate contracts are formulated - one between the public body and the client, and another between the client and the supplier. The contract between public body and client is comparatively simple and will encompass match-funding arrangements and the responsibilities of the client to participate in the process. For example, the contract may stipulate payment in timeboxes. Once the client signs off an incremental release, the supplier will bill them directly and they will pay in full. The client then sends the receipt to the public body who will reimburse them by an agreed amount.

Contracts, flaming torches, and the pitchforks of the baying mob

Let's not forget that there is a meaty and earthy purpose to an agile contract. It doesn't just set the parameters within which work can be progressed. Nor is it there just to make sharp practice harder. The contract also explains to the signatories - albeit in perhaps rather dull legal prose - what their responsibilities are. 

Why point this out? Well, remember that certain parties might not actually know what they are expected to do in order to satisfy their side of an agile contract. For example, a client might genuinely not know that they are expected to participate once every four weeks in the evaluation and signoff of deliverables, or how seriously the exercise is meant to be taken. If they don't appreciate this - if assumptions are made, and these details are left unsaid or glossed over - then a client might well cut up rough when final delivery doesn't meet their expectations. A PR disaster may be in the offing. Upset customers might seek redress publically as well as privately. Flaming torches and twitching pitchforks might start wending their way up the hillside, baying for Agilista blood.

That's why it’s important to baseline expectations in contractual form. Again, the purpose is not to define the scope in detail - it is simply there to establish a first-cut of requirements and to define the mechanism through which changes can be managed, and value delivered on an ongoing basis. It establishes the terms upon which an Agile understanding between client and supplier can be encouraged to develop. 

Conclusion

Remember that a well written contract will actually keep the signatories out of the courtroom as well as away from each other’s throats. Even if the relationship between client and supplier gets no further than a mutual toleration predicated on acceptable ongoing delivery, we can be justified in claiming success.

"It may be true that the law cannot make a man love me, but it can keep him from lynching me, and I think that's pretty important." 

- Martin Luther King, Jr.

Note: I've uploaded a video blog on the above: http://www.youtube.com/watch?v=eDBpgZ1qYvw

Ian Mitchell is Chief Scientist at proAgile Ltd. He can be followed on Twitter: dr_ian_mitchell

As I am sure you are aware, The Mother of All Fixed Time Projects was due for completion today. Global deployment and rollout was scheduled for 6pm on a timezone by timezone basis. But it's now clear that slippage is inevitable. The end of the world must wait.

I still remain fairly well disposed towards these Prophets of Doom. I've managed waterfall projects in the past, so it's easy for me to empathize with their outlook on life, and to admire the peppy confidence they exude when delivering their messages of impending cataclysm. I wish I could have delivered highlight reports with such breezy conviction.  Where I part company with these Oracles of Kismet is in their rather strained usage of supporting material. An Agile project manager should never be so hamstrung by the anachronistic interpretation of evidence, whether it be documented or not.

There is however at least one seraphic text to which I know even the Agile Practitioner can look for reliable guidance. I refer of course to The Ballad of Blasphemous Bill by Robert Service. I have found this to be of great help when struggling with contracts of uncertain scope, but which include requirements that are nevertheless fixed. Even Agile projects can find that these immoveable features are embedded within them, and dug in deeper than the proverbial Alabama Tick. It can take the bending of an inventive mind to bring such projects to fruition, as we will see.

The Ballad of Blasphemous Bill is a poem set in the Yukon of a century and more ago. The author has signed a contract with a trapper...the eponymous Bill. The terms are simple and ostensibly straightforward. If Blasphemous Bill dies, then the author promises to find his remains, and to give him a decent burial.

Ian Mitchell is Chief Scientist at proAgile Ltd. He can be followed on Twitter: dr_ian_mitchell

Years ago an article appeared in Project Manager Today which elaborated upon why projects so often fail. It certainly ties in with my own experiences of process dysfunction and the symptoms I have learned to identify.

Ian Mitchell is Chief Scientist at proAgile Ltd. He can be followed on Twitter: dr_ian_mitchell

Surely one of the most controversial of Agile practices is Fail Early, Fail Fast. The underlying principle is simple. If an initiative is obviously crocked then the worst thing you can do is to keep it going*. Ed Yourdon talked about this years ago when he exposed "The Death March" syndrome of many IT projects. There are political reasons why these death marches continue to happen, each one a case study in the pathology of denial. Attempting to save face, or to buy time, are but two of them. The irrational yet very human desire to insist on return from bad investments is another. And then we have the "ugly baby" syndrome - a refusal to accept the evidence of one's eyes when the spawn is your own creation.

Now let us turn to the UK Government's Agile initiative, which they proposed in their recent ICT Strategy. I have been among the first to lift that ill-starred babe from the crib, and shake my head sadly at the Quasimodo before me. "This folly is an unworkable chimera" I have groaned. "How can Government be Agile?  It will likely reject the substance of its own tissue". For whilst neither an abomination against God nor Science, an Agile Government is never likely to come to terms with its own contradictory form. To the bell tower with it then, before it blackens us all with its presence?

Well, others around me propose even worse. But if we were to strangle this child at birth, it wouldn't be Fail Early Fail Fast, it would be murder. I say we have to give it a chance. At the very least it must be given the opportunity to succeed, and to prove any potential it may have. After all we are stakeholders in the success of our governments, and there can be no reward without some degree of risk.

And so it is that I must refute each of the arguments that were recently presented in Computer Weekly's Public Sector IT section. In the relevant article - "Agile will fail GovIT, says corporate lawyer" - a series of points were expressed which seem to sound the very death-knell of the Agile Government initiative. Another dose of healthy cyncism, perhaps? Well no, not really. Not when, in my view, each of the four points made happens to be wrong.

1) "Government customers want to know up-front how much a system will cost". Don't we all? In fact the pressure away from variable-time/variable cost (known as "time and materials") contracting has been building for a while. These days it is not at all unusual - in my experience, it is actually normal - for Agile projects to be conducted on a fixed-price basis. This works just fine, because it is the scope that becomes variable, and there are a number of iterations in which scope can be revised. The DSDM MoSCoW approach is a good illustration of this. On a waterfall project, the supposed "fixed price" is invalidated due to inefficiencies in change response, and as we know, the client just ends up paying through the nose via change requests. So the desire for up-front pricing is a driver for Agility, and not a contra-indication for it.

2) "Government is also legally required to follow open procurement rules". Tell me about it, it's a complete ball-ache. However, this has absolutely nothing to do with the methodology followed, nor does it militate against fixed-price tendering. On each of the public sector projects that I have managed we applied open procurement practices when soliciting Agile suppliers on a fixed-price basis. And for the record, yes, these practices did include OJEU. Each tender response was judged using a pro-forma in which they detailed their approach to iterative-incremental delivery for the fixed price they stated.

3) "Agile offers insufficient means of remedy if things go wrong". Well, it offers the ability to fix things regularly on an iterative and incremental basis, which is far more opportunity than you'd get with linear alternatives once you'd completed design. That's the Agile remedy. By involving the client throughout the project, the chances of final dissatisfaction are correspondingly reduced. The project proves its success by means of ongoing delivery. Of course that means the client is in a weaker position to sue at the end of it, but that's because they've been empowered and made stakeholders in their own success.

4) "Agile is fourthly not suited to public sector management structures". Good, this is a point which does in fact have merit. Public sector management structures are notoriously unAgile; those who are interested can read my previous whinge about this very matter. But many large corporates and multinationals were also renowned for being unAgile, and they have adapted. There is no reason to believe that the same benefits cannot be leveraged in Government. In fact, that's exactly why the PRINCE2 standard - so long regarded as the Niagara of waterfall methods - is now being integrated with Agile approaches. This is something I'm working on now with the DSDM Consortium.

The misgivings we all have about Agile Government are no excuse for misunderstanding. It is admittedly an unlikely initiative, but it can work. We've already gained some early successes and we will continue do so. Above all though, we need to be engaged and listened to by the Government; they need to prove it is more than spin, and allow us to help. Then, as is always the case with Agile Methods, we will see further proof of success in the delivery.

Ian Mitchell is Chief Scientist at proAgile Ltd. He can be followed on Twitter: dr_ian_mitchell

The path to Agility, like the road to damnation, is paved with good intentions. First among these is the valuation of people and their variable needs over the strictures of definitive process.

All four of the values expressed in the Manifesto can be seen to align to this tenet. Each of them reminds us to look beyond process so we don't lose sight of what people actually need. A velvet revolution against faceless bureaucracy, Agile Methods are a thesis for humanity in this technical world:

Ian Mitchell is Chief Scientist at proAgile Ltd. He can be followed on Twitter: dr_ian_mitchell

A few weeks ago, the UK Government released its ICT strategy in which it set out the case for this Wild New Thing, Agility:

http://www.cabinetoffice.gov.uk/content/government-ict-strategy