What's productivity have to do with Quality?
Recently, I had the pleasure of hearing an esteemed colleague speak about the tradeoffs between quality and productivity. His view is that quality and productivity compete for the same resources and you really can't focus on both at any one time. If you want to have zero defects, it does take commitment - often in terms of time, resources and budget. Quality may be free in some circles, but I have seen QA budgets in the millions of dollars. Most QA teams are stretched thin and face a challenge with every release to complete their battery of regression tests within the time allotted. Getting the code out faster often requires tradeoffs that can threaten to adversely impact code and systems quality.

Many projects fight a never-ending battle of trying to meet their deadlines, including implementing enough "critical" features and still staying within budget.

Agile cuts straight to the challenge
Many projects - both Agile and non-Agile - exist in a gray area where their development managers try to get a handle on what needs to be done, the complexities of ever emerging technologies and the battle to produce quality code. When we (as technology professionals) fail, systems crash, functionality does not work as needed and we hear that, once again, a technology effort was over budget. Agile cuts straight to the core of the issue by focusing on techniques which yield running systems that meet the requirements customers really need in the fastest possible time frame - aptly called "sprints" by Scrum enthusiasts. The imagery is compelling as we see small technology teams plan short iterations that yield components which really work and allow us to kick the tires of a running system in record time.

Why does Agile work? How does it really improve Quality?
A lot of people know that I am not a hard-core Agile enthusiast. I am the guy who has said that Agile needs to mature into a more comprehensive framework or else risk losing its relevance in the software development community. I have certainly come to love many of the Agile practices as being the best solution to the tough challenges that technology professionals face every day.  One of the reasons that becoming Agile works is because it cuts straight to the challenge of dealing with shifting requirements that are hard to understand, codify and will, most surely, change before the project is done. Successful requirements gathering is critical to any technology effort. Obviously, some requirements are easy to specify and are not likely to change. Other requirements are almost impossible to understand and even harder to document and maintain. Agile holds the banner high with the view that requirements documents should not just be binders of shelfware.  Requirements must be maintained as living descriptions of what the customer really needs and wants. Of course that's not easy to do unless the specifications are implicitly part of the quality assurance process.

Test Driven Specifications
All requirements are not alike. Specifying the requirements for a life support system are very different than a large scale SOA business application. The algorithms for a trading system must be specified and documented or else Agile cannot be used in some of the most important financial systems. Test Cases must be kept current or else releases will be deployed with the same defects over and over again. One good way to improve quality is to make test cases which are robust enough to meet the needs of a requirements document. Building testing into the beginning of the process through continuous integration (CI) and automated unit testing is another key feature.

Testing is more than just the rubber stamp at the end of the lifecycle
Agility does a nice job of showing how testing must be built in from the beginning of the lifecycle. Continuous integration (CI) servers immediately take newly committed code and compile them, executing automated tests and of course identifying who broke the build (make sure you get his $2 penalty fine for the office cupcake fund).

Agile has long criticized requirements documents that simply sit unused on the shelf. The implicit (and well-founded) notion is that most requirements documents are outdated before the ink dries. Keeping the requirements documents updated is a huge chore and rarely completed in a satisfactory manner. Of course, there are some systems which must have documented requirements that have been reviewed and validated. It has also long been a service management practice to consider all reported incidents, once triaged and investigated, potential candidates for becoming test cases. Test Driven Requirements refers to using well-written (and current) test documentation as requirements specifications.

The day that I managed to get the development effort to stop
Many years ago, I was involved with a trading system that had little or no written test case documentation. I was responsible for the release management and I always gathered information for the testers regarding exactly which modules had changed in the release and any interesting anomalies that I had observed during the build, deploy and initial smoke testing. The testers had an exhaustive home-grown regression test process that suffered from little or no direct input from the customers (or even the customer liaisons). As luck would have it, I found myself in a meeting with some of these subject matter experts. I remember taking a political risk and challenging them by asking for one hour of their time to write test cases. I promised that they would see the value in exactly one hour if they gave me the chance.  I admit that I was pretty scared when I showed up at this internationally recognized trading exchange armed only with my trusty laptop computer.

The Golden Hour
In emergency medical services we talk about the “golden hour”.  This is the first hour of care after a traumatic incident. EMS professionals, including this writer, know that the care given during the golden hour is often the decisive factor in whether or not the patient will recover. My own golden hour came when I started interviewing the SMEs and coaching them in how to write effective test cases. Within the first hour, one of the men picked up the phone and called the development manager in charge of the project. The SME told the development manager to stop their work on a particular feature explaining that, “what we have asked for is not what we want.” Getting the subject matter experts to think about testing the application caused them to realize that they had originally requested  the wrong functionality. Cognitively, these very smart professionals had gone from passive to active mode and realized how the system should really behave.  Writing active and live test cases, and using them as requirements, is a strong best practice that should be used on both agile and non-agile projects to improve quality.

Quality through better process
Improving our processes through Agility is all about making the right choices to implement the right practices. More than that it is about becoming Agile in the way that we approach technology efforts. Frameworks such as SCRUM, XP, paired programming and many others have shown great success in improving quality and productivity. Please share with us your own views on what works and what doesn't as Agility grows into a framework that is synonymous with quality!

About the Author
Bob Aiello is the Assistant Editor for the Agile Journal, Editor-in-Chief for CM Crossroads and an independent consultant specializing in Software Process Improvement including Software Configuration and Release Management. Mr. Aiello has over 25 years experience as a technical manager in several top NYC Financial Services firms where he had had company-wide responsibility for CM, often providing hands-on technical support for enterprise Source Code Management tools, SOX/Cobit compliance, build engineering, continuous integration and automated application deployment. Bob is a long standing member of the Steering Committee of the NYC Software Process Improvement Network (CitySPIN), where he serves as the chair of the CM SIG. Mr. Aiello holds a Masters in Industrial Psychology from NYU and a B.S. in Computer Science and Math from Hofstra University. You may contact Mr. Aiello at This e-mail address is being protected from spam bots, you need JavaScript enabled to view it This email address is being protected from spam bots, you need Javascript enabled to view it   or link with him at http://www.linkedin.com/in/bobaiello.

This email address is being protected from spam bots, you need Javascript enabled to view it ' ); //

Comments (4)

BobAiello: ... Given all the feedback, that I have received, I should probably divulge that Dr. Adam Kolawa is the colleague who I heard speak about quality and productivity competing for the same resource. I heard him speak at a meeting of the NYC Process Improvement Network (www.nycspin.org) which I also highly recommend. Here the link to a review of Dr. Kolawa's book that I was priviledged to write some time ago http://www.cmcrossroads.com/content/view/10219/135/ I really appreciate all the notes that I have received on this article! Bob Aiello http://www.linkedin.com/in/BobAiello 1 September 23, 2008 Edgardo Gonzalez, MEng, PMP, CSM, CMC: ... Interesting article! In my world, quality and productivity cannot be separated. If the testers find defects it slows down your team velocity – which is one of the key measures of team productivity. So how you harmonize these two imperatives? Since quality can be defined as meeting the needs of the users – defined through requirements – I have been applying a technique that has resulted in consistent superior results, whether applied to a waterfall, iterative or agile approaches. First the business articulates their expectations via just-in-time business requirements and acceptance criteria documented via Analysis Realizations; second, the system architect and developers formulate sequence diagrams and views of participating classes to understand how these requirements are integrated in the system architecture via Design Realizations; third, the developers formulate requirements validation prototypes, integrating their understanding of requirements and the test scripts – before any line of code is written – to validate the understanding of what the business expects and formulate success criteria – we call it Test Realizations. I took on a project that was generating an average of 20 bugs per function delivered; through the application of this approach, we reduced it to one per four functions delivered. This was achieved with an incremental cost of about 6% which resulted in a 30%+ increase in velocity. The developers first though that this was a waste of time – some still fight it because they cannot or want to think before they act. In my world, if an individual cannot explain what they understand about a problem they need to tackle, they cannot do it; and furthermore, if there is no success criteria that you can assess their work completed against, the work would likely be incomplete or faulty. In summary, quality should be integrated in the mantra or DNA of a project team since in the end bugs and rework will end up costing you upwards of 20% in time and money to fix. Quality is not the responsibility of the test teams - period! 2 September 23, 2008 Steve Gonzalez: ... Thanks for the article. I especially liked your colleague’s thought that quality and productivity compete for the same space. Too often, managers believe that quality is the end result of productivity. Certainly it could be, but, if quality really is the goal, they’re nearly mutually exclusive. That’s not to say that productivity is a bad thing (it’s not), but developers do need to slow down and run more tests or else hasty coding can result in a considerable compromise of quality. What I do think works about agile frameworks like Scrum is really simple, but, in contrast to waterfall, still pretty radical. Instead of developing in a vacuum and doing tests at the very end, QA takes place every sprint, throughout the lifecycle. That means that problems with quality surface immediately, so they can be addressed long before the release date. http://agilemethodology.org/ 3 August 29, 2008 Tony Greening: ... I whole heartedly agree! As Bob Martin said at Agile 2008 (paraphrase), we need to be craftsmen and stop producing bad software. Quality should be desinged in using Acceptance Testing, Unit Testing, CI, etc. rather than being "caught" by QA. by then it's too late. 4 August 13, 2008

Write comment

Name

Comment

Write the displayed characters

Add Comment

Tags:

Click to add your tags...,

or Close