A prime directive of agile development is to maintain forward progress. In past articles, I've talked about metrics that gauge the quality of code and design. While these metrics help guide refactoring efforts that ensure application maintenance remains easy, they don't necessarily allow us to measure our progress in terms that make sense to the business customer. A good way to measure forward progress is to know where the development effort stands in relation to the functional features desired by business stakeholders. Running Tested Features (RTF) is a way to track progress by measuring how many features pass acceptance tests. In this article, I'll introduce RTF and present some tools that can be adopted to implement an RTF strategy.
RTF Explained
Running Tested Features is a metric used to measure the delivery of running, tested features to business clients, and collect their feedback. Suggested by Ron Jeffries as a way to increase agility and team productivity by continuously monitoring the software development effort from project inception through delivery, RTF is simple to define.
Break the software down into features that represent observable value to the business. For each feature, create one or more automated acceptance tests that validate the feature. Consolidate all features into a single, integrated product and continuously execute the automated acceptance tests. At any given moment, the project team should know the number of features that are passing all acceptance tests. This number represents the RTF metric.
Continuous RTF
RTF is a continuous measurement and, all things being equal, should grow linearly throughout the project. To explore what this means, we must first compare agile development with iterative development. Iterative development teams tend to timebox their iterations,
Advertisement
marking each iteration with a milestone focused on delivering a planned set of features to clients every two to four weeks. Iteration planning defines the work included in future iterations based on the amount of work accomplished in previous iterations. Yet within these iterations, teams should deliver functional and valuable features daily, hourly, or even more frequently. Teams should not be constrained to delivering features at the end of each iteration. While project management defines iterations that last weeks or months, primarily for planning, the development team works in iterations that lasts days, hours, or minutes. And each feature that is released throughout an iteration planning window is a feature that can be measured and contributes to RTF.
In The Agile Matrix, I talked about dividing teams into smaller teams according to the business processes they will implement and support. Each process team is responsible for fulfilling the requirements for a complete business process. But on many software development efforts, we aim to align each process team to the same iteration schedule. This is not necessary. Each process team can work within its own iteration planning window. Agile practices, including continuous integration, ensure individual process teams remain aligned, and the processes of continuous release, continuous builds, and continuous deploys encourages the smaller iterations that often go unacknowledged on many development efforts. It is these smaller iterations where software growth occurs, and where we continuously measure Running Test Features.
Figure 1 depicts a multi-dimensional agile development effort, where iterations overlap but where RTF experiences linear growth as each iteration nears completion. Critics might argue that RTF cannot experience consistent linear growth due to unpredictable setbacks throughout the development lifecycle. While it's true that teams can experience challenges that limit RTF growth for a short period of time, analyzing RTF reports over time will show positive growth.
Figure 1: Sample RTF growth
RTF Strategy
As discussed in An Agile Resolution, choosing the correct tools has a tremendous impact on a teams ability to remain agile. Agile tools are testable, non-invasive, easy to use, flexible, and lightweight. As RTF requires continuous measurement of automated acceptance tests, measuring RTF requires a tool. A few such tools exist in the open source space.
FitNesse is a testing tool, wiki, and web server all rolled up into one. FitNesse allows project team members to collaboratively define acceptance tests by creating tables on web pages using wiki markup. FitNesse wiki pages integrate with test fixtures that invoke application code, making FitNesse a testing tool that doesn't rely on web based testing, but does require test fixtures that have the ability to directly invoke application code.
Selenium simulates the user by executing its tests directly in a browser. SeleniumIDE is a plug-in for FireFox that allows you to record test scripts. Scripts can be included on an HTML suite page using Selenium Core, or converted to Java, C#, Perl, Python, or Ruby for inclusion in an xUnit style test case using Selenium Remote Control.
Both FitNesse and Selenium are relatively agile tools. FitNesse does require running the FitNesse server, as well as test fixtures that can interact directly with the code being tested, so the test code must be deployed with the application under test. Selenium Core comes with the same restrictions, but Selenium Remote Control does not. Instead, Selenium Remote Control provides a Selenium Server that is run and communicates with the browser via AJAX, allowing you to test a web application in a very non-invasive style. Below is a simple scenario showing how to use Selenium Remote Control to measure Running Tested Features in Ruby. The same techniques could be applied using Java, or .NET with FitNesse, or any other acceptance testing tool.
Example: Selenium Remote Control and Ruby
The first step is downloading Selenium Remote Control and starting the Selenium Server. The Selenium Server is a Java application bundled as a .jar file. To begin, we download Selenium Remote Control, unzip the file, and from a command prompt navigate to the install directory and execute the following command on Windows:
java -jar ./server/selenium-server.jar
Next, we create a Selenium test case. For demonstration purposes, I have created a simple Ruby web application that calculates the payment schedule for a loan given the interest rate, term of the loan, and the loan amount. We now want to script a test case that provides these three values. To do this, we'll use Ruby's unit testing framework in conjunction with Selenium's ruby client driver, as shown in Listing 1. The setup method instantiates the Selenium class and assigns it to the sel variable. The test_loan method issues Selenium commands that open the desired browser, types the input, and submits the values. We'll wait five seconds for the page to load before using another selenium command to access the page title to verify the returned page was what we expected. Additional commands can be issued to gain access to values on the page.
Now it's time to run the test case. We can execute the test using the following command, where -I specifies the location of Selenium's Ruby client driver:
The -I command-line option specifies the directories that are prepended to the Ruby $LOAD_PATH. Note that there was no need to specify the location of the web application we are testing. The URL for the web application is specified in the Selenium test case above, so the tests can be executed on a machine separate from the web application, meaning the test code is not deployed with application code resulting in non-invasive style of testing.
To continuously measure RTF, however, we'll want a repeatable build process, and output in a format that can be measured. Using Rake and Test::UI::Reporter, we can setup a robust build script that can be invoked by a build scheduler, and generate output in either html or xml format. After including the SeleniumTest into an acceptance test suite that uses Test::UI::Reporter for html output, as shown in Listing 2, we incorporate execution of OutputAcceptanceTests into our Rake build file, as showing in Listing 3.
Invoking the build is as simple as typing rake at the command prompt. After navigating to the build/report/acceptance directory, we can open index.html, revealing the results of the test cases, as shown in Figure 2 below.
Figure 2
Conclusion
Running Tested Features is a way to continuously measure the progress of software development in business terms. RTF allows software teams to understand the current state of the development effort throughout the lifecycle by measuring the number of functional features desired by business stakeholders that pass automated acceptance tests. But to reliably and continuously measure RTF, tools must be employed that enforce the discipline necessary for continuous and reliable measurement. Applying tools correctly, and in a way that fits your environment and culture is a critical success factor when adopting an RTF strategy. Open source tools such as FitNesse and Selenium integrate well with many build tools, and offer a great foundation upon which to build a robust RTF strategy.
About the Author
Kirk Knoernschild is Senior
Technology Strategist at TeamSoft, where he leads based on his firm belief in
the pragmatic use of technology. In addition to his work on large development
projects, Kirk shares his experiences through courseware development, teaching,
writing, and speaking at seminars and conferences. Kirk has provided training to
thousands of software professionals, teaching courses on UML, Java J2EE
technology, object-oriented development, component based development, software
architecture, and software process.
Error, missing joomlaboard config file!
Comments (2)
theIntuitionist: ...
fantastic article. there isn't another on one the net right now that will get you up and running on selenium and rails.
1
May 03, 2007
Rodney Eten: ...
Good article. This seems to be one of the final links in the agile chain we are using. Link back
Figure 1: Sample RTF growth
