RTF Explained 
Running Tested Features is a metric used to measure the delivery of running, tested features to business clients, and collect their feedback. Suggested by Ron Jeffries as a way to increase agility and team productivity by continuously monitoring the software development effort from project inception through delivery, RTF is simple to define.

Break the software down into features that represent observable value to the business. For each feature, create one or more automated acceptance tests that validate the feature. Consolidate all features into a single, integrated product and continuously execute the automated acceptance tests. At any given moment, the project team should know the number of features that are passing all acceptance tests. This number represents the RTF metric.  

 Figure 1: Sample RTF growth

As discussed in An Agile Resolution, choosing the correct tools has a tremendous impact on a teams ability to remain agile. Agile tools are testable, non-invasive, easy to use, flexible, and lightweight. As RTF requires continuous measurement of automated acceptance tests, measuring RTF requires a tool. A few such tools exist in the open source space.

FitNesse is a testing tool, wiki, and web server all rolled up into one. FitNesse allows project team members to collaboratively define acceptance tests by creating tables on web pages using wiki markup. FitNesse wiki pages integrate with test fixtures that invoke application code, making FitNesse a testing tool that doesn't rely on web based testing, but does require test fixtures that have the ability to directly invoke application code.

Selenium simulates the user by executing its tests directly in a browser. SeleniumIDE is a plug-in for FireFox that allows you to record test scripts. Scripts can be included on an HTML suite page using Selenium Core, or converted to Java, C#, Perl, Python, or Ruby for inclusion in an xUnit style test case using Selenium Remote Control.

Both FitNesse and Selenium are relatively agile tools. FitNesse does require running the FitNesse server, as well as test fixtures that can interact directly with the code being tested, so the test code must be deployed with the application under test. Selenium Core comes with the same restrictions, but Selenium Remote Control does not. Instead, Selenium Remote Control provides a Selenium Server that is run and communicates with the browser via AJAX, allowing you to test a web application in a very non-invasive style. Below is a simple scenario showing how to use Selenium Remote Control to measure Running Tested Features in Ruby. The same techniques could be applied using Java, or .NET with FitNesse, or any other acceptance testing tool.

Example: Selenium Remote Control and Ruby
The first step is downloading Selenium Remote Control and starting the Selenium Server. The Selenium Server is a Java application bundled as a .jar file. To begin, we download Selenium Remote Control, unzip the file, and from a command prompt navigate to the install directory and execute the following command on Windows:

Next, we create a Selenium test case. For demonstration purposes, I have created a simple Ruby web application that calculates the payment schedule for a loan given the interest rate, term of the loan, and the loan amount. We now want to script a test case that provides these three values. To do this, we'll use Ruby's unit testing framework in conjunction with Selenium's ruby client driver, as shown in Listing 1. The setup method instantiates the Selenium class and assigns it to the sel variable. The test_loan method issues Selenium commands that open the desired browser, types the input, and submits the values. We'll wait five seconds for the page to load before using another selenium command to access the page title to verify the returned page was what we expected. Additional commands can be issued to gain access to values on the page.

require 'test/unit'

require 'selenium'

class SeleniumTest < Test::Unit::TestCase

        def setup

                @sel = Selenium::SeleniumDriver.new("localhost", 4444,

                        "*firefox", "http://localhost:8181", 15000)

                @sel.start

        end

        def test_loan

                @sel.open("http://localhost:8181/Loan.html")

                @sel.type("rate", "6.0")

                @sel.type("term", "60")

                @sel.type("principle", "10000")

                @sel.click("clc")

                @sel.wait_for_page_to_load("5000")

                assert_equal("Loan Payment Schedule", @sel.get_title())

        end

        def teardown

                @sel.stop

        end

end

Now it's time to run the test case. We can execute the test using the following command, where -I specifies the location of Selenium's Ruby client driver:

The -I command-line option specifies the directories that are prepended to the Ruby $LOAD_PATH. Note that there was no need to specify the location of the web application we are testing. The URL for the web application is specified in the Selenium test case above, so the tests can be executed on a machine separate from the web application, meaning the test code is not deployed with application code resulting in non-invasive style of testing.

To continuously measure RTF, however, we'll want a repeatable build process, and output in a format that can be measured. Using Rake and Test::UI::Reporter, we can setup a robust build script that can be invoked by a build scheduler, and generate output in either html or xml format. After including the SeleniumTest into an acceptance test suite that uses Test::UI::Reporter for html output, as shown in Listing 2, we incorporate execution of OutputAcceptanceTests into our Rake build file, as showing in Listing 3.
 

require 'test/unit/testsuite'

require 'test/unit/ui/console/testrunner'

require 'test/unit/ui/reporter'

require 'stringio'

require 'SeleniumTest'

suite = Test::Unit::TestSuite.new

suite << SeleniumTest.suite

require 'fileutils'

FileUtils.mkdir_p 'build/report/acceptance'

#Test::Unit::UI::Console::TestRunner.run(suite)

Test::Unit::UI::Reporter.run(suite, 'build/report/acceptance')

Listing 2
 

desc "Execute all acceptance tests"

Rake::TestTask.new(:acceptance_test) do | tsk |

 tsk.libs << "./test;./selenium-remote-control-0.9.0/ruby"

 tsk.test_files = "OutputAcceptanceTests.rb"

end

Invoking the build is as simple as typing rake at the command prompt. After navigating to the build/report/acceptance directory, we can open index.html, revealing the results of the test cases, as shown in Figure 2 below.

Running Tested Features is a way to continuously measure the progress of software development in business terms. RTF allows software teams to understand the current state of the development effort throughout the lifecycle by measuring the number of functional features desired by business stakeholders that pass automated acceptance tests. But to reliably and continuously measure RTF, tools must be employed that enforce the discipline necessary for continuous and reliable measurement. Applying tools correctly, and in a way that fits your environment and culture is a critical success factor when adopting an RTF strategy. Open source tools such as FitNesse and Selenium integrate well with many build tools, and offer a great foundation upon which to build a robust RTF strategy.

About the Author
Kirk Knoernschild is Senior Technology Strategist at TeamSoft, where he leads based on his firm belief in the pragmatic use of technology. In addition to his work on large development projects, Kirk shares his experiences through courseware development, teaching, writing, and speaking at seminars and conferences. Kirk has provided training to thousands of software professionals, teaching courses on UML, Java J2EE technology, object-oriented development, component based development, software architecture, and software process.

{mos_sb_discuss:10}

Comments (0)

Write comment

Show/hide comment form