We have been taught that the best way to solve the tough challenges inherent in software development efforts is to treat software development as an engineering discipline. Stabilize requirements early, and then follow by analysis and design, implementation, verification, and deployment. Throughout each of these lifecycle phases, teams produce traceable artifacts where changes to one artifact is ideally traced through to other artifacts, eventually leading to the source code modules that must change. But traceability is fundamentally flawed due to the manual effort required to synchronize artifacts. As deadlines loom and pressure mounts, we do not possess the discipline to ensure consistency across all artifacts. Slowly, requirements specifications and design models diverge and no longer offer an accurate view of current system behavior or structure. Avoiding the issues inherent to traceability demands we shift from traceable artifacts to executable artifacts.
Executable Artifacts Defined
An executable artifact is any software artifact that is executable against the source code. Executable artifacts produce output that can be analyzed by project stakeholders to gain additional insight to the development effort. Example of executable artifacts include test scripts, a build and deploy process, UML graph generators, and documentation tools. While executable artifacts cannot serve as the only form of software artifact, they can complement nicely many of the software artifacts that we have grown comfortable with. In fact, executable artifacts have numerous advantages over other types of software artifacts.
They are an accurate reflection of the code at the point they were run. For most teams, ensuring documentation stays synchronized with the codebase is a daunting task. Because executable artifacts are run against the source code, we can always be certain that they reflect the current state of the source.
They can be run at any time, with little overhead. After initial setup of the executable artifact, they can be easily run against the source code, usually by issuing a simple command.
They encourage early and frequent delivery. To realize the benefits of executable artifacts requires source code that can be analyzed. Subsequently, to leverage that value requires teams to write code early and use the feedback from the executable artifact as a contributing factor in driving future development work.
They can be automated. Including the executable artifact as part of your build process means little manual intervention is required to generate their output. Additionally, by automating we reduce the likelihood of errors or omissions that is common with manual processes.
They maintain a reliable history. An executable artifact, run at frequent and regular intervals, produces output that can be analyzed over time to help the team identify trends in development.
Executable artifacts can
Advertisement
be applied throughout the software lifecycle, and directly support many of the macro lifecycle phases. Early in the lifecycle, as teams elicit requirements, automated acceptance tests can be created that verify that each requirement is implemented correctly. As design and architecture come into focus, generated metrics can serve as an objective measure of the resiliency of software structure. Documentation tools can be used that generate important documentation that informs the development team of application APIs. Coverage tools can be used to analyze the percentage of application under test.
Undoubtedly, creating executable artifacts requires tools that are non-invasive, easy-to-use, flexible, and lightweight. Table 1 illustrates some useful tools supporting the development lifecycle. All are open-source and can be adopted with minimal effort and overhead.
Phase
Tools
Requirements
FitNesse and Selenium are acceptance testing frameworks that can be used to script user acceptance tests.
Analysis and Design
UMLGraph generates UML class diagrams for Java source code.
JDepend is a static source code analyzer for Java that offers useful design quality metrics related to Java packages.
JarAnalyzer and AssAnalyzer are static source code analyzers that offer design quality metrics for Java Jar files and .Net Assemblies.
Implementation
PMD and FxCop are static source code analyzer that examine the source code for adherence to coding standards.
JavaNCSS is a source code analyzer for Java that reports the Cyclomatic Complexity number, a metric detailing the complexity of individual methods.
The xDoc (JavaDoc, NDoc, RDoc) family of documentation utilities generates important API information for Java, .Net, and Ruby, respectively. NDoc does not current support .NET 2.0, though alpha versions are available that do offer support. Recently, Microsoft has released Sandcastle, which produces MSDN style documentation by reflecting over .NET source assemblies.
Testing
The xUnit (JUnit, NUnit, and Test/Unit and RSpec) family of frameworks allows developers to create a suite of regression tests that serve to validate class behavior in isolation.
Emma and RCov are code coverage analyzers that report on the percentage of application classes that are under test.
Table 1: Useful Open Source Tools
Samples
Below, we examine three samples of executable artifacts and the output they produce.
Running Tested Features
In Keep Moving Forward with Running Tested Features, I presented some guidelines to consider when adopting an RTF strategy using Selenium. The automated acceptance tests created using Selenium IDE are executable artifacts that can be automated using Selenium Remote Control, creating the appropriate output that allows you to track the RTF metric.
Code Coverage
Code coverage is a useful metric that helps teams identify and manage areas of the application that are under test. As teams identify areas that are lacking the appropriate test coverage, they can target that area to improve coverage. Test coverage is an important factor in guiding the refactoring effort, as developers who can verify that the code they are changing is under test can rest easier knowing that any errors introduced as the result of refactoring will be caught.
RCov is a code coverage utility for Ruby development. After installing RCov, it can be easily included in a Rake build file. Listing 1 shows a snippet of code in a Rake build file that uses RCov to monitor test execution and report on code coverage. Figure 1 shows a portion of the output produced when run against a sample Loan Payment Calculator application, where each class is listed along with the total lines and lines of code, along with the coverage statistics. Selecting the link, app/Loan.rb, takes us to a detailed page where lines of code not currently under test are highlighted in red, as illustrated in Figure 2.
require 'rcov/rcovtask'
Rcov::RcovTask.new(:coverage) do |t|
t.test_files = FileList['test/*.rb']
t.rcov_opts << "--include app"
t.output_dir = "coverageout"
end
Listing 1: Sample RCov output.
Figure 1: Sample Code Coverage Report
Figure 2: Sample Detailed Code Coverage Report
Dependency Metrics
In Using Metrics to Help Drive Agile Software, I talked about Dependency Metrics. AssAnalyzer is a tool that reflects on .NET assemblies and generates useful metrics illustrating assembly dependencies. After some simple configuration describing the assemblies you'd like to analyze, running analyze.exe will produce two useful output artifacts. After execution against a few Mono assemblies, Figure 3 shows an analysis report that includes many useful dependency metrics, while Figure 4 illustrates a component diagram showing the relationships between the .NET assemblies analyzed, as well as their direct dependent assemblies.
Figure 3: Sample Analysis Report
Figure 4: Sample Component Drawing
Executable artifacts can be used to create a number of valuable outputs. As the number of executable artifacts increases, providing a single point of access to their outputs is beneficial. Creating a project dashboard website allows all project stakeholders to quickly gain an immediate assessment of project status. Updating the project dashboard with the results generated as part of the most recent run of a scheduled build ensures the outputs are always current. Figure 5 shows the default project dashboard for CruiseControl.rb. There are two projects currently configured for scheduled builds. Selecting a specific build for a project allows stakeholders to view information such as the coverage report and dependency metrics above.
Figure 5: Sample CruiseControl Dashboard
Rolling Your Own
There are many useful websites where you can find utilities that help create executable artifacts, such as Java-Source.net and CSharp-Source.net. But in some cases, you may have an original idea or specific need for which no tool currently exists. In these situations, there are other useful utilities that allow you to roll your own.
GraphViz is open source graph visualization software that includes several graph layout programs. One common program, dot, can create drawings of directed graphs. The layout algorithm implemented within dot chooses the most suitable layout based on the relationships between the nodes.
Cecil is a Mono project that can load .NET assemblies and browse the contained types. Cecil has a relatively straightforward API that allows you to examine class methods, attributes, identify dependencies, and more.
Byte Code Engineering Library (bcel) is an Apache project that can inspect Java .class files. Using bcel, you can examine class methods, attributes, examine relationships between classes, and more.
Using these tools in conjunction with others, allows you to create some powerful solutions. For example, UMLGraph uses GraphViz to generate its output file. JarAnalyzer and AssAnalyzer use bcel and Cecil, respectively, to parse compiled code. Once parsed, each have the option of writing their output to a dot compliant text file. Executing dot against the text file generates the component diagram, an example of which is seen in Figure 4.
Conclusion
Executable artifacts compliment and, in some cases, can replace artifacts once created manually. Among their many benefits, these executable artifacts offer useful feedback on the current state of the source code and serve as an excellent contributor in assessing the team's development status.
About the Author
Kirk Knoernschild is Senior
Technology Strategist at TeamSoft, where he leads based on his firm belief in
the pragmatic use of technology. In addition to his work on large development
projects, Kirk shares his experiences through courseware development, teaching,
writing, and speaking at seminars and conferences. Kirk has provided training to
thousands of software professionals, teaching courses on UML, Java J2EE
technology, object-oriented development, component based development, software
architecture, and software process.
Comments (1)
jrrousseau: ...
In the requirements categoty, GreenPepper (www.greenpeppersoftware.com) is a new product that looks really good. It helps build executable specifications. Its core engine is Open source
Home 
