Tool integrations are notoriously fragile; how can we fix this problem once and for all?
The first sign of trouble is the unusually long time it takes for your IDE to start one morning. Maybe you can't access the trouble tickets any more or the team members cannot build their applications or run the automated test suite.
Your first question is "What changed?" But before anyone has time to answer you already know the culprit ... during the night some (usually nameless person) upgraded some part of the technology infrastructure and now your development organization is staggering to a halt.
Throughout the lifecycle, from inception to definition to construction to verification to implementation, development and operational communities are supported by a complex set of heterogeneous vendor, open source and homegrown technologies. These communities must
Advertisement
optimize the productivity, sustain the quality and eliminate the risk involved in the development and deployment of applications. To do this, IT infrastructures must be highly integrated and interoperable. However, today's integration model is of numerous point-to-point integrations that are fragile and offer only minimal touch points from one solution to the next.
Instead of multiple point-to-point integrations, a more modern and scalable solution is to create a common, shared interoperability framework where tools register their capabilities and orchestrations of those tools are done on the basis of the needs of individual IT organizations. In this way each tool provider, whether vendor, open source or homegrown, will now only need to support one integration- to the framework. The tool provider can expose as much (or as little) as it chooses and the consumer of these technologies will have the ultimate control over how these technologies are orchestrated together. This liberates the consumer to select what it determines are the best-in-class technologies for its needs.
At the end of February 2006, the Eclipse Application Lifecycle Framework (ALF) Project delivered the Proof of Concept demo (see www.eclipse.org/alf ). This technology is a collaborative effort of vendors in the application lifecycle management (ALM) space led by Serena Software and including Catalyst, Compuware, Segue (being acquired by Borland), Secure and about 12 other vendors. Uniquely for Open Source projects, the ALF Project even has a leading technology consumer, Zurich-based UBS, guiding and steering the project.
Defining ALF
The basic idea behind ALF is a simple one: instead of multiple point-to-point integrations, which do not scale at the enterprise level, vendors and users will integrate each technology to a central framework according to a well-defined collaboration model. In this way, each tool provider is responsible for maintaining and sustaining only one integration, with the framework, and the framework takes over responsibility for information exchange amongst the tools.
This approach can only work if tool providers agree upon how the tools will communicate with the framework. This is where the modern concept of Service Oriented Architectures (SOA) facilitates an appropriate technology solution. In the past, several attempts have been made to define a logical data map for the software industry. These attempts have usually failed because they were too prescriptive; having only a single vendor's view of the world. In the ALF world, we establish "vocabulary groups" where like-minded domain experts from tool providers and tool consumers get together to define a common set of events and information elements according to well-defined use cases. Of course, the architecture of ALF is such that rich tools can define their own optional extensions if required.
How ALF works is simple. Each vendor creates an ALF-enabling Web Service for its tool either as an integral part of the tool or, for legacy tools for example, as a wrapper around the tool's existing API. Each tool registers with ALF and reports the events that it can execute. It also registers, for each of these events, the information payload that it will need if one of these events is triggered and the information payload it will return having executed the event.
If ALF did nothing else this would be a significant improvement on the fragile point-to-point integration model of today but ALF takes integration to the next level. In the point-to-point model we have, by definition, two tools connected together. However, as we know, modern enterprise development software infrastructure comprises many disparate tools all with some business-logical, if not technological, interdependency. In the ALF context we develop service flows that orchestrate together those business logic rules across multiple tools simultaneously and these flows can run for very long periods of time.
The ALF Proof of Concept demo is a good example of what we need to achieve. The example is of a production build event. When the production build is requested from an item tracking tool, the first thing that happens is that the code is checked out of the software configuration management tool and then the build management tool builds the executables. If the build is successful, this is reported back to the item tracking tool which then simultaneously initiates a security scanning tool and an automated testing tool. As each of these steps completes, it not only returns status information but also provides access to its logs for the user.
For the demo, each of the vendors developed a Web Services interface to the framework, paying no attention to the use cases that their tools would be integrated against. Steve Taylor, CTO of Catalyst Systems, told me that it took him just eight hours to package the existing Web Services of the Openmake build management tool to be ALF-compliant. He also spent less than one hour developing a general Web Service that can be used with open source or commercial toolsthat have a command line interface. "I was really surprised how straightforward it was. I just followed the ALF conformance guidelines and I had it working," he told me later.
Tim Buss at Serena implemented the build management use case that the ALF requirements team had developed independently. Using a BPEL designer and the ALF plug-ins for Eclipse, Tim was able to simply take the events registered by the ALF-compliant tools into the orchestration defined above. The whole orchestration process came together rather quickly. Notably, this orchestration shows conditional behavior and parallel process making it a very rich example of the simplicity and kinds of integrations that are possible with ALF.
Why will an ALF-enabled world make a difference?
ALF's benefits to tool consumers are huge. We will see the stabilizing of tool interfaces. This means that we will be able to upgrade our technologies without fear of breaking some part of the integration. This will also allow us to start standardizing on the technologies we use by replacing, piece by piece rather than wholesale, any older non-standard tools. Tool consumers will have greater choice to combine best-in-class technologies without having to stick to one vendor. The increasing stability of the software development infrastructure will result in less down-time for developers. The greater reliability along with richer tool-to-tool interoperability should directly result in productivity gains. And, ensuring the business logic defined for applications development is orchestrated and enforced through technology will result in higher quality, risk-reduced and cost-minimized development.
As vendors embrace the framework, they will be able to deploy more resources to their tools and thus deliver richer functionality in their tools' areas of domain expertise. This investment will result in better, more agile technologies in support of the ALM space. The framework will be ready to deploy in October of 2006 and by that time there should be a thriving ecosystem of ALF-compliant tools.
Ultimately, ALF will become a platform for innovation with vendors developing new technologies with ALF in mind from the beginning. We expect major new releases of tools, from both software vendors and the open source community, to be ALF-enabled and, perhaps as soon as two years from now, ALF-compliant solutions will become the de facto standard. As the inventory of ALF-enabled services accumulates, we will see a proliferation of consultants and vendors recombining those services into new and innovative products that cannot be conceived or developed today.
About the Author
With 25 years of industry perspective Kevin Parker has been framing and shaping technology direction both in the US and his native UK. He is currently Vice President, Market Development and Evangelist at Serena Software. Kevin is a sought after speaker and spoke at 50 conferences last year in 11 countries. This month Kevin mailed his article from Hyderabad, India.
