The Communication Gap
Knowledge management among developers is a key ingredient of successful software projects. On smaller teams, it's easier to ensure developers are not geographically separated. This close proximity of developers to each other allows for osmotic communication, where ideas flow into the background and are absorbed by other team members, as though by osmosis.[1] Team members tend to overhear background communications, and join in as necessary based on their vested interest in the discussion or ability to make relevant contributions. Larger teams do not have this luxury. It is not feasible to shove 50 developers into a project room and expect productive collaboration. On larger teams, it's common to break developers up into smaller teams.

                                                                Figure 1

Software tends to rot over time. When you establish your initial vision for the software's design and architecture, you imagine a system that is easy to modify, extend, and maintain. Unfortunately, as time passes, changes trickle in that exercise your design in unexpected ways. Each change begins to resemble nothing more than another hack, until finally the system becomes a tangled web of code that few developers care to venture through. The most common cause of rotting software is tightly coupled code with excessive dependencies. For large teams and large applications, managing dependencies is especially important. Excessive dependencies cause numerous problems:

Dependencies hinder the maintenance effort. When you're working on a system with heavy dependencies, you typically find that changes in one area of the application trickle to many other areas of the application. In some cases, this cannot be avoided. For instance, when you add a column to a database table that must be displayed on the user interface, you'll be forced to modify at least the data access and user interface layers. Such a scenario is mostly inevitable. However, applications with a well thought dependency structure make change easier since developers have a more complete understanding of the impact of change.

Dependencies prevent extensibility. The goal of flexible software architecture is to remain open for extension but closed to modification. The desire is to add new functionality to the system by extending existing abstractions, and plugging these extensions into the existing system without making rampant modifications. One reason for heavy dependencies is the improper use of abstraction, and those cases where abstractions are not present are areas that are difficult to extend. Abstraction aids large teams by exposing well-defined extension points, as well as encapsulating implementation complexity.

Dependencies inhibit reusability. Reuse is often touted as a fundamental advantage of well-designed object oriented software. Unfortunately, few applications realize this benefit. Too often, we emphasize class level reuse. To achieve higher levels of reuse, careful consideration must also be given to the package structure and deployable unit structure. Software with complex package and physical dependencies minimize the likelihood of achieving higher degrees of reuse.

Dependencies restrict testability. Tight coupling between classes eliminates the ability to test classes independently. Unit testing is a fundamental principle that should be employed by all developers. Tests provide you the courage to improve your designs, knowing flaws will be caught by unit tests. They also help you design proactively and discourage undesirable dependencies. Heavy dependencies do not allow you to test software modules independently. Teams with few tests cannot respond to change easily due to the inability to access the impact of change.

Dependencies cause integration issues. It's easy for separate teams to plow forward, usually under tremendous pressure from looming deadlines. They operate under the false assumption that if they can simply reach the final feature destination, they can quickly pull things together toward the end of a project. This Big Bang approach to integration does not work. As individual modules are pulled together, common issues that surface include degradation of overall system performance, incorrect levels of behavioral granularity provided by system modules, and transactional incompatibilities. More frequent integration brings many of these issues to the forefront earlier in the project.

Dependencies limit understanding. When working on a software system, it's important that you understand the system's structural architecture and design constructs. A structure with complex dependencies is inherently more difficult to understand. Clear and concise dependencies that are well-thought allow teams to more easily access the impact of change.

Checks and Balances
Managing dependencies between classes, package, and components throughout the development lifecycle is imperative when developing a software product that accommodates change and meets the business objectives. While breaking larger teams into smaller groups of developers helps establish fewer channels of communication, these smaller teams tend to introduce other architectural and integration issues. For larger projects, it's likely these issues will affect your software before your first production deliverable. Agile practices applied throughout the lifecycle help minimize dependencies and encourage more frequent integration. In Activating the Lifecycle, I discussed the importance and value of an automated and repeatable build and supporting infrastructure. Such an environment is the foundation of an agile team, and serves as a system of checks and balances to ensure large teams maintain forward, sustainable, and cohesive progress.

Separate teams should be disciplined in releasing their changes to the source code repository on a frequent basis. Frequent builds help identify integration issues almost instantaneously, and help avoid the risk associated with Big Bang integration. Automated unit tests verify the integrity of code released by each team, while more complete integration and functional tests can verify behavior spanning team boundaries. Ensuring the complete system is free of compilation errors allows for frequent profiling of the application to identify serious issues related to performance, memory, and other environmental constraints. A shared environment that closely resembles the future production environment is also a critical aspect to help identify environmental issues. Additional techniques help add to this system of checks and balances to ensure development teams minimize dependencies and emphasize integration earlier in the software lifecycle:

• Dependency Metrics and Diagrams. Utilities, such as JDepend and JarAnalyzer for Java, are available that allow teams to obtain real and accurate feedback on the structural relationships between software packages and components. Understanding module and package dependencies allows developers to somewhat objectively evaluate the cost of change. As dependencies become excessive, such utilities help guide developers in prioritizing areas of the system that are higher priority candidates for refactoring.

• Levelized Build. Cyclic dependencies, recognizable as bi-directional relationships between packages or components, are especially dangerous as neither can be tested or deployed in isolation. One way to enforce acyclic relationships is building components individually, and including in the build and test execution path only those modules required to compile and test the component being built.

• Parallel Build. As systems grow, build times tend to slow. This is natural as more code must be compiled and more tests are introduced that exercise more code. When build time degradation occurs, it's important to identify ways to increase the efficiency of your build process. One technique is to build components in parallel. If dependencies are managed carefully, components with no dependencies on each other can be built simultaneously.

• Component Tests. Components built in isolation, can be tested in isolation. For each component represented by a deployable unit, creating a corresponding test component helps ensure the functional integrity of the component. Test components are typically manifest as suites of individual unit tests, and aid any refactoring to help maintain the design integrity of the component.

• Integration Tests. Test suites that verify component integration certainly help ensure that the interfaces between components remain compatible, but also help identify behavioral issues, as well. Incorporating these test suites into your build process ensures that integration tests are frequently executed, reducing the likelihood that extended periods of time pass between periods of integration.

Agility is an ability to respond to change as quickly and as easily as possible. While team structure and process are important success factors on large software projects, excessive dependencies prevent teams from changing a software system easily. Leveraging and extending proven agile practices improves feedback and strengthens collaboration, helping teams manage excessive dependencies, encourage architectural consistency and resiliency, and prevent issues from arising late in the lifecycle due to delayed integration.

[1] Cockburn, Alistair. Agile Software Development. Addison-Wesley Professional, 2006. ISBN: 0321482751

Comments (3)

kameroniza emmanuell: ... this is a great idea it was very intelligent dont pay know mind to mira jisenusa she only 9 and being stupid. 1 November 13, 2007 mira jisenusa: ... how appealing to the body 2 November 13, 2007 Will: ... Agile seems great, but everyone talks about investigating requirement details, coding, testing, refactoring, and building in the iterations in such a way to produce a production quality build. I haven't seen a lot of discussion (only by Scott Ambler) on establishing or envisioning an architecture, and also little on how to adapt an agile process to a regulated environment where many formal deliverables are required with the required traceability. The focus seems to be on the deliverables instead of the work, perhaps it makes sense to utilize tools where possible that can produce the deliverables for you so time is wasted creating the deliverable at the same time as the work. Any thought? Typically, regulated environments require a few architecture/design options to be considered and the best design is selected. Perhaps for heavier processes, there are more iterations due to the number of decisions that must be made before stable coding and testing in one framework can begin. Various designs need to be identified and proven with code and tests. An overall architecture framework, and individual component design. Perhaps these happen in their own iterations and the iteration reviews examine the builds to determine which design to move forward with. Any thought? 3 October 20, 2007

Write comment

Name

Comment

Write the displayed characters

Add Comment

Tags:

Click to add your tags...,

or Close