Software development is seen as
chronically chaotic and complex to the extent that project management can achieve
little control over projects or outcome [1,
2].
Recently, we have come into a new era of hope; hope of getting people -
real people, users, both naïve and sophisticated - more involved with, more
relevant to, and more visible in software development.
This hope is not merely on one
front. In software engineering, agile methods (such as Extreme Programming)
offer the promise of manageable projects, honest schedules, and increasingly
useful end products. Similarly, usability methods (such as Scenario-Based
Design) promise the possibility of clean, usable interfaces and interactions. Both
of these areas rely heavily on heuristics.
Heuristics are defined by two central ideas: they are guidelines, rather
than fixed boundaries, and are designed to evaluate and explore the problem at
hand in an efficient manner.
However, there remains a divide;
software engineers largely look at the internals of software and usability
engineers look at the externals. Heuristic methods, such as those described
above, have revolutionized and re-energized each field and offer us a bridge.
Rather than one group seeking to tightly control the design process, all
developers can (with these methods) use their energies to identify and leverage
design opportunities. When all developers share and integrate, rather than make
fiefdoms, all can make the best use of their resources.
A New Hope
As software
engineering matured, new human-centric methods emerged that can deal with the
socially constructed needs of conformability and changeability [3]. Similarly, the ‘software
psychology' of the seventies matured into usability engineering [4]. There was a drive to treat programmers and users
as humans, not merely logicians and input devices. The remainder of this section will describe
the overlap between these two areas and how they can mutually benefit each
other.
Connections Between
Extreme Programming and Scenario-Based Design
Extreme Programming (XP) [5] is a
developer-centered agile method that is well-documented, well-studied, and embodies core principles
common to most agile methods [6,
7].
Scenario-Based Design (SBD) was created not only to design user
interactions with software, but to manage larger-scale issues of how people and
software interact. Though they focus on
different areas of development, XP and SBD share many common and analogous
practices.
1. Story-Driven Development
In XP's
method for eliciting and documenting system needs, developers (and often users)
write and discuss stories about the system within the context of its
use-thereby making validation issues as visible to the designers as
verification issues. Similarly, SBD uses stories of users interacting with the
system, written in the form of scenarios, to help guide developers in
constructing the system. Stories are
dynamic; they show goals and sequences, but they also show uncertainty,
decision-making, and other complex aspects of the environment. Formal
requirements often de-emphasize, avoid, or design these aspects out of the
process. It is through this process of
contextualization that the potential impact and value of the technology becomes
central to the design, thus increasing the usefulness and usability of the end
product. Acceptance criteria for
heurists lie in users and uses, rather than test protocols generated at some
arbitrary point in development.
2. User Involvement
The logic
behind user involvement in the development project is simple; the more active a
role the user takes in development, the more likely it is for the product to
represent user's needs. As with XP, users are a critical part of SBD
development efforts. SBD does not express a static role for the user. Users can
author, analyze, and discuss scenarios, analyze claims, participate in brainstorming,
or even share in developing solutions - all elements of participatory design.
Implicit in all participatory design is a shift in power; the developer still
maintains a degree of control, but the engaged user-participant wields more
power than the typical user-as-subject of traditional development efforts.
3. Testing Emphasis
A biased
view is that a 'complete and correct solution' is derived from a proof-based
design, and thus needs no further testing. Correctness and completeness, while
valuable in theoretical computer science, are not achievable in real-world
software development. Heuristic methods thus always require thorough testing.
XP emphasizes the role of defects in users' perceptions of a system, and that
catching errors early through comprehensive tests at the unit, integration, and
user level, minimizes costs [8]. Similarly, SBD has a heavy focus on testing via
claims analysis (in the form of claims and tradeoff analysis); for each claim
about the system, both positive and negative impacts are explicitly explored
and reviewed over time. Another form of testing in SBD is careful observation
of user interactions (in part because users do not always understand the
implications of what they are doing).
4. Iteration and Redesign
Because
design and development are tightly coupled, with agile methods there is an
explicit assumption that the system will need to undergo continuous
modifications, especially in initial development. The typical notion of
‘correcting one's mistakes' is not appropriate here; it comes with a strong
implication that the correctness and completeness are possible, and this is
contrary to the heurist's creed. Because
technology and context are tightly coupled, the organization will change with
the introduction of the new technology [3], possibly in ways that designer did not intend.
As with XP,
SBD rejects the assumption that interactions can be 'gotten right' at the first
attempt. iteration takes the form of a task-artifact cycle [9] in which development creates scenarios. These
scenarios are then used to understand an interaction, which is used to develop
a system feature (or artifact). Claims analysis of is used to evaluate the
positives and negatives of that artifact, the claims are used to modify the
scenarios, and the cycle continues. Iteration allows the heurist to break from
any existing task to take advantage of the fortuitous occurrence.
5. Multiple Inputs
Both XP and
SBD de-emphasize the traditional view of development as solitary effort;
whether emphasizing participatory design, stakeholder review of regular
deliverables, or pair programming, the goal is to constantly review all
artifacts. This can be chalked up to the maxim that ‘two heads are better than
one.'
Meeting of the Minds: An Integration Opportunity
Scott
Ambler has begun drawing practical connections between his own work on Agile
Modeling and User-Centered Design and User Experience Design [10]. Some work has been done in recognizing the
overlap and working to integrate the two areas; in particular, Alexander and
Maiden's Scenarios, Stories and Use Cases
[11] has looked at
story driven development from an integrated standpoint, although heuristics are
not emphasized. Lynn Miller reported on
the development organization at Alias, where separate agile and usability teams
worked in interconnected, parallel development tracks, using an iterative,
heuristic design approach [12].
Table 1. Common features between the methods,
and how they are implemented
We have
shown the areas where XP and SBD are similar and compatible (
). The integration opportunity is
one of synergy, the belief that the whole will be greater than the sum of the
parts. By combining heuristic approaches in software engineering with
compatible approaches in usability engineering, we will have a unified
development process (an admirable goal), and also an improved process centered
on people, their tasks, and their needs, but bounded by achievable limits and
value considerations. Tables 2 and 3 show the complementary features of the two
approaches. These are features that are not shared, but can provide new
insights to the other.
Table 2. Complementary features of
Extreme Programming, and how Scenario-Based Design can benefit
Table 3. Complementary features of
Scenario-Based Design, and how Extreme Programming can benefit
To conclude
that we have all of the answers denies the fundamental, exploratory tenet of
heuristics. XP and SBD have brought advances to software engineering and
usability engineering, but advancements still can and need to be made,
especially through integrating these two methods. Aside from the core
components of system and interaction, we have opportunities for better system
integration into environments, better documentation, lower-cost and lower-risk
projects, and longer-term system viability. Integrating and sharing between heuristic
methods will improve these and other development aspects.
A Bold New Future
Uniting heuristic methods not only
provides an avenue for improving those methods, but also creates the potential
for faster adoption. Below we describe some mechanisms for developing and
implementing combined methods in industry, education, and research.
In Industry
Industry often leads academia.
Industry likes what works, and clearly XP and SBD work. We must encourage
developers to strengthen their skills in both areas. One concern common in
industry is funding. Fortunately, one of the principle advantages of heuristic
methods is efficiency; this is clear from a management standpoint [13], and new models are emerging showing the cost
advantage of agile methods [14] and usability [15].
Those software developers that
believe in heuristic methods such as Extreme Programming should not let
themselves be fooled by simple guidelines for interface development. These
crutches are playing off developers' tendency to be drawn towards 'correct and
complete' approaches. Instead, they should look for and be pointed towards
heuristic interface design methods, such as those found in SBD.
In Research
Research communities learn from
industry by monitoring it for success, failure, and opportunities for
improvements. Researchers have time to analyze fine detail, the freedom to try
sometimes aggressive approaches, and the requirement to explain and connect
practice to theory. Industry can take advantage of this thoroughness. SBD has
roots in industrial and academic research, but research based around real-world
problems [9]. The balance found in grounded, practical research
is a fruitful testing ground for new ideas, as it demands rigor in both
academic thoroughness and industrial applicability.
Researchers
must work for the cross-pollination of software and usability engineering.
Coordination and collaboration are necessary, but research projects need not
focus on method development in both areas. Many research projects require
software development and user interface development, and the methods that have
proved productive in the field can (and should) be adopted in the lab. This
cross-use may be sufficient to begin cross-pollination, as the researcher will
inevitably develop and refine the methods and heuristics, and publish these
additions, alterations, and improvements.
In Education
We have
long recognized that students leave typical computer science undergraduate
programs without the skills to develop software in industry. Undergraduate
programs in software development, software engineering, information systems,
and HCI are at the forefront of the march towards addressing this problem.
These programs are one venue for coursework that integrates agile methods and
Scenario-Based Design. 'Usability and usefulness‘ will (where appropriate)
supplant 'correctness and completeness‘ as a mantra. The ideal education will
give students a good theoretical understand as well as practical experience
with cutting-edge industry practice. One final note: the best possible
implementation of these concepts as course material would be integrated, in
either a single course or a coordinated pair of courses.
Final Comments
Heuristics are neither formal nor
wrong; this fact violates the beliefs of many about appropriate, 'scientific'
methods. Heuristics are powerful tools [16], and those of us who use heuristic methods would
benefit greatly from joining together and learning from each other. The goal is
not merely to justify ourselves and our work, but to advance the state of
knowledge. Working towards an integrated community of discourse among heurists
in industry, research, and education will help us toward both ends.
Acknowledgements
The authors
would like to thank Steve Sawyer, Jack Carroll, and Scott Ambler for their
comments on an earlier draft of this paper. They would also like to thank Arun
Batchu, Robert Biddle, Fred George, Kyle Larson, Scott McCrickard, Mary Beth
Rosson, and Laurie Williams for sharing their insight into methods.
About the Authors
Joshua B.
Gross has spent ten years in the software development industry, as a software
engineer, software architect, and project manager, working for Fortune 500
companies and startups. He holds an MS in software engineering from the
University of St. Thomas. He is currently an independent consultant while he
completes his PhD at Penn State.
John M.
Daughtry III has been involved in software development since 2000. Currently,
he is an Associate Research Engineer and PhD candidate at Penn State. He holds
an MS in Information Sciences and Technology from Penn State, and a BS in
Computer Science from North Carolina State University.
Jason Chong
Lee is a PhD student at Virginia Tech exploring ways to integrate agile
software development and usability engineering methodologies. He holds a BS from the University of
Virginia, and is currently collaborating with Meridium, Inc. to research and
develop agile usability methodologies.
References
[1] F. P.
Brooks, Jr., The Mythical Man-Month:
Addison-Wesley, 1975/1995.
[2] F. P. Brooks, Jr., "No Silver Bullet:
Essence and Accidents of Software Engineering," IEEE Computer, vol. 20, pp. 10-19, 1987.
[3] W. J. Orlikowski, "Using Technology
and Constituting Structures: A Practice Lens for Studying Technology in
Organizations," Organization Science,
vol. 11, pp. 404-428, 2000.
[4] J. Nielsen, Usability Engineering: Morgan Kaufmann, 1994.
[5] K. Beck, Extreme Programming Explained: Addison-Wesley Professional, 1999.
[6] A. Cockburn, Surviving Object-Oriented Projects: Addison-Wesley Professional,
1997.
[7] A. S. Koch, Agile Software Development: Evaluating the Methods for Your
Organization: Artech House Publishers, 2004.
[8] B. Boehm, Software Engineering Economics: Prentice Hall, 1981.
[9] J. M. Carroll, Making Use: Scenario-Based Design of Human-Computer Interaction.
Cambridge, MA: The MIT Press, 2000.
[10] S. Ambler, "Agile Usability: User
Experience Activities on Agile Development Projects."
[11] I. Alexander and N. Maiden, Scenarios, Stories and Use Cases: John
Wiley & Sons, 2004.
[12] L. Miller, "Case Study of Customer
Input for a Successful Product," presented at Agile 2005, 2005.
[13] M. Poppendieck and T. Poppendieck, Lean Software Development: An Agile Toolkit
for Software Development Managers: Addison-Wesley, 2003.
[14] M. Denne and J. Cleland-Huang, Software By Numbers: Low-Risk, High-Return
Development: Prentice Hall, 2003.
[15] R. G. Bias and D. J. Mayhew, Cost-Justifying Usability, 2 ed: Morgan
Kaufmann, 2005.
[16] G. Gigerenzer and P. M. Todd, Simple Heuristics that Make Us Smart.
Oxford, UK: Oxford University Press, 1999.
|
