"If you don't know where you are going, you will wind up somewhere else." Yogi Berra

This article is an excerpt from Ken Pugh’s upcoming book – “Lean-Agile Acceptance Test Driven Development” to be published by Addison-Wesley.  Debbie, the developer, and Tom, the tester, are introducing acceptance test-driven development to Cathy, the customer.

The Triad – Tom, Debbie, and Cathy – are in their second meeting together.  Debbie describes an example of an acceptance test and four ways that an acceptance test can be executed.

An Example Business Rule
Debbie, Tom, and Cathy continue their discussion of Acceptance Test-Driven Development.  Debbie is talking about a previous project where she and Tom created tests in collaboration with the customer.

“The business representative, Betty, presented us with a business rule for giving discounts that she had obtained from one of the stakeholders.  The stakeholder wanted to give discounts to the firm’s customers.  The discount was to vary based on the type of customer.   We had already completed implementing a previous requirement that determined the type of customer.  Here’s the rule that Betty gave to us:”

If Customer Type is Good and the Item Total is less than or equal $10.00,
   Then do not give a discount,
   Otherwise give a 1% discount.

If Customer Type is Excellent,
   Then give a discount of 1% for any order.

If the Item Total is greater than $50.00,
   Then give a discount of 5%.

“This rule seems pretty clear.  It uses consistent terms such as ‘Customer Type’ and ‘Item Total’.  We had previously gotten from Betty the definitions of those terms.  (Footnote – See Eric Evan’s Domain Driven Design) For example, the ‘Item Total’ did not include any taxes or shipping.   But even with that consistency, we had an issue.  Tom and I looked at the rule and tried to figure out what the discount percentage should be if a customer who is good had an order total greater than $50.00.  So the three of us made up a table of examples”.  (Footnote – See Brian Marick’s Example Driven Development at http://www.exampler.com)

Debbie continues, “The first two rows show that the limit between giving a Good customer a discount or a 1.0% discount is $10.00.  The ‘less than or equal’ in the business rule is pretty clear.  But we wanted a test to ensure that my implementation produced that result.  We put a ‘??’ after the ‘1.0’ in the third example, since it was unclear to us whether that was the right value.”

“The fourth example indicates that we understand that the discount for an Excellent customer starts at the smallest possible Item Total.  The fifth and sixth entries show that the discount increases after $50.00.”

“Betty took this table back to the stakeholder.  He looked it over and said that the interpretation was correct.  He did not want to give a 5% discount to Good customers.   So we removed the ‘??’ from that result.  We now had a set of tests that we could apply to the system.”

Tom interjects, “But these were not all the tests.  Being trained as a tester, I like to consider other possibilities.  For example, what if the Item Total was less than $0.00?  I asked Betty whether this would ever happen.  She said it might be possible, since the Item Total could include a rebate coupon that was greater than the total of the items.  So I added the following possibilities:”

Tom explains, “It didn’t seem right to apply a discount percentage that would actually increase the amount that was charged to the customer.  Based on this example, Betty went back to the stakeholder and confirmed that the percentage should be 0% if the Item Total is less than 0 for any customer.”

“These examples were the acceptance tests for the system.  If Debbie implemented these correctly, Betty would be satisfied.” Tom continued, “Now it was a matter of how we were going to use these tests to test the system.”

Implementing the Acceptance Tests
Debbie states, “Tom and I needed to apply these tests to my implementation.  There are at least four ways we could do this.  First, Tom could create a test script that operates manually at the user interface level.  Second, I could create a test user interface that allows me or Tom to check the appropriate discount percentages.  Third, I could perform the tests using a unit testing framework.  A standard unit testing framework is generically called XUnit.  Fourth, Tom and I could implement the tests with an acceptance test framework.  Let me show you some examples of how we could have dealt with each of these ways”.

Test script
Debbie continues, “In this case, the program had a user interface that allows a customer to enter an order.  The user interface flow was much like Amazon or other order sites.  The user entered an order and a summary screen appeared, such as this one:”

“Now what Tom would have to do is create a script that either he or me would follow in order to test each of the six examples.  He might start by computing what the actual discount amount should be for each case.  Unless the order summary screen shows this percentage, this is the only output he can check to make sure the calculation is correct.  So here’s the addition to the table:”

“The script would go something like this:

1. Logon as a Customer who has the Rating listed in the table.
2. Start an order and put items in it until the total is the specified amount in the Item Total column on the test.
3. Check that the discount on the Order Summary screen matches Discount Amount in the table.”

“Then the test would be repeated five more times for all six cases.  Either he or I would do this once I’ve completed implementing the discount feature.”

Tom interrupts, “Of course, I’d want Debbie to do this before she turned the program over to me.  Better for her to get carpal tunnel syndrome, then for me.”

Debbie glances over at Tom with a wry smile on her face.  She continues, “Actually, neither of us wants to get carpal tunnel syndrome.  It’s bad for our tennis game.”

“This test that checks for the correct discount amount needs to be run not just once, but for all the possible combinations.  You can imagine what might have happened if there were ten discount percentages for each of ten different customer types.  I’d definitely have let Tom do the tests so his hand would not be in shape for our next tennis match.  So let’s look at the next possible way to run these tests.”

Testing User Interface
“To simplify executing the tests, I could set up a user interface that connected to the discount calculation module in my code.  This interface would only be used during testing.  But having it would cut down on the work involved in showing that the percentage was correctly determined.  The user interface might look like:”

“With this UI, Tom or I could more quickly enter in all the combinations that are shown in the test table.  It would cut down on the possibility of repetitive motion injuries.   Our workman’s compensation insurance likes that idea.”

“We would still need to run Tom’s original script for a couple of instances to make sure that it were properly connected up to the discount percentage module.  But unless there was a large risk factor involved, you might just run it for a few cases such as:

“This UI has penetrated into the system.  It exposes a test point within the system that allows easier testing.  Let me give you an analogy between the difference between this method and Tom’s original test script.  Suppose you want to build a car that accelerates quickly.  You know you need an engine that can increase its speed rapidly.  If you could only check the engine operation as part of the car, you would need to put the engine in the car and then take the car on a test drive.  If you had a test point for the engine speed inside the car, you could check how fast it sped up without driving the car.  You could measure it in the garage.  You’d save a lot of time in on-the-road testing if the engine wasn’t working properly.  That doesn’t mean you don’t need to test the engine on the road.  But if the engine isn’t working by itself, you don’t run the road test until the engine passes its own tests. “

“If you’re not into cars, let me give a context diagram, as below.  The Order Summary Screen connects to the system through the standard user interface layer.  The Discount Percentage Screen connects to some module inside the system to some module.  Let’s call that the Discount Calculator. By having a connection to the inside, a tester can check whether the internal behavior by itself is correct.

XUnit Testing
Debbie explains, “The next way I could perform the testing is to write unit tests for the discount calculator.  These unit tests are usually written in the program language of the application.  Here’s a sample of what these tests look like.  I know you’re not a programmer.  But as you can see, it can be a little harder to determine exactly what is being tested.”

class DiscountCalculatorTest
{
testDiscountPercentageForCustomer()

{
DiscountCalculator dc = new DiscountCalculator()
assertEquals(0.0, dc.computeDiscount(10.0,
         Customer.Good));
assertEquals(1.0, dc.computeDiscount(10.01,
         Customer.Good));
assertEquals(1.0, dc.computeDiscount(50.01,
         Customer.Good));
assertEquals(1.0, dc.computeDiscount(.01,
         Customer.Excellent));
assertEquals(1.0, dc.computeDiscount(50.0,
         Customer.Excellent));
assertEquals(5.0, dc.computeDiscount(50.01,
         Customer.Excellent));
}

}

“Luckily Tom has done some programming, so he can read the code.”  Debbie quickly shoots a smile at Tom.  “However any time we make a change in the tests that Betty and the stakeholder can read, I also have to make a change in these tests.  That’s a little bit of waste.  And to effectively produce software, we’d like to eliminate that waste if possible.  So Betty, Tom, and I selected the next mode of testing.”

Automated Acceptance Testing
“The three of us agreed that the examples in the table accurately reflected the requirements.  So there would be less waste if the table did not have to be converted into another form for testing.  There are several available test frameworks that can use this table or a slightly altered form of this table to directly drive the tests.  I’m not going to get into the details of them at this time.  I think the author is going to have references to them later on in this book.”

“What I would like give is how we use one of these frameworks. The one we use is the Framework for Integrated Test or known as Fit.  It was developed by Ward Cunningham.  (Footnote:  See http://fit.c2.com and R Mugridge & W Cunningham, Fit for Developing Software: Framework for Integrated Tests, Prentice Hall PTR (2005)).  The program is now coordinated by James Shore.). “

“With Fit, you describe the tests with a table similar to the one we used with Betty.  I’m not going to get into the syntactic details.  So as to not confuse you, I’m going to use the same table that we started with.  And besides, we’re not trying to push a particular test framework.  So getting into the minute details is beyond the scope of this book.”

“So here’s the test.  It looks just like the table that Betty presented to the stakeholder.”

“Now when we run this table as a test, Fit executes code that connects up to the Discount Calculator.  It gives the Discount Calculator the values in Item Total and Customer Rating.  The Discount Calculator returns the Discount Percentage.  Fit compares the returned value to the value in the table.  If it agrees, the column shows up in green.  If it does not, it shows up as red. You can’t see the colors in the black and white that this book is printed in.  The first time I ran the test, I got the following table as the output of Fit.”

Tom winks and says, “With this table as the results, it was clear that Debbie had an issue with the Discount Calculator.  That avoided my wasting time telling her something she already knew.  Like how I’m going to beat her in the next tennis tournament.”  Debbie glances back and saw a grin on Tom’s face.  She continues, “Tom didn’t even see this. Like my last serve against him.  I saw that there was mis-coding on my part.  I fixed it.   When all the tests passed, I turned it over to Tom.  Betty saw the passing tests as confirmation that the calculation was working as desired.”

Recap

• Examples of requirements clarify the requirements
• The examples can be used as tests for the implementation of the requirements
• Tests for business rules can be executed in many ways
  • Creation through GUI of transaction that invokes business rule
  • Development of a GUI that invokes the business rule directly
  • A unit test implemented in a language’s unit testing framework
  • A test using an acceptance test framework, such as Fit, that talks to the business rule module

Trackback(0)

TrackBack URI for this entry

Comments (0)

Write comment

Show/hide comment form

Name

Comment

smaller | bigger

Subscribe via email (registered users only)

Write the displayed characters

Add Comment Preview