Agile Testing Inside the Sprint: One User Story From Card to Done

Agile testing in the sprint: a developer, tester and product owner map examples for one user story
In this post
  1. What does a tester do on an agile team?
  2. Three amigos and example mapping on one story
  3. Writing the acceptance criteria
  4. Agile testing and the test pyramid
  5. When is the story done, and when is testing done?
  6. Sources

Take a team that runs two-week sprints and says it does agile testing. A developer finishes a user story on day 8 of the sprint. The tester picks it up on day 9, finds a gap in the rules, and the story rolls into the next sprint. The tester saw the story after it was too late to change it cheaply.

This is the story the team was working on.

"As a library member, I want to renew a borrowed book online, so that I do not pay a late fee".

The story looks simple, but on day 9 the tester asks what should happen when another member has placed a hold on the book. The developer had allowed the renewal. The product owner, who speaks for the library staff, expected it to be refused. The team lost a sprint over that one question. It could have been asked in a short meeting before anyone wrote code.

What does a tester do on an agile team?

The ISTQB Certified Tester Foundation Level syllabus, v4.0.1 (ISTQB, September 2024) describes a "whole team approach" in section 1.5.2. Any team member with the right skills can do any task, and everyone is responsible for quality. In practice, testers help business people write useful acceptance tests. They also agree on the test strategy and the automation approach with developers.

The syllabus also expects testers to take part in planning (section 5.1.2). In release planning, testers help write testable user stories and acceptance criteria. In iteration planning, they judge how testable each story is, break it into tasks, and estimate the test effort. Most of the tester's best work on the renewal story should have happened before day 1 of the sprint.

The syllabus also says the whole team approach may not fit every context. Safety-critical work may need a high level of test independence, which means testing by people separate from the authors of the code. Credit the approach, then tailor it to the risk of your product. If you are the only tester on your team, the same planning habits apply, and our post on being the only QA on the team covers that situation.

Three amigos and example mapping on one story

Go back to the renewal story, but this time start before the sprint. A developer, a tester and the product owner sit down together for 25 minutes. The Agile Alliance glossary calls this the three amigos. The product owner asks what problem the team is trying to solve. The developer asks how they might build a solution. The tester asks "What about this, what could possibly happen?" of each rule in the story.

A simple way to run that talk is example mapping, which Matt Wynne of Cucumber described on December 8, 2015. The story goes on a yellow card. Each rule goes on a blue card, and concrete examples of that rule go on green cards beneath it. Questions that nobody in the room can answer go on red cards. Wynne says a well-understood, well-sized story should take about 25 minutes to map. The ISTQB syllabus names example mapping too, as one of the sessions where testers explore, challenge and help improve user stories (section 3.1).

For the renewal story, the map might look like this.

  • Rule: a member can renew a book at most 2 times. Priya has renewed once, so her renewal works. Sam has renewed twice, so the system refuses and shows his due date.
  • Rule: a book with a hold cannot be renewed. Another member placed a hold this morning, so the renewal is refused.
  • Rule: an overdue book can be renewed if nobody has placed a hold. The book is 2 days late, and the late fee already charged stays on the account.
  • Red card: One member asks for a hold at the same moment another member renews online. Which request counts first? Nobody in the room knows. The product owner takes it away to ask the library staff.

The red card records a defect in the story, found before the code exists. ISTQB treats this kind of review as static testing, which means finding problems without running the software. When the team catches a wrong rule here, fixing it takes a conversation. On day 9, the same rule took a whole sprint to fix.

Writing the acceptance criteria

The syllabus describes three parts of a user story, called the "3 C's" (section 4.5.1). The Card is where the story is written, and the Conversation explains how the software will be used. The Confirmation is the acceptance criteria. These are the conditions the story must meet before stakeholders accept it. The syllabus says they can be seen as the test conditions that the tests should exercise (section 4.5.2).

It names two common formats for them. Rule-oriented criteria are a checklist or a table of inputs and outputs, and the blue cards above already fit that format. Scenario-oriented criteria use the Given/When/Then format from behavior-driven development (BDD). Here is the hold rule written as a scenario:

Given a member has borrowed "The Hobbit"
And another member has placed a hold on it
When the first member tries to renew it online
Then the renewal is refused
And the member sees the current due date

The syllabus says acceptance criteria should describe both positive and negative scenarios. A team that writes only the happy path tends to ship the hold defect from the opening. In acceptance test-driven development (ATDD), the team turns these criteria into test cases before the story is built (section 4.5.3). Positive cases come first and negative cases follow. Then the team covers non-functional checks such as performance or usability.

This is also the point where a tester can write the automated check straight from the criteria. The usual alternative is to test by hand first and automate "later". That route tends to fail, because each sprint adds more manual reruns and leaves less time to automate anything.

Agile testing and the test pyramid

Not every example on the map needs a browser test. The test pyramid in section 5.1.6 of the syllabus shows that tests differ in size and speed. Tests at the bottom are small, isolated and fast, so a team needs many of them. Tests at the top are end-to-end checks that run slower, so a team needs only a few. The original model by Mike Cohn used three layers, called unit, service and UI tests.

Here is how the renewal examples might map onto it.

  • Because the renewal limit is pure logic, unit tests can check 0, 1, 2 and 3 previous renewals. The boundary values are 1 and 2. After 1 renewal another is allowed, and after 2 it is refused. Their neighbours, 0 and 3, should allow and refuse it in the same way.
  • Because the hold rule depends on stored hold records, it fits a service or API test that calls the renewal function with a hold in place.
  • One end-to-end UI test shows that a member can log in, renew a book and see the new due date. A pass shows that this one path works across the parts. It does not prove other paths work, so add more end-to-end tests only where the links between parts carry risk.

The tester does not have to write all of these tests. In a whole team, a developer may write the unit tests while the tester chooses the boundary values they cover. What the tester checks is whether the full set of tests, across every layer, covers every rule on the map.

When is the story done, and when is testing done?

The story is done when it meets every item in the team's Definition of Done. Passing acceptance tests are only one of those items. A done story does not always mean testing is done. An end-to-end workflow may cross several stories and several sprints. A load test or a security test may need days of setup and may not fit inside any single story. Those tests belong in the test plan, which sets its own objectives for when testing is complete. A check that the Definition of Done requires cannot move there just because it takes longer.

Nopmark's ISTQB CTFL training teaches the whole team approach, acceptance criteria, ATDD and the test pyramid as the syllabus defines them.

When the tester joins refinement, the hold question gets asked on a red card instead of on day 9. The rule then becomes a scenario, the scenario becomes an automated check at the right layer, and the team can finish the story in the sprint where it was planned.

Sources

Keep reading