Module 02 · Running a campaign · 12 min
From a failure to a defect
Qualify a failure before anyone opens a ticket, write a defect a developer can reproduce, and close it only on a new passing test.
Review
From the previous lesson: a test is skipped with the reason "flaky, to look at later". What does the report prove about the behaviour it covers?
The course's answer
Nothing. A skipped test neither passed nor failed: it is a gap to report, and "flaky" is not a reason to skip it. A flaky test counts as failing until it is fixed.
A failure is not yet a defect
A failed run says the product and the expected result disagree. It does not yet say which one is wrong. Before anyone writes a defect, the failure is qualified:
| The problem is in | The evidence | What follows |
|---|---|---|
| The product | The expected result comes from a specification someone owns, and the product differs from it | A defect, assigned to the team that fixes the product |
| The test | The expected result was never specified, or is out of date, or the test checks a detail nobody required | The test is fixed, and the reason recorded |
| The environment | Connection refused, missing data, a service not started: the test never reached the product | Fix the environment, rerun, keep both runs |
There is a fourth case, rarer and important: the product does exactly what was specified, and the specification missed something. That is not a defect in the code. It is a gap, and it goes back to the product owner as a missing requirement, not to a developer as a bug.
A defect someone can reproduce
A defect report is written for a developer who was not there. They must be able to see the problem without asking you anything. It has:
- a title that says what happens, not what you feel: "Critical travel claim routed to queue-travel", not "Routing broken";
- the steps, with the exact data: which claim, which product line, which urgency, which build, which environment;
- the expected result, and its source: the routing table, version, section;
- the observed result, exactly as it appeared, with the screenshot or the log line;
- the severity: how bad it is for the people who use the product. Priority, how soon it must be fixed, is a separate field, and usually the product owner's call.
It does not contain the fix. Guessing at the cause in the report sends the developer in a direction before they have looked.
Closing it
A defect is closed on a new run that passes, on a build that contains the fix. Not on "fixed, trust me", and not on a test that was changed to pass. That run becomes a regression case, kept in the suite for good, so that the same defect cannot come back unnoticed.
Write one
In CAMP-03, case CT-02 failed: claim 7731, travel product line, urgency critical, build 2.3.0-142, test environment. The card says queue-travel. Write the defect.
The course's answer
Title. Critical travel claim routed to queue-travel instead of queue-A
Steps. On the test environment, build 2.3.0-142: as an agent entitled to the travel line, open claim 7731 (travel, urgency
critical) and read the queue on the triage card.Expected.
queue-A. Source: routing table, ops runbook, "critical goes to queue-A whatever the product line".Observed.
queue-travel. Screenshot of the card attached. The same happens for critical liability claims (R-07 in the routing suite).Severity. Critical: an emergency claim waits in an ordinary queue. Priority: to set by the product owner.
| Your defect | ||
|---|---|---|
| A | Exact data and build, expected result with its source, observed result as seen, severity separate from priority, no guessed fix | ✅ |
| B | All of it, plus "probably the if on product line in the routing function" |
⚠️ Remove the guess. If you know the cause, say it to the developer, not in the record |
| C | "Routing is wrong for travel claims, please fix" | ❌ No data, no source: the developer starts by asking you everything you knew |
Once this defect is fixed, which case proves it, and what happens to that case afterwards? CT-02, rerun on the fixed build: a pass closes the defect, and CT-02 stays in the suite as a regression case.