Module 02 · Running a campaign · 15 min
Exercise: the status, and the report
CAMP-03 is in its last day. Compute its figures, write the status, then tell the product owner about the exit criterion it does not meet.
Review
From the previous lesson: why does a settled case count against quality?
The course's answer
Because it ran and failed: someone accepted the deviation, but the product still did not do what was required. Counting it as a success would turn a decision about a failure into a pass.
CAMP-03, on its last day
The frame is the one from lesson 1: requirements EX-01 to EX-07, EX-07 not testable, ten planned cases on build 2.3.0-142. Here is where the runs stand:
| Case | Requirement | Verdict | Note |
|---|---|---|---|
| CT-01 | EX-01 routing | Passed | |
| CT-02 | EX-01 routing | Failed | Defect DEF-12, critical: critical travel claims go to queue-travel |
| CT-03 | EX-02 entitlement | Blocked | No account without the motor entitlement in the test environment; Sara asked IT on Monday |
| CT-04 | EX-02 entitlement | Not run | |
| CT-05 | EX-03 card fields | Passed | |
| CT-06 | EX-03 card fields | Failed | Defect DEF-13, major: summary missing when the description has 2,000 characters |
| CT-07 | EX-04 description length | Passed | |
| CT-08 | EX-04 description length | Not run | |
| CT-09 | EX-05 card never lost | Settled | Resent after 5 minutes instead of 1; accepted in writing by the product owner for this release |
| CT-10 | EX-05 card never lost | Passed |
EX-06, the audit log, has no case: nobody wrote one.
The exit criterion for system testing, approved at the start: every testable requirement has a passing test, and no open defect of critical severity.
1. The figures
2. The status
The product owner asks: "where are we?". Write the status, in a few lines.
The course's answer
Not done. EX-02 (entitlement, very high importance) has not been tested at all: CT-03 is blocked on a test account Sara asked IT for on Monday, and CT-04 has not run. CT-08 has not run either. EX-06 (audit log) has no case. EX-07 (response time under load) cannot be tested here.
Open defects. DEF-12, critical: critical travel claims go to
queue-travel. DEF-13, major: the summary is missing at 2,000 characters.Accepted deviation. CT-09: cards are resent after 5 minutes instead of 1, accepted by you for this release.
Figures. Progress 70%, quality 57% (4 passed out of 7 executed), execution coverage 67%, preparation coverage 83%.
| Your status | ||
|---|---|---|
| A | Starts with what is not done, entitlement first; names both open defects with their severity; the settled case as a deviation; figures last, side by side | ✅ |
| B | All the facts, but it opens on "70% progress, 57% quality" | ⚠️ The reader stops at the first line. Put what they must act on there |
| C | "Good progress, 4 of 7 passing, two bugs being fixed" | ❌ Entitlement, the most important requirement, is not mentioned, and it was never tested |
3. The exit criterion
The release meeting is tomorrow. The exit criterion is not met. Write the message to the product owner.
The course's answer
The system exit criterion is not met, for two reasons:
- EX-02 has no passing test. It has not been tested at all: CT-03 is blocked on a test account without the motor entitlement, requested from IT on Monday. Until it runs, nobody knows whether an agent can open claims they are not entitled to.
- DEF-12 is open, at critical severity: critical travel claims are routed to an ordinary queue.
Also outside the criterion, and to keep in view: EX-06 has no case, and EX-07 cannot be tested here.
The decision is yours: release with these gaps accepted and recorded, or wait for the account and the fix. If you need the account sooner, IT is the one to ask.
| Your message | ||
|---|---|---|
| A | Says the criterion is not met, gives each reason with its evidence, and leaves the decision explicitly to the product owner | ✅ |
| B | All of it, plus "I recommend a No Go" | ⚠️ You may be asked for your view, and you can give it. Keep it apart from the facts, and do not let it read as the decision |
| C | "Testing is 70% complete, the remaining risk is low" | ❌ An untested entitlement control is not low risk: nobody knows. And "low" is a judgement you were not asked to make for them |
| D | Lowers the criterion to "no open blocking defect except DEF-12" | ❌ Changing the criterion to meet it is the one thing the person running the tests may never do |
4. Your own case
Your own case · Part 1, module 02
The frame of your next campaign
Frame the next test campaign of a product you work on. What is in scope and what is out, with the reason. The requirements that cannot be tested in your environment, and why. The exit criteria, and who decides when one is not met. Do not invent an exit value nobody approved: leave it to confirm and name who confirms it.
Saved in this browser. It goes into your record.
That is part 1. You can design the tests of a behaviour that has one right answer, run a campaign, and say where it stands without claiming more than was tested. Part 2 takes the behaviours no assertion can test, like ClaimTriage's summary, and the same discipline: evidence first, and decisions left to the people who own them.