Testing Agent
From user story to test report: it prepares the tests, runs its own and guides you through yours.
For any product, with a part dedicated to generative AI. Every decision stays with a person.
Installs in your coding agent: GitHub Copilot, Claude Code, Codex or Cursor.
- Free during the test phase
- Windows, Mac
- English, French
- With or without Azure DevOps
What the video shows, shot by shot
- From user story to test report: it prepares the tests, runs its own and guides you through yours. Installs in GitHub Copilot, Claude Code, Codex or Cursor.
- For the whole product team: product owner (prepares, decides), developer (code tests), tester (screen tests), quality lead (coverage, report).
- Four kinds of tests, one single tracker: components and integration, automated; system and screens, guided step by step; AI evaluation, measured.
- The example: an HR assistant that answers employees and opens their leave requests.
- You give the user story in your coding agent; it splits the acceptance criteria, AC1 to AC3, and asks you to confirm.
- 1. It writes the test cases; you approve them in one go: "Cases approved: 5."
- 2. It writes and runs the code tests: two tests passed, saved with their proof.
- 3. For screens, it guides you step by step: the value to type is ready to copy, you act in your product. The employee writes: "I'm not requesting anything, just asking."
- A request opens in their name all the same: the tester marks "Not as expected" and describes what they see.
- 4. It evaluates the AI on 20 reference questions: recall of expected facts 0.967, above the 0.85 threshold, but two actions failed, including a request opened without being asked. Verdict: Fail.
- The bug is found before production: the agent says what is left to decide, and you decide.
- Everything exports to Excel, one sheet per topic: summary, cases, results, steps, defects, actions, requirements.
- Install it for free: moamen.fr
The journey, from user story to test report
Seven steps. Pick one: on the right, what the Testing Agent does, what you do, and what comes out.
Product ownerPreparation
It reads the user story and splits it into acceptance criteria. You confirm the split.
The Testing Agent
- Reads the user story
- Splits the criteria
- Spots what is implicit
You
- Confirm the split
What comes out
Numbered criteria
Product ownerPreparation
It writes the cases step by step, each at the right level and linked to its criterion. You approve them.
The Testing Agent
- Picks the level
- Writes the steps
- Links each criterion
You
- Approve the cases
What comes out
Approved cases
DeveloperAutomated: component, integration
It writes and runs the tests in your project, then keeps the proof of each result.
The Testing Agent
- Writes the tests
- Runs them
- Keeps the proof
You
- Review the test code
What comes out
A verdict per case
TesterManual, guided: system
It guides you step by step in your product. You say what you see, it records it.
The Testing Agent
- Prepares the test sheet
- Guides step by step
- Drafts the defect
You
- Test in the product
- Say what you see
What comes out
A verdict in your name
Product teamMeasured: AI evaluation
It measures the AI answers on reference questions, against the thresholds you set.
The Testing Agent
- Builds the questions
- Runs the evaluation
- Gives the verdict
You
- Set the thresholds
What comes out
Pass or Fail, with proof
Product ownerDecision
It says what is left before closing. The decision, Go or No Go, stays yours.
The Testing Agent
- Lists what is left
- Refuses to close without proof
You
- Decide Go or No Go
What comes out
A decision in your name
Quality leadClosure
It measures progress and exports the whole tracking to Excel, ready to share.
The Testing Agent
- Measures progress
- Exports to Excel
You
- Share the report
What comes out
A seven-sheet workbook
The interfaces
They come in the zip and open in your browser, with nothing more to install. Their content comes from what the agent writes from your user stories: the same screens, filled with your project.
The screens, in the order of the journey
Evaluation report
examples/04-hybrid-agent
Verdict Fail
- Definition of Done
- Fail · 2 assertion failures
- Definition of Trustworthy
- Pass · thresholds met, no veto
| Check | Measured | Required |
|---|---|---|
| Facts recall | 0.967 | 0.85 |
| Slice ticket-action (6 cases) | 1.000 | 0.85 |
| Veto: unsupported claim | none | none |
Assertion failures
H-13· tool called when none was expected: open_hr_ticketH-19· required output field missing or empty: ticket_reference
Test phase: 5 free seats
The Testing Agent is in its test phase. The first 5 people who create their space and take a seat use it for free, for the whole test phase.
- The 5 seats go to the first 5 people, with no selection.
- Each of them can invite 5 people of their choice.
- Once the seats are taken, you can ask for access in writing: I read every request myself.
Free and secure. No newsletter without your agreement. Delete your account in one click.
Already signed up? The same button signs you in.
Free for the whole test phase. If it ever becomes paid: 30 days of notice, nobody pays without having agreed to it, and your data stays yours.
Why it exists
On an AI product, I had to test two things at once: code that has an exact result (a link expires after thirty minutes) and generated answers, which have none. Classic test tools only handle the first kind, and acceptance testing fell almost entirely on me.
My method covered both, but it lived in my head and on my machine. I built the Testing Agent in my spare time so that a whole team could apply it in its own assistant, from the first test case to the final report.
In detail: technical notice, chapter 1, The problem (PDF)
Method
A written method that tells what is tested from what is measured.
- Definition of Done
- What is tested, against an exact result.
- Definition of Trustworthy
- What is measured, on a reference set, against a threshold set by a person. Without an approved threshold, the verdict is "not ready".
- Three behaviours
- Deterministic, probabilistic, hybrid.
Based on the author's practice and on the classic test levels. It does not claim any standard.
Read the article: where the Definition of Trustworthy comes from, and how it works
In detail: technical notice, chapter 2, The method (PDF)
Learn the method
The method the Testing Agent applies is taught in a free online course, with no account and nothing to install. Part 1 covers designing and running tests, part 2 evaluating generative AI.
Follow the learning pathQuestions
Does my data leave my machine?
No. The Testing Agent opens no connection to the outside and sends nothing to its author. For updates, your assistant only reads a public file on moamen.fr, every 7 days. Your AI assistant and Azure DevOps, if you use it, remain subject to their own terms.
Which AI model do I need?
Your assistant's. The Testing Agent brings the method, the tools and the rules; your assistant brings the reasoning.
How do I update it?
Nothing to do: every 7 days, your assistant looks on moamen.fr for a new version and tells you only if there is one. It says what it changes and installs it only with your agreement. Your tests, evidence and profile are kept, with a backup copy, and the previous version stays until you confirm everything works. To check right away: "Using the Testing Agent, check for updates".
And without Azure DevOps?
You give it your user stories as text. Everything else works locally: cases by phase, runs, tracking, dashboard.
The download is blocked on my work computer. What should I do?
Some company networks block sites they don't know yet. Your assistant tells you and stops. Ask your IT team to allow moamen.fr, then paste the same sentence again. Don't use another network or device to get around the filter.
Is it safe to let my assistant install it?
It first checks that the file is the one published here, using its checksum, then announces every action and asks for your agreement. The instructions it follows are in the archive, readable by anyone.
Can it validate a test for me?
No. It prepares the evidence and tells you where to set the result. Setting it stays your act; it then records it in your name.
Why a flat white?
It's the only deliverable of my day that passes all its tests on the first run.
Technical details
For IT and for anyone who wants to check everything: supported tools, rules, security, what is in the zip, the fact sheet and the checksum.
You will need
- A coding agent, that is an AI assistant in agent mode, among: GitHub Copilot in VS Code, Claude Code, Codex, Cursor.
- Python 3.10 or later. If it is missing, your assistant offers to install it, with your agreement.
- An internet connection for the installation: two Python dependencies (PyYAML and jsonschema) are downloaded. Afterwards, the Testing Agent uses none.
Everything it does
From acceptance criteria to test cases
- Reads the whole user story in Azure DevOps: description, acceptance criteria, parent Feature and Epic, comments.
- Classifies each acceptance criterion: what can be checked exactly, what has to be measured.
- Spots a criterion changed in the story and flags the cases to review and the results to rerun.
- Picks the level of each test: component, integration or system.
- Writes the cases step by step, with their expected results, which you approve one by one or all at once.
- Sets the entry and exit criteria of each level.
- Writes and runs the code tests, and reads your JUnit reports and Azure DevOps builds.
- Links each requirement to its cases, and records what stays out of scope, with the reason.
From the first run to the test report
- Walks a screen test step by step, with the values to copy, and keeps the screenshot as evidence. A person still does the test.
- Records every result with its author's name.
- Follows defects until they close, which takes a new passing test.
- Assigns actions to named people, and says what waits on whom, and since when.
- Measures progress by phase, by user story and by importance, in tables you can sort and filter, and exports the whole record to Excel.
- With Azure DevOps: publishes the plan with one suite per story, tracks each acceptance criterion, reads results back from Test Plans and pipelines, prepares the evidence note, the bug draft and the closure file.
Decide whether an AI is reliable enough, with evidence
- Checks that everything is ready to evaluate, and lists the missing decisions with their owner.
- Links business risks to the slices to watch, such as regulated cases or a second language.
- Writes the scenarios and the manifest: thresholds, slice minimums, vetoes.
- Builds the reference dataset, with the data protection rules.
- Chooses the evaluators, and calibrates an AI judge against human annotations before relying on it.
- Runs the evaluation and returns a verdict per gate, with the evidence of each case. It also audits an evaluation that already exists.
With your tools
The Testing Agent calls no tool itself. It goes through the tool's MCP server, which your assistant already uses, with your own permissions.
- Azure DevOps and Azure Test PlansSupported
- Through the official Azure DevOps MCP server. The Testing Agent prepares or records; your assistant writes, after your agreement. A Basic + Test Plans licence is needed to create plans.
- No trackerSupported
- The tracking lives in your project, as text files, with its interface. Several people can record at the same time, each in their own copy: the Testing Agent settles the merge.
- Your automated testsSupported
- JUnit XML reports, which most continuous integration tools can produce, and evaluation reports in JSON.
- Jira, Monday and othersNot yet
- Not supported today.
What it refuses, what it recommends
The register refuses
Enforced by the code: a write that breaks the rule is rejected.
- Closing a defect without a passing retest.
- Approving a case whose expected result is still "to confirm".
- Assigning an action to the agent rather than to a named person.
- Taking a Go / No Go on a campaign that is not finished.
- Recording a write without an author.
The instructions given to your assistant
Followed by the assistant, not enforced by the code.
- Do not invent a threshold or an importance.
- Set a result in Azure Test Plans only with your agreement, in your name.
- Create a bug in Azure DevOps only with your agreement.
- Mark a manual test as passed only if a person says they ran it.
An assistant can depart from an instruction. The register keeps a record of who decided what, with the state before and after.
In detail: technical notice, chapter 5, Rules and human decisions (PDF)
Security, and what it writes on your machine
At run time, the Testing Agent opens no connection to the outside: it talks to your assistant through standard input and output, and its dashboard only listens on your own machine. At installation, two Python dependencies (PyYAML and jsonschema) are downloaded from PyPI. Every 7 days, your assistant reads a small public file on moamen.fr to know whether a new version exists; nothing is sent.
- %LOCALAPPDATA%\TestingAgent
- A copy of the Testing Agent (elsewhere: ~/.testing-agent/app).
- testing/ in your project
- The tracking, as versionable text files, with the log of every write and the screenshots attached to results.
- Your assistant's configuration
- A testing-agent entry, for example in .vscode/mcp.json.
- ~/.testing-agent
- Your profile: name and role.
- Keys and tokens
- The Testing Agent asks for none and stores none. Azure DevOps is called by your assistant's Azure DevOps MCP server, with its own permissions.
To uninstall, delete these items. The details are in the installation guide in the archive.
In detail: technical notice, chapter 4, Data and files (PDF)
Good to know
- Designed for trials and team use. Have your IT department approve it before a deployment.
- It does not check a person's identity: the record relies on what they declare.
- Behaviour depends on the AI used: the same instruction can be followed differently from one assistant to another.
- Other MCP clients can be configured by hand; they have not been verified.
- The files are not signed. The archive holds no launcher script: installation goes through your assistant.
- The source code is not public. Free during the test phase, no resale or redistribution.
- A single author, with no guaranteed support.
In detail: technical notice, chapter 8, Limits and recommended use (PDF)
What's in the zip
Eight skills, which are written method instructions, and a small local program that acts, keeps the state in your project and refuses some writes, so that decisions stay with people.
- 8 skills
- The method instructions the assistant follows: classify, design, evaluate, track.
- 28 tools (an MCP server)
- A small program the assistant calls to act: read and write the tracking, run evaluations, prepare the evidence and the bug draft.
- A tracking register
- In your project: every decision is signed, a journal keeps the before and after, and some writes are refused.
- Interfaces
- A dashboard and a local web interface: what waits on each person, a test step by step with values to copy and screenshots attached, what the agent did and why, and an export to Excel. People see and decide for themselves.
- Scripts
- The tracking and evaluation scripts, and the guided installation your assistant runs.
Also included
- The user guide, by role, in English and French
- Evaluation examples
- The terms of use, the installation guide, the version history
Details and notice
Free during the test phase; if it becomes paid, 30 days of notice, and nobody pays without having agreed to it. Usable alone or as a team, for personal or professional projects. No resale or redistribution, and no reuse of the method for another product or to train an AI: to share it, share this page. Provided as is, without warranty. Your data stays on your machine. By installing it, you accept these terms (full text in CONDITIONS.md, in the archive).
Details
- Version
- 0.8.3
- Released
- 28 October 2026
- Size
- 416 KB
- Author
- Moamen Elmasry
- Technology
- Python, MCP, text files
- Languages
- English, French
- Format
- .zip archive
For IT
The assistant checks this checksum on its own during installation. It is for anyone who wants to check the archive or approve it on a company computer. It is also published in testing-agent-0.8.3.zip.sha256.
SHA-256 checksum
b6f5bd8c15dbef045a67f57a0208166650253afc0b03ad19760298f7c6fbe0abTechnical notice
For anyone who wants to go further or has to approve the tool: the problem, the method, the architecture, the data, the rules, Azure DevOps, installation, limits, version history.
Open the notice (PDF) (13 pages, 389 KB)Free. If it saves you time: buy a flat white. The donation link is coming soon.
DON-001
Given the Testing Agent saved you time,
when you buy a flat white,
then the developer stays caffeinated.
Minimum: €5. Maximum: no assertion, probabilistic outcome.
A question, some feedback?
Message me on LinkedIn.
Message me on LinkedIn