Risk-based testing: check your scores against escaped defects
Take a QA lead whose team uses risk-based testing to plan each sprint. Every story gets a risk score, the team tests the high scorers first, and the low scorers…
You have read this

Two months ago I started using Claude Code to write tests every day. Not as a demo. As part of the job, on a real suite, with real deadlines.
The headline: it made me faster, but not in the way the marketing promised. The work didn't disappear. It moved. I spend less time typing tests and more time reviewing the ones the agent writes. That trade is worth it, most days. Some days it isn't. This post is the honest version of how it actually works, so you can decide where it fits in your own week.
The speed-up is real, and it is uneven.
What got faster:
What did not get faster:
So the net is positive, but the gain is concentrated in the mechanical parts of testing, not the thinking parts. If your week is mostly mechanical, you will feel a big lift. If it is mostly judgment, you will feel a smaller one.
The difference between a tester who gets value from an agent and one who gets frustrated is almost always the prompt. A vague prompt gets you fifty shallow assertions that all check the same happy path. Here is the loop I settled on.
Bad prompt: "Write tests for the login page."
You will get a generic happy-path test and three throwaways.
Better prompt: "Write Playwright tests for the login form. Valid login redirects to /dashboard. Invalid password shows the error 'Incorrect credentials' and stays on /login. The account locks after 3 failed attempts and shows a lockout message. Email field rejects input without an @ before the request fires."
Notice what changed. You handed it the behavior, the expected messages, and the edge rules. The agent is good at translating a clear spec into code. It is bad at inventing the spec. So you keep the spec.
Tell it which framework, which patterns, and what to reuse. "Use our existing LoginPage fixture. Follow the structure in tests/auth/. No hard waits, use web-first assertions." Without this, the agent invents its own conventions and you spend the saved time reconciling them.
Do not ask for the whole suite at once. Ask for one scenario. Read it. If it understood the intent, ask for the next three. If it did not, fix the prompt before you scale the mistake across forty tests. The agents that feel magic in a demo feel expensive when you let them run unsupervised.
A useful habit: ask the agent why it asserted what it asserted. "Why did you check the URL instead of the page heading?" Half the time the answer is reasonable. The other half it reveals that the agent guessed, and now you know which lines to distrust.
An agent that writes tests is a junior engineer who types very fast and never gets tired. You would not merge a junior's first draft without reading it. Same rule here.
Review every assertion, not every line. You do not need to scrutinize the syntax. The agent rarely gets syntax wrong. You need to scrutinize the assertions, because that is where it quietly tests the wrong thing. A test that passes while checking nothing useful is worse than no test. It gives you false confidence and a green pipeline.
Never let it touch the things that define correctness. I do not let the agent write the assertions for our payment flow or our data-integrity checks without line-by-line review. The cost of a wrong test there is a real bug shipped behind a green check. For low-risk surface area, I am more relaxed. Match your scrutiny to the blast radius.
Watch for the confident-but-wrong test. The most dangerous output is not the test that fails to compile. It is the test that runs, passes, and asserts something subtly incorrect. The agent will assert that a toast message appears without checking the text inside it. It will check that a list has items without checking they are the right items. Read for what the test does not check.
Keep a human-readable diff between you and main. Everything the agent produces goes through normal review, same as any pull request. No agent output goes straight to the protected branch. This sounds obvious. Under deadline pressure it is the first discipline people drop, and it is the one that bites.
After two months, the failure modes are predictable, and they map almost exactly onto the parts of testing that were always the skilled part.
Read that list again. Every item on it is judgment, product knowledge, and exploratory instinct. Those are the skills worth investing in, because they are exactly what the agent offloads back onto you. The mechanical work is shrinking. The thinking work is not.
Using Claude Code to write tests changes which skills pay off, and it raises a new question: how do you test systems that include AI, and how do you validate AI-generated work without being fooled by a green pipeline? That is the gap the ISTQB CT-GenAI certification is built for. It covers using generative AI in testing and testing AI-based systems, taught by accredited instructors, so the judgment you keep applying has a real framework behind it rather than guesswork.
Is Claude Code better than Copilot or Cursor for writing tests? They are closer than the debates suggest. The bigger variable is your prompt and your review discipline, not the tool. Pick the one that fits your IDE and editor workflow, then invest your energy in giving it clear specs and reviewing its assertions.
Will AI test generation replace QA engineers? It replaces the mechanical part of writing tests, not the judgment part. Knowing what to test, prioritizing by risk, and finding the gaps nobody scripted are still human work. The role shifts toward review and strategy, it does not vanish.
Can I trust AI-generated tests in CI? Only after a human reviews the assertions. The dangerous failure is a test that passes while checking the wrong thing. Treat agent output like a junior engineer's first draft: useful, fast, and not mergeable unread.
What is the single biggest mistake people make with AI test generation? Asking for the whole suite at once and trusting it. Generate one scenario, confirm the agent understood the intent, then expand. Scaling an unreviewed prompt is how you end up with forty confident, useless tests.
How much time does it actually save? For boilerplate, test data, and starting drafts, a lot. For deciding what matters and reviewing output, none, because that work moved to you. The net is a real gain if your week is heavy on mechanical test writing.
Ready to test AI systems and use generative AI in your own testing with a real framework instead of trial and error? The ISTQB CT-GenAI certification from Nopmark covers exactly this, taught by accredited instructors with hands-on practice. See the CT-GenAI program.
Take a QA lead whose team uses risk-based testing to plan each sprint. Every story gets a risk score, the team tests the high scorers first, and the low scorers…
You have read this