AI-Generated Code Doubled the PRs. QA Headcount Did Not.

QA engineer absorbing the testing workload from AI-generated code alone at empty desks
In this post
  1. The half of the ledger nobody measured
  2. What 300 testers actually reported
  3. Why does AI-generated code produce these specific defects?
  4. The trust number that should end the argument
  5. How do you turn this into a headcount case?
  6. Where Nopmark fits
  7. FAQ
  8. Ready to test AI-built software properly?

Every serious study of AI-generated code so far has been written from the developer's seat. Stack Overflow, JetBrains, GitHub, LinearB, DORA: all of them measure how much faster code gets written, and none of them measure what happens to that code after it leaves the keyboard. A survey of 300 QA engineers has now filled in the missing half of the ledger, and the short version is that the productivity gain was real and the cost landed somewhere nobody was counting.

The half of the ledger nobody measured

So the industry has spent two years building a very precise picture of one side of a two-sided system. Developers report satisfaction. Merged pull requests go up. LinearB's dataset of 8.1 million pull requests shows teams adopting AI merging 98% more of them than before. Those numbers are not wrong, and the people citing them are not lying.

But a pull request is not a shipped feature. Between the merge and the customer there is a testing function, and until this month nobody had asked that function what the extra volume did to it. DeviQA's State of AI-Generated Code 2026 report, published on 20 July 2026, is the first study of the question built from the tester's seat rather than the developer's: 300 respondents, 40% automation QA, 40% manual QA, 20% SDET, and weighted toward people who have been doing this a while, with 52% carrying five or more years.

That weighting matters, and I will come back to it, because it undercuts the most common explanation offered for why testers are unenthusiastic.

What 300 testers actually reported

Sixty-five percent of them work on teams actively generating code with AI. Another 16% see occasional use. So this is not a survey of holdouts describing a technology they have not touched.

Here is what they said happened next:

  • Bug volume: 52% report it went up. 2% report it went down. That ratio is 26 to 1 against, which is not a mixed result and not a wash.
  • Testing workload: 58% report it increased. 45% report no change. Nobody in the sample reported receiving additional QA headcount to absorb it.
  • Methodology: 74.2% changed how they test in response, mostly by scrutinising edge cases harder, widening regression scope, and re-verifying requirements that the generated code quietly drifted from.
  • Trust: the mean rating for AI-generated code was 3.16 out of 5. 61.3% parked it at exactly 3. Zero respondents out of 300 gave it a 5.

Read the second and fourth bullets together and you have the whole story. The work went up, the confidence did not, and the org chart stayed exactly where it was.

Why does AI-generated code produce these specific defects?

This is the part worth sitting with, because the defect profile is not random. When the 300 were asked what they actually find in the generated output, the ranking came out like this:

  • Logical errors: 58%
  • Unhandled edge cases: 52%
  • Duplication and throwaway code: 42%
  • Non-compliance with requirements: 42%
  • Integration problems: 26%
  • Performance issues: 19%
  • Incorrect or outdated APIs: 13%
  • Security vulnerabilities: 10%

Look at the top four. Every one of them is a defect class that survives compilation, survives a linter, survives a unit test the same model wrote, and survives a code review conducted by someone who is reading for style. Logical errors and requirements drift do not announce themselves. They sit there looking like working code until somebody interrogates the software against what it was actually supposed to do, and that interrogation is a testing activity, not a coding one.

So the cost did not vanish. It moved. It moved from the part of the pipeline that is measured by dashboards into the part of the pipeline that is measured by nothing, and then it accumulated there.

The security number is the one people misread. Only 10% of QA respondents flagged security vulnerabilities as a top defect class, and the temptation is to conclude that what the models produce is basically secure. External research puts mapped vulnerabilities in 40% to 45% of generated samples. The gap between those two numbers is not a contradiction. It is a statement about what QA is positioned to see. Testers are catching what testing catches, and the security defects are going somewhere else, which means an AppSec function needs to be in this conversation and mostly is not.

The trust number that should end the argument

The standard rebuttal to tester scepticism is that it is a familiarity problem. Give it time, the argument goes, and the people who are wary now will come around once they have worked with the tooling long enough to calibrate.

The survey tested that directly, and it does not hold. QA professionals with eight or more years of experience showed no higher trust in the output than people under two years in. This is the opposite of what happens on the developer side, where the productivity gains and the enthusiasm both concentrate among less experienced engineers. On the testing side, experience does not soften the assessment at all, which suggests the caution is a read on the code rather than a gap in the tooling.

And zero out of 300 is a striking number in survey work. Not one person, in a sample large enough to catch every kind of opinion, was willing to say they fully trust it.

How do you turn this into a headcount case?

Most testers lose the budget argument because they bring the wrong evidence. Defect counts do not work. Saying "we found 400 bugs this quarter" invites the reply that the team is expensive, and saying "we found fewer bugs this quarter" invites the reply that the team is unnecessary. Either way you have handed over a number that reads as cost.

The DeviQA figures give you a different shape of argument, because they are about throughput and ratio rather than volume. Four numbers, in this order:

  1. The input went up. 98% more merged pull requests on AI-adopting teams (LinearB, 8.1M PRs). Whatever the testing function was sized for, it was not sized for that.
  2. The defect density went up too, not just the volume. 52% report more bugs against 2% reporting fewer. So this is not the same quality of work arriving faster.
  3. The defects are the expensive kind. Logical errors and requirements non-compliance, the two categories that need a human to compare behaviour against intent, sit at 58% and 42%. These do not get caught by adding another gate to CI.
  4. The absorbing capacity did not change. No respondent reported additional QA headcount.

That is a capacity argument, and capacity arguments get funded. Notice also what the fourth number does that the first three cannot: it converts a quality complaint into an arithmetic one, and arithmetic is the language the person holding the budget already speaks.

One caution before you paste these into a slide. This is a single vendor-published survey of 300 people, and it is self-reported. Cite it as what it is. The finding that senior testers are no more trusting than junior ones is the most load-bearing claim in the whole report, and it is also the one most worth checking against your own team before you present it as settled.

Where Nopmark fits

If your team is reviewing AI-generated code all day, the skill that pays is not prompt writing. It is test analysis: reading a requirement, working out what the code would have to do wrong to satisfy it superficially, and designing for that. The ISTQB Certified Tester AI Testing syllabus covers the specific failure modes of AI-produced and AI-assisted software, including how to test systems whose behaviour is not fully deterministic. For testers whose test design is still self-taught, the Foundation Level material underneath it is where equivalence partitioning, boundary analysis, and decision tables stop being exam vocabulary and start being the thing that catches the 58%.

FAQ

Does AI-generated code actually have more bugs, or do testers just look harder at it? Both are happening, and the survey captures them separately. 52% report an increase in bug volume, and 74.2% report changing their testing approach. The methodology change is real, but a stricter process would raise detection, not the underlying defect rate that 52% are describing.

Why are logical errors the top defect category? Because a logical error compiles. Syntax problems and bad API calls fail loudly and get fixed before a tester ever sees them, so what reaches QA is the class of defect that looks correct and behaves wrongly.

Should QA be catching the security vulnerabilities too? Only 10% of respondents named security as a top defect class while external analysis finds vulnerabilities in 40% to 45% of AI-generated samples. That gap says the work belongs with an application security function, not that the vulnerabilities are absent.

Is this survey representative? It is 300 self-selected QA practitioners, published by a QA services company, and it is weighted toward senior staff, with 52% at five or more years. Treat it as the strongest available datapoint on the QA side, not as a census.

What is the single number worth quoting? Zero out of 300 gave it full trust, and the eight-plus-year group rated it no higher than the under-two-year group.

Ready to test AI-built software properly?

The ISTQB Certified Tester AI Testing course from Nopmark covers exactly this territory, taught by accredited instructors, with hands-on work on the defect classes that generated code actually produces.

Keep reading