Episode Details

AI coding agents lifted pull requests per author 20 percent and incidents per pull request 23.5 percent in a 2025 benchmark, compounding to about 48 percent more incidents per engineer.

That 48 percent is my arithmetic: 1.20 times 1.235. The benchmark itself, published November 12, 2025 from a survey of more than 50 engineering leaders, reported the two inputs and a change failure rate up about 30 percent. A startup that adopts coding agents without changing how it verifies and releases code should expect that curve. A startup that changes four practices has a defensible chance of bending it.

Why does more generated code produce more incidents?

Generation got cheap and verification did not. A coding agent opens a pull request in minutes, but a human still reads it at human speed. When the queue grows, reviewers skim, and fluent code slides through. A sample of 470 open source pull requests found 10.83 issues per AI coauthored change against 6.45 for human changes, with logic errors 75 percent more common and security issues up to 2.74 times higher. That sample came from a vendor of an AI review product, so weigh it accordingly.

Security data points in the same direction. A study of more than 100 models on 80 tasks found 45 percent of completions introduced an OWASP Top 10 vulnerability, with cross site scripting failing 86 percent of the time. Those tasks were built to contain security choices, which makes the figure a laboratory result, but the pattern matches what reviewers report.

What testing practice catches agent written bugs?

Write the acceptance tests before the agent writes the code, and do not let the agent edit them. When one model writes both the implementation and its tests, the tests tend to restate the implementation, and a failing test can be quietly weakened to turn the build green. Protect the test directory so changes need a human owner.

Then measure whether the tests detect anything. A paper from Google describes mutation testing across 776,740 code changes, with surviving mutants shown to authors during review. Engineers rated 15 percent of surfaced mutants as productive at first and 89 percent once selection improved. Coverage percentage alone cannot tell you that.

How should code review change when agents write the code?

Read the specification and the tests before the diff, use a reviewer that is independent of the generating agent, and cap pull request size. Run static analysis, secret scanning, and dependency checks before a human looks, so the human spends attention on intent and design.

On a submarine, nobody verified their own valve lineup. A second qualified watchstander checked it from the procedure, not from the first person’s memory. An agent confirming its own pull request is the same mistake. The reason to insist on independence is that errors from one source are correlated, and a second pass from the same model shares the first pass’s blind spots.

Which design and release practices limit the damage?

Typed interfaces, narrow modules, and a conventions file loaded at the start of every agent session give the agent a target and give tests a place to attach. Duplication is the quiet cost: an analysis of 211 million changed lines found copy and pasted code rising from 8.3 percent to 12.3 percent between 2021 and 2024, while refactoring fell from 25 percent of changed lines to under 10 percent.

On release, separate deployment from exposure with feature flags, send a small share of traffic to each new version, and roll back automatically when error rate or latency breaches a service level objective. A defect that passes every gate should reach few customers for a short time. Tag every incident by whether an agent wrote the change and which gate should have caught it.

Why does this matter to investors and enterprise customers?

In technology diligence, buyers ask for the incident log and the release process be