Episode Details
Back to Episodes
Stack Overflow Survey Exposes the AI Code Review Gap
Description
Stack Overflow’s 2025 Developer Survey, covered by ADTmag on January 20, 2026, found developer trust in AI generated code fell from 40 percent to 29 percent even as adoption reached 84 percent. That gap between how much developers use AI generated code and how much they trust it is not a confidence problem a better rollout plan fixes. It is a missing control, and the fix is a provenance tagged human review gate that catches an AI authored pull request before it reaches a production system, not after. Forty six percent of developers now say they actively distrust the code their own tools produce, and they are merging it anyway.
That last fact deserves more attention than a headline usually gives it. A developer who distrusts a tool but keeps using it has not solved a trust problem. The developer has outsourced the problem to whoever reviews the pull request next, and the research on what that reviewer actually does with it is not encouraging. A January 2026 study of more than 18,000 merged pull requests across 2,100 open source repositories found that 80 percent of AI co authored contributions from outside collaborators merged with no explicit review at all, and that 86.9 percent of the repositories studied had no written policy on AI coding agent use at all. The review step most engineering teams assume is happening is, in most of the repositories studied, simply not occurring.
why did developer trust in ai code fall to 29 percent
Trust fell because the failure mode is specific, not vague. Sixty six percent of developers in the Stack Overflow survey described AI output that is almost right, but not quite, the exact category of defect that defeats a quick read and requires a reviewer to trace the logic line by line. Veracode’s testing of more than 100 large language models across 80 coding tasks found that 45 percent of generated code samples contained a security flaw, and the failure rates were not evenly distributed: 86 percent for cross site scripting, 88 percent for log injection, against only 14 percent for cryptographic misuse and 20 percent for SQL injection. The models have gotten good at the defects that appear often in public training data and have not moved on the defects that do not, and that pattern has held steady across model generations even as the code reads more fluently with each new release.
A Stanford and NYU study published at the 2023 ACM conference on computer and communications security found something more unsettling than the raw defect rate. Developers given access to an AI coding assistant wrote measurably less secure code than a control group working without one, and they were more likely to believe their code was secure precisely when it was not. That is the mechanism behind the trust numbers. A tool that produces confident, plausible, wrong code introduces a second defect behind the first one. It degrades the reviewer’s own ability to tell the difference, because the code no longer looks like something written in a hurry. It looks finished.
what review gate actually catches ai generated defects
Submarine reactor operations run on a rule that has nothing to do with speed and everything to do with consequence: no single operator touches a safety critical control alone, and the second person in the room is not there to make the work go faster. I watched that rule enforced on a boat where the cost of a missed step was measured in reactor safety, not sprint velocity, and the underlying logic transfers directly to a pull request that touches an authentication boundary or a safety relevant code path. The fix for AI generated code is the same shape: a pull request carrying AI authorship metadata should route automatically to a mandatory second reviewer, and that routing should not depend on a developer remembering to flag it or a teammate remembering to ask.
GitHub’s own guidance for reviewing Copilot generated code describes an eight step process that starts w