The Lights-Off AI Coding Factory Is a Trap

The Factory That Runs Itself Is Missing Something Load-Bearing

The pitch hooks you. Stripe, Ramp, WorkOS, and Brex say roughly 75% of their code now ships from agent factories. Dan Shapiro named the end state: the lights-off software factory — tickets in, PRs out, no human reads the code. Simon Willison profiled StrongDM running a version of it. The math feels clean. Build time drops from days to minutes. Review looks like the choke point. So you automate review too. Flip the switch. Lights off.

Here’s the uncomfortable claim: it works for two to six months, then it breaks in a way agents can’t fix. This isn’t theory. HumanLayer’s Dex tried it in July 2025, took three production incidents by November, and his cofounder spent two full weeks in vanilla VS Code — not even Cursor — rewriting the patterns by hand. If you’re building a vertical AI tool and your agent queue is growing faster than your review discipline, your clock is already ticking.

Review Was Never the Bottleneck. Trust Was.

In 2022 your shop ran on a simple beat. You write a ticket. Someone builds it. Someone else reads the diff. Ship. Builds took hours or days. Reviews took hours. They matched, and review did two jobs: it caught bugs and it kept a shared mental model alive.

Agents crush the build step to minutes. You look at the graph and decide review is the new bottleneck, so you juice review. Automated linting. Agent PR review. Browser bots that run regression suites and DM you a cute video when they pass. Awesome. Review is faster.

But you quietly killed the second job. Nobody on your team has read that code. Nobody knows why the geometry module now imports from three unrelated subsystems. That review step was load‑bearing in a way throughput never showed.

Codebases Rot in Three to Six Months

If you’ve actually run an agent factory, you know this arc. Weeks one through eight feel like magic. Ticket velocity jumps 4–5x. You finally ship the year-old backlog. Bugs get PRs before your coffee cools. You start believing the vibe-coding demos apply to production.

Around month three, a small edge case pops in a subsystem you haven’t touched since the agent wrote it. You ask the agent to fix it. It patches. Looks good. Two days later a different subsystem breaks — the first one was silently coupled to it through a helper the agent extracted six weeks ago and nobody noticed. You ask the agent to fix that, too. Another patch. The third failure is where you stop.

“Brownfield” used to mean a ten-year-old Java monolith. Now it’s the codebase your team generated four months ago through a harness that neither you nor the agent can navigate under pressure. That’s the failure mode. Not “agents write bad code.” Agents write code nobody has a map for.

Shotgun Surgery, Now at Machine Speed

Martin Fowler named it in Refactoring: shotgun surgery — one behavior change forces edits across a dozen scattered files because responsibilities were never cleanly separated. That’s the telltale sign of a codebase that stopped compounding value and started compounding pain.

Agents default to shotgun surgery for a simple reason: the training loop rewards fixes, not futures. Look at SWE-bench Multilingual. It scores traces on two signals: did you flip FAIL_TO_PASS, and did you keep PASS_TO_PASS. Take fastlane__fastlane-19304 — two Ruby params default to nil and crash on .empty?. The human fix is two lines. An agent can wrap a function, monkey‑patch, extract a validator, or paste checks across fourteen call sites. Every path scores the same. There’s no penalty for torching maintainability.

Calvin French‑Owen from the Codex team said it cleanly at AI Council: RL makes models great at whatever the verifier can measure in seconds. Tests run in seconds. Bad architecture charges interest over weeks. There’s no fast oracle for “did this make the codebase easier to change six months from now.” So labs don’t train against it. Models don’t optimize for it. Your agent happily couples three modules that should never meet, and the verifier gives it a gold star.

Claude Code beating aider, cline, and codebuff — going from zero to ~$9B in revenue in a year — came from Anthropic training the model inside the harness against those verifiers. It’s a real breakthrough. It also means the model is elite at what’s measured and blind to what isn’t.

Front-Load the Alignment, Not the Cleanup

Here’s what works for teams that aren’t blowing themselves up: get alignment before code. The pre‑AI factory learned this decades ago, then forgot it.

Do the concrete thing. Before an agent touches anything non‑trivial, a human writes a one‑page architecture note. Not a novel. One page. What’s changing. Where it lives. What it touches. What’s explicitly out of scope. You review that with the person who will read the PR. Then let the agent code.

This buys you two wins. The agent lands closer to right because the context is set early — the same reason a crisp PR reviews in ten minutes and a mystery PR eats two hours. And at least one human now holds the mental model before the diff exists. When something snaps in month five, someone can open the file and know why it looks that way. That shared understanding — not the code — is the real artifact.

What To Do Monday Morning

If you’re running a real product on an agent‑heavy workflow, do this before your next sprint:

Audit your blast radius. Pick three files your agent modified last month. Open them cold. Can you explain them in ninety seconds without reading line‑by‑line? If not, you’ve already lost the map. Rewrite those areas with a human plan first.

Reinstate one review checkpoint you deleted. Not all of them — you don’t need 2022’s ceremony. Pick the spot closest to your core domain logic: the pricing engine, the geometry solver, the compliance rules. Any PR that touches that module gets a human reading every line. Everywhere else, keep the agents humming.

Write the architecture proposal before the ticket. Not after. Not in parallel. Before. One page. Twenty minutes. Then hand it to the agent. The Stripes and Ramps of the world get 75% throughput not by killing review, but by front‑loading design so review is trivial.

The lights‑off factory is real. StrongDM looks like it’s running one. HumanLayer is too — in a modified form — after the rewrite. What nobody is running is the naive version: agents in, code out, nobody reads it. That version’s been tried. Around month four, it rewrites itself in vanilla VS Code. Don’t spend two weeks learning that lesson the hard way.