A dark factory turns a written intent into merged code with nobody in the middle. No one watches the logs, no one babysits the agent, no one clicks merge. You inspect the artifacts afterwards — if you inspect anything at all.
That part is genuinely easy, and you can build it in a weekend. It is also not the part that decides whether any of this works.
Borrowed from manufacturing. A dark factory is a plant that runs without lighting because there are no people on the floor to see by. The software version is the same claim: the work happens, and nobody is watching it happen.
That is the whole difficulty. Every hard problem here is a trust problem, and every trust problem is solved the same way — by moving a check out of your head and into the repository, where it runs whether you are there or not.
Shift left — everything before the pull request. How does intent get into the queue cheaply and without ambiguity?
Shift right — everything after it. How do you know this is safe to merge without reading it?
Build the generator first and you get a machine that produces thirty pull requests a day that you personally have to review. That is strictly worse than writing the code yourself.
Select a phase to see what it needs, how it works, and the characteristic way it fails.
Phase 9 is what makes it a factory rather than a script. Without it, quality is frozen at whatever you configured on day one.
Ordered by how much of your attention they deserve, which is close to the inverse of how much attention they usually get.
An AGENTS.md that states the architecture, the invariants, and the exact commands. Plus a one-command reproducible environment and a single container image shared by the sandbox and CI. If you have explained something to an agent twice, it belongs here.
Linters, tests, verifiers, and agentic review — ordered cheapest and most deterministic first. This is the component that decides whether the factory is an asset or a liability. Everything else is plumbing around it.
build this firstAn issue tracker as the single source of truth, with labels for state and risk and native dependencies for sequencing. Your job shifts: you no longer take a ticket and implement it — you do the thinking, and the ticket is your output.
your new jobA small database holding tasks, claims, and an append-only event log. Uniqueness enforced by the schema, claims taken with a conditional update, and a lease reaper for sandboxes that died. Unglamorous, and the top source of subtle bugs.
idempotency or bustA container per task with the full toolchain, able to run the entire stack — integration tests, a database, a headless browser. Scoped credentials only. If the agent cannot run what you would run, it cannot verify what you would verify.
size it generouslyPolls for work, claims it, launches the sandbox, opens the pull request, watches for signals, wakes the agent to fix them. A few hundred lines. Everybody should write their own, because it is where your specific tools plug in — and because it is easy.
easy · not differentiatingScreenshots, recordings, and test output attached by the agent. This turns review from re-doing the verification into checking that it was done, which is the difference between a thirty-second approval and a twenty-minute one.
underratedA readable rules file mapping paths to blast radius, deciding what merges on green and what waits for you. Keep it human-legible: when something bad lands you need to see why it was classed safe and change that.
a file, not a modelA scheduled job that reads recent review comments, finds what recurs, and proposes moving it down the ladder — from expensive review to cheap verifier to free lint rule. This is the ratchet that makes the system improve on its own.
makes it a factoryEvery check belongs on the lowest rung that can express it. Findings climb down over time; nothing should climb up.
Formatters, linters, type checkers, dependency-boundary rules. Write the fiddly ones you never wrote before — agents do not mind fiddly, and they will iterate against a rule twenty times without complaint.
Your end-to-end suite is the real specification — it is what would survive a rewrite in another language. A behaviour that exists only in a unit test is not specified, it is merely currently true.
One natural-language assertion about a diff, judged by one model call answering one question. Path-globbed so triggering is deterministic. This is how architectural taste gets encoded instead of re-explained.
A full reviewer reading the diff, run as several narrow lenses rather than one broad prompt. Its job is to catch what the lower rungs missed — and anything it catches twice should become a verifier.
A single assertion about a diff that resolves to true or false, evaluated by one model call that judges only that one thing.
It beats one large review prompt for four reasons. Triggering is deterministic — a path glob decides, so there is no "the reviewer didn't notice". Each call carries one concern, and a model asked fifteen things at once quietly drops several. It is cheap enough to run every time. And it is legible: you can read a verifier and know exactly what it enforces, which you cannot do with a four-hundred-line review prompt.
Historically this kind of rule lived in a staff engineer's head and got enforced one pull request at a time. Written down once, every agent obeys it forever.
# deterministic trigger, LLM judgement - id: api-does-not-import-db paths: ["src/api/**/*.py"] exclude: ["src/api/app.py"] severity: block assert: > No file imports from db, directly or via an aliased import. The API layer depends only on lib. Only app.py, the composition root, may touch db. # one property per verifier. "and" means two. - id: money-is-integer-cents paths: ["src/**/*.py"] severity: block assert: > No monetary value is a float or Decimal. Amounts are integer minor units, because floats silently lose money. # $ verify --base main PASS api-does-not-import-db FAIL money-is-integer-cents routes/checkout.py:44 total = 0.0
Partition by blast radius, not by how nervous the change makes you feel.
| Class | Typically | Policy |
|---|---|---|
| low | Docs, tests, internal tooling, dev scripts, the factory's own code, non-user-visible refactors | Auto-merge on green |
| med | Backend behind existing tests, dependency bumps, internal APIs | Auto-merge where verifier coverage exists for the touched paths; otherwise review |
| high | Auth, payments, migrations, anything user-visible, infrastructure and deploy config, secrets handling | Always a human review |
Begin with your own tooling and the factory's own repository auto-merging, and everything user-facing reviewed. Move one category across at a time, and only when the verifier coverage for those paths has earned it.
A path that was low-risk when you wrote the rule grows into something load-bearing. Re-read the risk map on a schedule — it is the one config file that silently goes stale while looking correct.
Ranked, roughly, by how much time each one takes before you work out what happened.
Building the orchestrator first. It is the fun part and the part that does not matter. Build it before the verification layer and you have constructed a device for generating homework.
Non-idempotent enqueueing. Two agents, two sandboxes, one branch, races that sometimes work — the worst kind. Enforce uniqueness in the schema on day one rather than trying to be careful.
Verification that lives only in unit tests. You will believe you are covered and you will not be. The test: could you regenerate this subsystem from scratch and have the suite catch the difference?
Reviewing everything. The metric is not pull requests shipped. It is the percentage that merges without you — and you raise it with better checks, not with more agents.
Rolling back instead of fixing forward. Reverting fixes one pull request. Asking which verification gap let it through, and closing that, fixes the entire class.
Under-provisioning the environment. If the agent cannot run integration tests, it will write code that passes unit tests and breaks the system — and you will blame the model.
A verifier that never fires. A path glob that quietly matches nothing reads as coverage while providing none. Test every check in both directions: it must fail on a violation and pass on a clean diff.
Not feeding it. At small scale this is the real limiting factor. A factory with an empty queue is an expensive way to run zero jobs.
Your job stops being implement the ticket and becomes own the environment the factory runs in. A mechanic does not build your car, but can diagnose it — the expertise is still deeply technical, it just operates a level up.
Autonomy is earned, not enabled. You do not flip a switch and run at full speed. You widen what merges without you exactly as fast as the verification layer earns it, and not one category faster.
Write the rule down, not the correction. Anything you have explained twice belongs in the repository — as context if it needs explaining, as a check if it needs enforcing.
Fix forward, always. When something bad lands, the question is never whether to revert. It is which check was missing.
Your end-to-end tests are the specification. Everything else is an implementation detail the factory is free to rewrite — and eventually will.