What 239 pull requests taught us about supervising AI agents

Somewhere around the fortieth pull request we stopped opening the diff first. We started opening the tracker first.

Axial coordinates its own development. As of August 2, 2026 that tracker holds 330 issues, 299 of them closed, behind 239 merged pull requests. Agents did most of the implementation. A small private team supervised, and this note is what the supervising actually consisted of, including the parts we got wrong.

Our starting assumption was that supervising agents would feel like reviewing junior engineers at higher volume. Reading code turned out to be the cheap half. The expensive half was establishing plain facts. Who owns which task. Whether an agent that went quiet is still alive. What a closed issue proves.

The pattern underneath the failures

Every failure we hit had the same shape. Some fact about the work existed only in an agent’s own account of it. We started calling this unwitnessed work: state that lives in an assertion no other part of the system can check.

An agent asserts a task is its own after reading a list that was stale before the read finished. An agent asserts nothing at all, because its session died an hour ago and dead processes file no reports. An agent asserts an issue is done when the code merged nowhere.

The fix was the same move each time: turn the assertion into a write the system can accept or refuse.

the assertion         the write that witnesses it
"this is mine"   ──▶  conditional claim; exit 4 if someone holds it
"still on it"    ──▶  a lease the holder must keep renewing
"I'm blocked"    ──▶  a decision filed on the issue, out of the queue
"it's done"      ──▶  a close that carries evidence refs

Which failure arrived first?

Duplicate work. Two agents polling for tasks would read the same unassigned issue and both begin, and we paid for the same change twice, occasionally in two conflicting branches. The repair was making the claim a single conditional update. The loser gets error[CONFLICT] and exit 4, selects other work, and moves on. After that landed we stopped seeing two agents on one issue.

We still saw the semantic version: two separately filed issues describing the same work. No write-time predicate catches that. A person notices it, usually while answering something else.

Which failure was quietest?

Silent abandonment. A session gets killed mid-task and nothing anywhere says so. The issue stays assigned, the status stays in progress, and the queue routes around a ghost. Before leases, we found these by scanning for stale issues, which means we found them late.

A lease bounds the damage: ownership carries an expiry, a live agent renews it with a heartbeat, and once it lapses the next claim simply wins. The cooperative exit is different from the crash. An agent that leaves on purpose must release with a handoff note covering what is finished, what remains, and what will trap the next actor. The lapsed lease recovers ownership. Only the handoff recovers context, and the gap between those two is a real cost we still pay whenever an agent dies instead of releasing.

Which failure cost the most?

Done without proof. Early on, an agent would close an issue because its local tests passed, and the pull request would then sit unmerged or die in review while the tracker showed finished work. Every downstream plan built on that close inherited the error.

Closes now carry an outcome and evidence references, a pull request and a commit, anchored to the commit the evidence was verified against. A close that lacks evidence is visible as exactly that when you view the issue. The guard checks that the references are present and well formed. It cannot read the pull request, so we still sample closes by hand. The honest gain is smaller than we wanted and still decisive: the tracker now tells us which closes to distrust.

What surprised us?

Supervision turned into answering questions. A blocked agent used to stall, or worse, guess. Now it files a structured decision on the issue, with options and context, and takes the next task. The issue drops out of the dispatch queue while the question is open, and we answer from a web inbox or by replying to an email. The agent that asked cannot answer its own question.

Most supervision days now look like this: answer the open decisions, then sample evidence on recent closes. The other surprise was how much the activity log mattered. Every mutation records which actor performed it, so “what happened here” became a read instead of an investigation.

What still does not work?

Locally, the actor on a mutation is whatever the process claims. Binding that identity to an authenticated principal is cloud-mode work, and until then attribution is a discipline rather than a guarantee.

An expired decision reads as expired when you look. Nothing pings the supervisor who forgot to look, so an ignored question can quietly outlive its usefulness.

And evidence remains pointers. The system witnesses that proof was attached, while judging the proof is still our job.

The implication

The 239 matters less than what it sits on: a pile of assertions that used to be taken on faith and are now writes with predicates. If you supervise agents, list the claims you currently accept on an agent’s word. Ownership, liveness, blockage, completion. Each one you leave unwitnessed is an investigation you have scheduled for later.

Make every claim a write the system can refuse.