The pilot's bottleneck changes

The Pilot model says the human directs and the agents execute. I still think that's the most useful reframe in the method.

It has one flaw, though, and it took me a while to see it. Stated plainly like that, it reads as static — as though being a pilot is one fixed posture you adopt on day one and hold forever.

It isn't. What a pilot is holds constant. What constrains a pilot does not, and it changes shape about 3 times on the way up. Every time it moved, I kept doing the thing that had worked at the previous stage and wondered why it had stopped working.

The model that reads as static

Here's the failure it produces.

You learn to be a pilot with one agent. The skill you build is reading output carefully — you review everything, you catch the drift, you develop good instincts about where the agent goes wrong. That skill is genuinely what makes stage one work, and you get good at it.

Then you add agents. And the skill that made you effective becomes the thing holding you back, because reading everything doesn't scale past about 2 streams. You're not doing anything wrong. You're doing the previous stage's right thing.

Nobody tells you the constraint moved. The model just says "direct the work", which is true at every stage and useless for knowing which stage you're in.

Attention: one agent, and you're watching

At the first stage, the bottleneck is your attention.

Trust is low, and correctly so — you don't yet know where this agent is reliable. There's no self-verification, so the only quality gate is you reading the output. Work is synchronous. You start it, you sit there, you watch.

The question you're asking, constantly: do I need to read this?

The answer at this stage is yes, all of it, and that's fine. This stage is where you build the judgement everything above depends on — which kinds of task it nails, which it fakes, what its confident-and-wrong looks like. You cannot skip it, and trying to is the most common way an agent rollout falls over.

The move out of this stage is not more agents. It's building something that checks the work so you don't have to be the only thing that does.

Review throughput: several streams, each checking itself

Second stage. You've got tests, a build, a lint step, an end-to-end check against a real environment — a loop the agent runs on itself before you ever look. Now more agents is a sensible idea, because each one arrives pre-checked.

The bottleneck moves to review throughput. You hand-write almost nothing. You read final diffs rather than watching keystrokes, and the constraint is how fast you can form a judgement on several finished pieces of work.

The question becomes: can I review this fast enough?

This is the stage most teams are climbing into now, and it's where the classic span of control problem arrives — the old management question of how many things one person can meaningfully supervise, except the things now produce output faster than any human report ever did.

The practical shift is that you stop optimising your reading speed and start optimising what arrives. A diff that comes with its own passing test is quicker to accept than one that doesn't. That's not a review skill. It's a contract skill — tightening what each job is allowed to return so the review is cheap.

Trust in the loop: more than you can read

Third stage, and the honest one. Output exceeds what you can personally read. Not "is difficult to read" — exceeds.

Plenty of people treat this as the point where quality has to drop. It's the point where the question has to change.

"Did you read the code?" stops being the right question, because the answer is no and pretending otherwise is theatre. What replaces it:

Stuart Leo

What context was the model missing, and how do we fix that for next time?

That's a different job. You're no longer the inspector — you're debugging the system that produces the work. A bad output is no longer a thing to correct. It's evidence that something upstream was underspecified, and the fix belongs in the brief, the gotcha, or the anchor rather than in the diff.

The bottleneck here is trust in the loop — and trust is a technical property, not a feeling. It means the verification is independent, it runs where the builder can't reach it, and a dispute actually blocks a merge.

This is also the stage the closing gate exists to make survivable. If you can't read everything, you need a moment where a human checks the evidence against what was asked, and the answer to that question goes straight back into the contextbase. Without it, work at this scale is unreviewed by construction.

Knowing what to automate

Fourth stage, and I'm not going to pretend I live here. What I can see from below is that the constraint changes again — to identifying what's worth automating, and matching the guardrails to each kind of work.

You monitor by exception. Most things run without you seeing them, and the skill is knowing which categories genuinely need a human in the path and which have earned their way out.

The question: is this something an engineer would have done? Not is this correct — you've stopped being able to check that directly — but is this the shape of decision I'd have wanted made this way?

What stays constant

Four stages, four constraints, four questions:

What doesn't move is the posture. At every stage you own the outcome, you set the quality bar, and you make the calls nobody should be delegating. A pilot at stage 4 is not less accountable than one at stage 1 — they're accountable for different things, and through different instruments.

That's why I'd rather deepen the Pilot model than replace it. The reframe was right. It was just missing its stages, and without them it reads as though the job never changes.

The honest cost is that each transition asks you to give up the skill that made the previous stage work. Stage 2 asks you to stop reading everything, which feels like lowering your standards. Stage 3 asks you to stop reading the code at all, which feels much worse. Both are real losses, and both are the price of the next stage. If you refuse them, you don't stay at your current stage — you stall, with more agents than you can hold.

And there's a second axis worth checking against this one, because they move independently: how risky the work is, versus how many agents you run. Two ladders, not one.

Same pilot, different instruments

I spent months at stage 2 trying to be excellent at stage 1. Reading everything, catching everything, proud of it — and quietly capping the whole system at my own reading speed.

The fix wasn't working harder or trusting blindly. It was noticing that the question had changed, and that do I need to read this had become the wrong thing to ask.

What a pilot is holds constant. What constrains them does not — and knowing which stage you're in tells you what to fix.

Start here: see the Pilot model, how agent teams work, or read the method.