ArticlesFoundations

Don't loop on confidence. Loop on evidence.

"The agent says it's done" is not a stop condition.

Stuart LeoAugust 3, 20265 min read

Here's a failure mode that gets more expensive the more sophisticated your setup becomes.

You build the whole thing properly. A worker agent does the job. A checker agent reviews the worker. An audit agent reviews the checker. Every layer confirms the one below it, everything comes back green, and the report is clean.

And every single one of those agents was reading another agent's report.

Nothing in that system ever touched reality. It's consistent and unverified — which is exactly how a single agent fails, only later, more expensively, and with far more green lights on the way down.

Everything is consistent, nothing is verified

The trap is that consistency feels like verification. Three agreeing sources sounds like corroboration. It isn't, if all three are downstream of the same unchecked claim.

I hit a small version of this on a data migration. The migration agent reported success. The verification agent confirmed the row counts matched. The summary agent reported a clean migration. All true, all useless — because the verification agent had asked the migration agent for the counts rather than querying the database. One select count(*) would have shown the target table at 40 rows instead of 40,000.

Nobody lied. There just wasn't a single point in the chain where anything left the conversation and checked.

This is the same instinct behind self-testing code — Martin Fowler's point that a suite you can run is worth more than a belief that the code works. What's changed is that the thing telling you it works is now articulate, confident, and generates its own supporting detail.

What an anchor is

An anchor is a signal no agent in the run can produce by asserting it.

That's the whole definition, and the test in it is produce by asserting. Can the agent make this true by writing a sentence? If yes, it isn't an anchor.

  • "The tests pass" — not an anchor. That's a claim.
  • npm test exiting 0, in output you can see — an anchor.
  • "The migration completed" — not an anchor.
  • A query returning 40,000 rows — an anchor.
  • "The endpoint is live" — not an anchor.
  • A 200 from a real request to the deployed URL — an anchor.

The pattern is that an anchor comes from outside the model's mouth. A test that ran, not one that should pass. A query that returned rows. A deploy that resolved. A metric that moved. The agent can report these, but it can't make them true by reporting them, and that gap is the entire point.

Stuart Leo

An agent that grades its own homework will pass itself. Not from dishonesty — from doing exactly what you asked, which was to produce a final message.

Why "testable" was always in the gate

If you've been running C², you've had a partial version of this the whole time and may not have noticed which word was carrying it.

The prompt brief quality gate has 6 items, and item 3 is testable acceptance criteria. Not "clear" criteria. Not "specific" criteria. Testable — meaning there exists a thing you can run that comes back true or false without anyone's opinion. That word was doing anchor-work before I had a name for it.

What was missing was making it explicit, because "testable" is easy to satisfy loosely. "The refresh flow works correctly" passes a casual read and fails the actual test — there's nothing to run.

So the fix is a naming discipline, not a new process step. Every brief names its anchor, and it lives inside the acceptance-criteria item rather than becoming a seventh:

- [ ] Testable acceptance criteria — and name the anchor: the one signal
      that decides done, which the agent cannot produce by asserting it.

      Weak:   "Token refresh works."
      Anchor: `npm test -- auth/refresh` exits 0, AND a real refresh
              against staging returns a new token pair (paste the response).

The gate stays 6 items. Item 3 now asks a sharper question: which signal decides, and can the agent fake it?

Frozen rules — the ones an optimiser would bend

There's a companion idea, and it covers the case where the anchor itself comes under pressure.

Some rules are non-negotiable precisely because they're the ones something optimising for "done" would want to relax. Mark them frozen. Mine, earned the hard way:

  • The write-path check after any constraint migration. A near-empty table is a broken writer, not low adoption. I learned that distinction from getting it backwards.
  • The file-size cap. Every argument for exceeding it has been reasonable and every one was wrong.
  • Never fabricate data to make the software look like it works. Including test fixtures that quietly paper over a real failure.

Every frozen rule exists because it got bent once and something broke. They belong in the contextbase next to the gotchas, and they belong in the brief when the work touches them.

A system is only as honest as the things inside it that refuse to move.

Making the verifier independent

One more mechanical point, because "add a reviewer" is where most people stop and it isn't sufficient.

Independence isn't an attitude the reviewer has. It's three conditions:

  1. A different model from the one that wrote the code. Two models catch different blind spots. One model checking itself catches the ones it already missed.
  2. Inputs the author doesn't control. Run the check where the builder can't reach it — in CI, from the acceptance criteria and the diff. A builder running the verifier on their own machine, on inputs they assembled, is self-assessment with extra steps.
  3. Teeth. A dispute blocks the merge. A reviewer whose objection is advisory gets overruled at 5pm on a Friday.

A bench agent gets you most of the first condition for free, because a separate process is a separate context by construction. The other two you have to build.

The honest cost is that anchors slow you down at the start. Writing "the tests pass" takes 3 seconds. Working out which command, against which environment, returning what, takes 5 minutes and sometimes reveals that you can't verify the thing at all — which is unwelcome information arriving at the least convenient time. That's the price, and it's the good kind, because the alternative is finding out in production.

What makes a result count

More agents don't make a system more trustworthy. They can make it more confident, which is worse, because confidence at scale looks like corroboration and reads like a clean report.

The fix isn't more reviewers. It's making sure at least one thing in the loop touches something the model can't talk its way past. Name the anchor. Freeze the rules that will come under pressure. Give the verifier a signal rather than a summary.

A result counts when something the agent cannot fake says so.

Start here: see how to write acceptance criteria an agent can check, how a bench agent reviews work, or read the method.

FAQ

How do you know when an AI agent has actually finished a task?
When something the agent cannot produce by asserting it says so. A test that actually ran and passed. A query that returned rows. A deploy that resolved. That signal is the anchor. "The agent says it's done" is a report about the work, not the work — and an agent that grades its own homework will pass itself.
Why do AI agents report success on work that is broken?
Because completing the task and reporting completion are two different actions, and only the second one is easy. A model optimising for a plausible final message will produce one whether or not the underlying work holds. It isn't lying — nothing in the loop ever required it to check.
Does adding more review agents fix the problem?
Not on its own. If every reviewer is reading another agent's report rather than a real signal, you get a system that is internally consistent and entirely unverified. It fails the same way a single agent fails — later, more expensively, and with far more green lights on the way down. At least one node has to touch reality.