The last two editions built toward a decision, and stopped one question short of it. The replacement test asked which parts of your AI would move with you if you swapped the model tomorrow, and which were quietly chained to a vendor. The bakeoff answered how to find out, before a renewal forces the question, whether a cheaper model can actually do the job — by running both models on real work and scoring each on a check that decides whether a piece of work is acceptable. Both editions leaned hard on that word, acceptable, and both left the harder question sitting underneath it, untouched.
Here it is. When the agent reports back that the work is finished, how do you know it is? Not whether the answer reads well on the page. Whether the thing the work was supposed to produce actually exists, in the place it was supposed to exist, in the state it was supposed to be in. The invoice posted. The record updated. The file written where the next step will look for it. "Done" is not a feeling and it is not a summary. It is a claim about the world — and this edition is about why the machines are worst at checking that claim precisely when it is theirs to make, and what to do instead.
The worker cannot grade its own work
Start with the uncomfortable finding, because everything else follows from it. A study published this summer set out to measure how often an AI agent that says it finished a task actually did. The answer, on one coding benchmark where the agent assessed its own success, was that roughly three in four runs reported success while failing — the agent closed the task, declared it done, and was wrong.[^1] A second benchmark in the same work came out less severe but still bad enough to end any comfortable assumption that a confident completion message means the work is complete.[^1] We carry these as the paper's reported figures, attributed to its author, not as numbers we generated ourselves — but the direction is not in dispute, and it is the direction that matters.
The title the researcher gave the paper is the whole lesson in five words: From Confident Closing to Silent Failure. Confident, because the agent does not hedge — it reports success in the same even tone whether the work is finished or not. Silent, because nothing announces the gap: no error, no crash, no red flag, just a plausible summary of a task that was never completed and a system that has moved on as though it were. The reason is structural, not carelessness. The machine that did the work is the worst-placed thing in the building to certify it, for the same reason a student is the worst-placed person to mark their own exam: the judgement is made by the same process, with the same blind spots, that produced the work. Self-report is not verification. It is the work, describing itself.
The cheapest reviewer is a coin toss
The obvious fix is the one most teams reach for first: have a second model check the first. If one agent's self-assessment cannot be trusted, put a reviewer model behind it and let the reviewer decide whether the work is really done. It is cheap, it is fast, and it feels like independence.
The same study measured exactly this, and the result is the one worth pinning to the wall. Asked to tell a real success from a confident false one, the model judges performed barely better than a coin toss.[^1] The best of them landed only just above even odds — poor enough that the point stands on its own: a reviewer that is barely better than a coin toss is not a reviewer. It is a second opinion from the same kind of mind that produced the first, inheriting the same blind spots and adding the appearance of scrutiny without the substance. Two models agreeing that the work is done tells you they agree. It does not tell you the work is done. And a vendor's own completion message — the green tick the platform shows you when its agent finishes — is the same problem wearing a friendlier face: the party that did the work, certifying the work, and asking you to take the certification on trust.
Which leaves a plain conclusion. Certification of "done" cannot come from the machine that did the work, from another machine of the same kind, or from the vendor that sold you either. It has to come from somewhere the work cannot argue with.
What should exist, and where?
Somewhere the work cannot argue with turns out to be the world itself — the actual state of the systems the work was supposed to change. The practitioner whose discipline these pages have been drawing on frames the move as a single question, asked before the work starts rather than after it ends: what should exist, and where?[^2] It sounds almost too simple to be a method, and that is its whole value, because it forces you to write down — in advance, in terms a system can be read against — what "done" would actually mean.
Four things go on that list, and they are worth naming precisely.[^2] The outcome: the concrete thing that should exist when the work is finished, stated as a state of the world and not as an activity — a posted journal, not "reconcile the ledger." The access it depends on: the permissions, connections, and inputs the work needs in order to produce that outcome at all. The quality bar: what separates an outcome that is merely present from one that is actually right, because a record can exist and still be wrong. And the evidence: the specific, readable signal — a row in a system, a status field, a document at a known location — that would let someone other than the agent confirm the outcome without taking the agent's word for it.
The access line does quiet, disproportionate work, so give it a rule of its own. If the access an outcome depends on cannot be satisfied — the connection is missing, the permission was never granted, the input never arrived — then the honest state of that work is not done. It is not failed, either. It is blocked, and blocked is a different word that points at a different fix: a missing dependency to go and secure, not a defective result to go and repair.[^2] The reason this matters is that a silent failure and a genuine block look identical in an agent's summary — both come back as "done" or as a confident approximation of it — and only a list written in advance, that separates the outcome from the access it required, lets you tell one from the other.
Read it at the first promotion
The evidence line is where verification stops being a checklist and becomes a habit. The practitioner's phrase for where to look is the first promotion: the moment a result first moves out of the agent's own workspace and into a system that other people, or other steps, will rely on.[^2] That is the point where you read the truth — not from the agent's report of what it did, but from the state of the system that now owns the result. Did the record actually post? Read it from the ledger, not the summary. Did the file land where the next step expects it? Look in that location, not in the log. The agent's account of its work and the world's account of the same work are two different documents, and the second is the only one that can certify the first.
This is the same instinct the bakeoff ran on, followed one level down. There, the check had to be shown to reject a known-bad file before either model was trusted to pass — you proved the check could say no before you believed it when it said yes. Here, the same rigour applies to the word "done" itself: you decide, before the work runs, what state of the world would make it true, and where that state would be visible, so that when the agent reports success you are not reading its confidence. You are reading the ledger.
Your Waypoint
Take the single most important thing one of your agents does unattended, and before it runs again, write its four lines. The outcome, stated as a state of the world. The access it depends on, with the standing instruction that unmet access returns blocked, not done. The quality bar that separates present from correct. And the evidence — the exact place in a system of record where someone who does not trust the agent could go and confirm the result for themselves. Then wire the acceptance of that work to the evidence line and nothing else: not the agent's summary, not a second model's opinion, not the platform's completion tick. Read the state at the first promotion, from the system that owns the result. It is an afternoon's work on one workflow, and it converts "the agent said it was done" into "the ledger says it is."
And here is what you are holding when you have done it. Every layer this series has examined can be rented. The model can be swapped, and the last two editions showed you how to price the swap. But the judgement of whether the work is real, safe, and finished cannot be bought from the same market that produced the work, because a machine cannot be both the worker and the honest witness to its own output, and a vendor's completion message is a claim, not a proof. Verification is the one layer that has to be yours. It is where "done" stops being something a supplier asserts and becomes something your own organisation can read for itself — and in an era when the work is increasingly done by systems you did not build and do not control, the capacity to certify the outcome independently is not a nicety of good governance. It is the difference between owning a result and merely being told you have one.
The full reading of this week's model economics — the price collapse, the licence terms, and the verification discipline behind them — is in The Capability Brief — Edition 20, and the archive sits at the intelligence hub.
References
[^1]: The false-success finding is drawn from Laksh Advani, From Confident Closing to Silent Failure, arXiv 2606.09863, 1 June 2026. The paper reports that roughly three in four agent runs on one coding self-assessment benchmark claimed success while failing, and a lower false-success rate on a second benchmark; and that model-based judges asked to distinguish a real success from a false one performed barely better than a coin toss. Carried as the paper's reported findings, attributed to its author and surfaced through practitioner analysis, not asserted as an independent BusinessGPS result. Model-based judges asked to distinguish a real success from a false one scored only about 0.65 on a 0-to-1 discrimination scale (0.54 on the coding benchmark), barely above the 0.5 even-odds line — no model judge reliably catches false success.
[^2]: The verification method — asking before the work begins what should exist and where (an outcome stated as a state of the world, the access it depends on, a quality bar, and the readable evidence that would prove it), treating an outcome whose required access cannot be satisfied as blocked rather than done, and reading the result at the first promotion (the point where it moves into a system others rely on) rather than trusting the agent's own summary — is the practitioner's method (Nate B Jones); presented as observe-and-adapt guidance and attributed as method, not asserted as a BusinessGPS finding.
The Way Pointer is published by BusinessGPS — the transformation capability platform helping organisations make strategic choices about what they build, what they buy, and what they own.