An Agent Believes Whatever Comes Back
A person reading a suspicious result squints at it. An agent builds on it. So every tool in the stack is held to one rule: when it could not do something, the output says so.
Hand a person a code scan that found nothing in a large, old repository, and they get suspicious. Hand them a search that returned four results where they expected forty, and they look again. That suspicion is doing real work. It is the reader filling in what the tool did not say.
An agent does not squint. It takes what came back as the state of the world and builds the next step on it. If a tool returns fifty rows, the agent reasons about fifty. If it returns nothing, the agent concludes there is nothing there. The more of the work an agent does unattended, the less of that human suspicion is left anywhere in the loop.
That changes which failures are expensive. An error is cheap: it stops the agent, and someone looks. The expensive output is the plausible one, a result that is quietly incomplete and reads as complete.
One rule across the stack
Every product at Code Reality Labs is held to the same rule: when a tool could not do something, it says so in the output, where whoever reads it next will see it. An incomplete result is fine. An incomplete result that presents itself as complete is a defect.
Here is what that looks like in each layer.
- TheAuditor reports a part of a project it could not analyze as a finding, with the reason and the place, so that “no findings” can only mean clean. The case is made in a gap is a finding, not a silent zero.
- Curator says what it knows, says plainly when it only half knows it, and says nothing when it knows nothing, because a wall of text is not an answer.
- Arbiter takes a session’s status from whether the session is actually alive, not from what its last message implied, and derives its counts from that state, as described in supervise a fleet of coding agents.
- Warden tracks what a session costs while it runs. A budget you set speaks up before it is reached and stops the work when it is, so an overrun does not first appear on a bill. More in see what a coding session is costing you.
- BenchProctor counts a finding, by default, only when it names the weakness the case is labeled with, and its answer key is checked against what the code actually does, because a wrong answer key punishes the tools that get it right.
A rule is not a guarantee
Writing the rule down does not make it true everywhere. Software grows faster than anyone’s memory of it, and new code finds new ways to come back short without saying so. The places where the rule does not hold yet are bugs, and they get found the same way every time: by using the tools on real work and checking their answers against the source, not against how confident they sounded.
That hunt does not end, and we would rather say so than imply it has. Each time it turns something up, the fix goes in where the gap was, and the output learns to say one more thing about itself.
Why it matters more over time
AI coding work is moving toward longer runs with fewer people reading each step. Every step a human no longer reads is a step where a quiet gap can pass for a complete answer. A stack meant to carry that work has to make its outputs carry their own caveats, so the agent is told what was not covered instead of having to guess.