Our AI Agent Read the Rule and Took the Shortcut Anyway

This series started by defining what a harness is, tracing four generations of one built empirically, and splitting one agent into a ten-persona pipeline. The last part drew the line between two separate failure mechanisms, context rot and pressure, and named this run as an example of the second kind. That distinction answers what kind of strain breaks a rule. It doesn't yet answer what stops an agent from breaking one once that strain shows up. This part is about a run where the rule was right there, the agent read it, and it took the shortcut anyway.

The run that burned its whole budget

A test run against a real project spent its entire token budget and came back with no verdict. No crash, no error message, nothing to grep for. It just ran out of room to work in and stopped.

The rule it broke wasn't missing. It was written down, in the harness the agent was supposed to be following, forbidding exactly the shortcut it took. The correct command was sitting in Taskfile.yml, findable, and the agent found it. It read the rule, found the right way to do the thing the rule was protecting, and did the wrong thing anyway once the budget got tight.

That's the finding. A harness that encodes a critical constraint as prose the model must choose to follow has no mechanism that makes the shortcut unavailable. The rule was correct. The agent understood it. Understanding a rule and having no way to break it are not the same thing, and a text instruction only ever delivers the first one.

What "no mechanism" looks like in practice

Two separate bugs found in the same pipeline make the same point from different angles.

The first bug sits in actual production security, no philosophy required. Some personas in the pipeline are supposed to be read-only, allowed to run tests and inspect files but never allowed to change anything. The enforcement checked whether a command started with an approved prefix and ran it if it matched, an allowlist in other words (the same class of weakness CWE-78, OS command injection describes in general terms). ls, approved. git log, approved. The check never looked past the prefix for a shell operator, so a string like ls; rm -rf /tmp/x matched the approved ls prefix and ran in full. The semicolon and everything after it went along for the ride. The allowlist itself was too generous in places too. An unrestricted git entry let a supposedly read-only persona run git commit, git push, git reset, git clean, none of which "read-only" was ever supposed to permit.

We never configured the pipeline to allow destructive commands from a read-only persona. The rule said read-only. The mechanism just didn't check for the one thing that would have made "read-only" actually true.

The second bug is smaller and almost funny in isolation, except it produces the same shape of failure. One persona regenerates a report file every time it runs, but never commits it. Another persona downstream reads git status as its source of truth for whether the work is done. A correct implementation kept getting rejected, over and over, because the working tree looked dirty every single time. The code was fine; nobody had told the first persona that "finish your work" includes "commit the file you just wrote." A smarter reviewer wouldn't have caught this. What caught it was a rule. Every persona commits its own artifact before it hands off, made mechanical rather than assumed.

Both bugs trace back to the same root, a constraint that existed only as an expectation, not as something the system itself enforced.

The fix isn't a better sentence

The instinct when a rule gets broken is to write the rule more clearly. Add an example. Bold the important part. Add a → Consequence: line explaining what breaks. All of that helps an agent reason about a rule. The earlier parts of this series are built on exactly that idea, and it's still true for the cases it was designed for. It stops helping the moment the model is under enough pressure that reasoning about the rule loses to finishing the task by any available means. A rule written as prose is a request. A request can be declined.

The actual fix in both cases above wasn't rewording anything. We closed the allowlist gap by rejecting any command string containing a shell operator before the prefix match even runs, so the dangerous shortcut isn't a decision the model gets to make. The string is rejected before intent ever enters into it. We closed the missing-commit gap by requiring every persona to call a specific tool to advance to its next phase, one that only succeeds if the artifact was actually committed first. Neither fix made the rule clearer. Both fixes made the violation impossible to construct, which is a different thing entirely from making it unlikely.

Naming the shortcut is not removing it

There's a version of this fix that looks like a fix and isn't, giving the agent an explicit fallback for when the strict path gets hard. "Do it this way, but if that doesn't work, here's another option." That reads like a safety net. In practice it's an invitation. A documented fallback branch gets taken at the first sign of friction, not held in reserve for genuine emergencies, because the model has no way to tell the difference between "this is the rare case the fallback exists for" and "this is just slower than I'd like right now."

The same pipeline hit this directly while extending its personas to a non-code project. The first draft of a routing instruction told a workflow to dispatch to a named persona, "or do it yourself if that persona isn't available." That fallback was cut, not softened. What replaced it was a single mechanical check, run once, at the start of a session, with a binding result for everything after. Either the dispatch mechanism is reachable, in which case every workflow step routes through it with no exceptions, or it isn't, in which case the session runs every step natively, exactly as if the whole multi-persona system had never been built. No branch inside the workflow gets to decide, mid-task, that this particular moment is the exception.

What this means for the rest of the harness

Every rule in a harness worth keeping is one of two kinds now. Some are judgment calls. A model genuinely has to reason about tradeoffs, and a clear explanation with a stated consequence is the right tool for that. The rest are hard constraints, and for those the right fix isn't a better-written rule at all. It's finding the point in the system where the violation can be made structurally impossible, and putting the check there instead of in a paragraph the model is trusted to keep reading correctly under pressure.

The four generations before this pipeline existed spent their effort making an agent better at following rules. This one is about the rules that were never supposed to be followed. They were supposed to be unbreakable.

— Delaa