Spec Kit, BMAD and OpenSpec, and the Cost of Planning First

The last five parts of this series were about a harness watching an agent act. Retry logic and a ten-persona pipeline, the allowlist that let a shell operator through, the rule an agent read and broke anyway once the budget got tight. This part is about a different kind of harness, one that runs before any code exists at all, gating what gets built rather than how it gets built. Three tools claim to do that job the same way. They don't. Reading all three end to end turned up a controlled benchmark that says the disagreement has a real cost, and a mechanism worth keeping regardless of which tool it came from.

What three tools claiming the same thing actually do differently

GitHub's Spec Kit, BMAD-METHOD, and OpenSpec all sit in the same category and turn a request into a written spec before an agent starts writing code. Installing and reading all three, not skimming READMEs but the actual skill definitions each tool hands an agent, turned up three different answers to the same design question of how much should stand between a request and the first line of code.

Spec Kit runs a fixed sequence. Constitution, then specify, then plan, then tasks, then implement. Each stage gates the next; a checklist has to pass before the pipeline advances. BMAD-METHOD routes adaptively. Its own entry-point workflow checks whether a change is small enough to skip straight to implementation, and only falls back to full multi-stage planning when the request doesn't fit that fast path. OpenSpec drops the gate model entirely. Its own README calls this "fluid not rigid," and its implementation skill can run at any point, before or after other artifacts exist, artifacts editable anytime.

Three teams, the same stated problem, three different answers about how much ceremony a spec deserves before code gets written.

The benchmark that should worry anyone about to adopt one of these

Scott Logic ran a controlled comparison that's worth taking seriously precisely because it isn't a hit piece. The same engineer rebuilt the same ~1,000-line feature (a deleted circuit-management system for a go-kart tracking app) twice, once through Spec Kit's full pipeline, once through plain iterative prompting with immediate verification. The numbers.

The execution-time gap is 7x. The review-time gap is 13.75x, larger, not smaller, which is the number that actually matters. A pipeline whose whole design goal is producing enough upfront clarity to make implementation and verification cheaper produced more markdown and more review burden for the same feature, with a bug the iterative approach didn't introduce. In the author's own words, "For now, the fastest path is still iterative prompting and review, not industrialised specification pipelines."

Where the three tools actually earned their differences

Cost isn't the only axis these tools differ on. Each ships its own mechanism for deciding whether a review finding is real, and reading all three side by side showed a spread in rigor that has nothing to do with gate model or ceremony level.

Spec Kit's review skill assigns severity by a short heuristic. A finding is Critical if it violates a stated principle or leaves a requirement with zero coverage, High if it's a duplicate or conflicting requirement, and so on down the list. Nothing in that skill checks whether the underlying claim is actually true before grading how bad it would be if it were.

OpenSpec's verification skill states its own method plainly. Search the codebase for keywords related to a requirement, then "assess if implementation likely exists." Its own words include "don't require perfect certainty." And its own stated bias under uncertainty is to downgrade. Prefer a Suggestion over a Warning, a Warning over a Critical, whenever unsure. That's a defensible default for a tool trying to avoid false alarms. It's also the opposite of what you'd want from a step whose entire job is catching real problems before they ship.

The one that verifies before it grades

BMAD-METHOD's review protocol does neither. Before a finding gets a severity, it has to be verified. Read the cited line, follow the callers and the guards upstream, until the claimed bad outcome can be confirmed or refuted. Its own stated principle is that code failing loudly on a situation you never showed it could reach is correct behavior, not a defect. Verified findings get one of five verdicts, not three. false (checked, doesn't happen, state what disproves it) and maybe-false (couldn't settle it, state what would) sit alongside high/medium/low, so an unverifiable claim doesn't get silently folded into "probably fine" the way OpenSpec's downgrade bias would treat it. Verified findings that survive route to exactly one of three outcomes. Patch it now if the fix is trivial and adds nothing new, halt and ask a human if it isn't, or log it for later if it's a pre-existing issue rather than something this change caused.

Taking the mechanism, not the tool

None of that made a case for installing BMAD-METHOD. Its own review protocol lives inside a workflow that needs two separate runtimes (Python for its rendering step, Node for its CLI wrapper), plus five agent personas and a dozen skills that have nothing to do with reviewing findings. Installing the whole framework to keep one page of classification logic would mean carrying a build pipeline, a second language runtime, and a foreign project structure into every repo that wanted the one thing actually worth having.

So we didn't install it. We read the mechanism, verified it against the source, and rewrote it as a standalone skill. Verify a claim before classifying it, keep the five verdicts including the two that record uncertainty honestly instead of hiding it, route every survivor to patch, halt, or defer instead of leaving a flat severity list someone has to re-triage by hand. A second skill does the same extraction from OpenSpec, taking not its artifact graph or its CLI but the one discipline worth keeping. When two claims about the same code disagree, don't trust whichever one is closer at hand. Search the actual code for evidence before deciding which claim is true.

That's the tradeoff, and it isn't one to resolve too cleanly. What got left behind is real. BMAD's escalation ramp for surprises mid-implementation, OpenSpec's dependency-graph resolution across a whole planning artifact set. The bet is that the two mechanisms we kept were the load-bearing part, and everything around them was the particular way each tool chose to package it.

What this changes about the wiki-and-mechanism thesis from parts 2-5

Every mechanism this series has described so far ran at execution time. .git/info/exclude keeping AI context out of a project's own repo (see part 2), the allowlist fix that rejects a shell operator before a model ever gets to decide whether to use one, the gate that only lets a persona advance once its artifact is actually committed. This audit found the same underlying principle holding one layer earlier, in the planning stage, in three tools built by three separate teams who arrived at three different answers about how much of it to apply.

It's worth naming the boundary rather than overselling the parallel. Scott Logic's own numbers, and the same complaint independently voiced in Spec Kit's own issue tracker ("it's more a tool for a product owner than for a developer"), both point at something the execution-side mechanisms in this series don't share. A planning pipeline's cost is measured in hours of human review, paid every time it runs. The allowlist fix runs in milliseconds and never asks anyone to read 4,839 lines of markdown to find out if it worked. A mechanism that's obviously worth it at execution time, where the alternative is a security hole or a silently dropped requirement, isn't automatically worth the same ceremony at planning time, where the alternative might just be a shorter conversation with the agent.

What survived the audit either way is two rules. Verify a claim before deciding how bad it is, and when two claims disagree, go read the code. Those two rules don't care whether they're running before the first line gets written or after the last one ships.

The first article in this series defined the discipline this one keeps returning to. The model provides reasoning, the harness provides everything that makes the reasoning trustworthy enough to act on. That's true one layer up from where this series started looking.

— Delaa