We Built a Harness Before It Had a Name

The first part of this series defined Harness Engineering as the discipline of building the operational environment around AI models to make them reliable and controllable. This second part is the concrete version. It is what we actually built, empirically, over four generations of projects, before any of the formal frameworks existed.

The first generation started with a single CLAUDE.md

Our first attempt was a basic CLAUDE.md with project rules and a few conventions. We then thought it would be sufficient to make the agent follow them.

As you may imagine, it did not work as expected. It did not fail catastrophically. The agent respected the rules, but without context. One rule said never to commit directly to main; we hadn't written a → Why: line explaining why, or a → Consequence: line explaining what breaks if the rule is violated, so the first time a hotfix felt urgent, the agent committed straight to main anyway and broke a deployment that depended on the PR review catching a missing migration. It didn't know why the rule existed, so it couldn't weigh it against the pressure of the moment, and it made the same class of mistake more than once before we noticed the pattern.

Rules without reasoning get ignored the moment pressure shows up.

The second generation introduced a structured wiki

We replaced the single file with a directory tree, each part of the project's knowledge given its own place.

wiki/
├── index.md           ← Story status, active branch, current work
├── rules.md           ← Full rule reference with consequences
├── architecture.md    ← Project structure and module boundaries
├── database.md        ← Schema, entities, queries
├── auth.md            ← Auth, roles, firewalls
├── testing.md         ← Test commands, suite organization
├── reference.md       ← Gotchas, configs, quick-reference tables
└── workflow.md        ← Branch naming, PR process, release steps

Four things made the difference.

• Single source of truth for rules, never duplicated across files
• → Consequence: on every rule, so the agent knows what breaks if the rule is violated
• Pattern routing table (patterns/index.md), a registry mapping component types to their canonical code examples, consulted before writing the first line
• Wiki-first discipline, meaning the agent reads the wiki before touching source code, inverting the natural "read the code" instinct

Two outcomes emerged. First, the agent stopped guessing project conventions. It had an authoritative source. Second, when a new convention was established, documenting it in one place propagated it to every future session.

The third generation added a shared brain and gotcha tracking

This time we moved the wiki out of any single project, into a central location shared across all of them.

<shared-location>/<project>/
├── claude/
│   └── CLAUDE.md
├── wiki/
│   ├── rules.md
│   ├── patterns/
│   └── ...
├── context/
│   └── ...
└── memory/
    └── ...

Each project pushed its wiki to this shared repository, making lessons from one project visible to agents working on others. A gotcha discovered on project A was available to project B without rediscovery.

The .git/info/exclude pattern was critical here. We excluded CLAUDE.md, the wiki, and the context folder from each project's own git repository and tracked them only on the shared brain. This kept the codebase clean, avoided CI re-runs on documentation changes, and let knowledge cross projects without coupling them together.

The fourth generation split the wiki into two layers

We split it into one layer for what changes every session, and another for what only changes when a decision actually settles.

.context/              ← Volatile session continuity
├── 00-index.md        ← What to read for the current task
├── 01-decisions.md    ← Append-only decisions log
├── 02-mechanics.md    ← Current product/domain design
├── 03-architecture.md ← Current technical structure
├── 04-open-questions.md← Explicitly tracked unresolved questions
└── 05-session-log.md  ← One entry per session

.wiki/                 ← Stable technical reference
├── rules.md           ← Rules with Why + Consequence
├── patterns/          ← Canonical code shapes by type
├── architecture.md    ← Stable project structure
├── testing.md         ← Test conventions and gates
├── reference.md       ← Permanent gotchas and configs
├── workflow.md        ← Branch/PR/release procedures
└── gotchas.md         ← Hidden platform behaviors

The distinction matters. .context/ is updated live during a session and survives /compact. .wiki/ changes only when a decision becomes permanent. One is the agent's short-term memory. The other is its long-term knowledge base.

The decisions log (.context/01-decisions.md) is the most significant innovation. Every significant decision gets recorded the same way, with the same four fields.

• The decision itself
• The constraint or tradeoff that forced it
• What got ruled out, and why
• The later decision that replaced this one, if any

This creates an audit trail that no formal Harness Engineering framework has described yet. The academic paper that first formalized this discipline explicitly lists it as an open problem. Here it was running in production, hand-written by the agent as decisions were made.

How the system maps to formal Harness Engineering

Here are the five layers of a production harness, defined in Part 1, mapped against what this wiki system actually implements.

Where the system stands against the formal framework

The pattern routing table (patterns/index.md) is the clearest example of something the academic literature hasn't described. It tells the agent exactly which file to read before writing each component type, and the agent has to check it and mark it verified in a visible tracking block before writing the first line. No formal Harness framework has an equivalent mechanism; the formal literature treats "context" as a general retrieval problem instead of a routing problem with a fixed answer per component type.

Two more gaps sit next to it. The Why/Consequence structure on every rule gives an agent enough to reason about edge cases the rule's author never enumerated, where standard engineering guidelines assume a human reader who already understands the tradeoffs. And the dual-repo architecture keeps the AI harness in a separate git boundary from the code, so code pushes don't pollute CI with documentation changes, and lessons from one project reach agents on another. The formal literature doesn't address either problem at all, since it assumes a harness is scoped to one project.

Measured against the same framework's H3 maturity level (adaptive harness), the gaps are just as concrete. When a gate fails, nothing traces which agent action caused it. Automated failure attribution isn't present. An agent can still claim "done" without verification; victory declaration detection exists only as a convention, not a mechanical check. There's no context drift metric to measure output quality degradation near context limits, intervention logging is manual (owner corrections live in conversation, not a structured log), and the decisions log itself is hand-written by the agent rather than generated from execution traces.

What the evolution taught us

Start with consequences, not rules. A rule without a consequence is advice, and an agent follows advice only when it aligns with convenience; a rule with a consequence is a constraint, and the agent knows what breaks, which changes its behavior even under pressure.

Splitting memory by lifetime, .context/ vs .wiki/, was the architectural decision that mattered most in the whole system. When everything lives in one file, nothing is findable. When session context and permanent reference are deliberately separated, both become more reliable. The decisions log in particular turned out to be the single most valuable file we ended up with, more than rules.md, more than patterns/: it's the agent's audit trail, our review artifact, and the cross-session memory that survives context limits and compaction.

None of this is finished. Each generation added capabilities we didn't know we needed until the previous one broke. The fourth is better than the first, and the fifth will be better than the fourth. Building a harness isn't a project with a deadline for us; it's a discipline that evolves with whatever we're working on next.

The two parts of this series together tell a complete story. Harness Engineering as a formal discipline, and Harness Engineering as a practice that emerged organically from the needs of AI-assisted development. If the formal framework survives, it will be because practitioners built these systems first and the academics named them later.

— Delaa