We Built a Harness Before It Had a Name

The first part of this series defined Harness Engineering: the discipline of building the operational environment around AI models to make them reliable and controllable. This second part examines a concrete implementation: a system built empirically over four generations of projects, before any of the formal frameworks existed.

Generation 1: The first CLAUDE.md

The first attempt was a single file. A basic CLAUDE.md with project rules, a few conventions, and the hope that the agent would follow them.

It failed. Not catastrophically. The agent respected the rules, but without context. One rule said never to commit directly to main; it had no → Consequence: line explaining why, so the first time a hotfix felt urgent, the agent committed straight to main anyway and broke a deployment that depended on the PR review catching a missing migration. It didn't know why the rule existed, so it couldn't weigh it against the pressure of the moment, and it made the same class of mistake more than once before the pattern became obvious.

The lesson: rules without reasoning are ignored under pressure.

Generation 2: The structured wiki

The second project introduced a structured wiki:

wiki/
├── index.md           ← Story status, active branch, current work
├── rules.md           ← Full rule reference with consequences
├── architecture.md    ← Project structure and module boundaries
├── database.md        ← Schema, entities, queries
├── auth.md            ← Auth, roles, firewalls
├── testing.md         ← Test commands, suite organization
├── reference.md       ← Gotchas, configs, quick-reference tables
└── workflow.md        ← Branch naming, PR process, release steps

Key innovations:

• Single source of truth for rules, never duplicated across files
• → Consequence: on every rule: the agent knows what breaks if the rule is violated
• Pattern routing table (patterns/index.md): a registry mapping component types to their canonical code examples, consulted before writing the first line
• Wiki-first discipline: agent must read the wiki before touching source code, inverting the natural "read the code" instinct

Two real outcomes emerged. First, the agent stopped guessing project conventions. It had an authoritative source. Second, when a new convention was established, documenting it in one place propagated it to every future session.

Generation 3: Shared brain and gotcha tracking

The third project added cross-project knowledge sharing. The wiki moved to a central location shared across projects:

<shared-location>/<project>/
├── claude/
│   └── CLAUDE.md
├── wiki/
│   ├── rules.md
│   ├── patterns/
│   └── ...
├── context/
│   └── ...
└── memory/
    └── ...

Each project pushed its wiki to this shared repository, making lessons from one project visible to agents working on others. A gotcha discovered on project A was available to project B without rediscovery.

The .git/info/exclude pattern was critical: CLAUDE.md, the wiki, and the context folder were excluded from each project's own git repository and tracked only on the shared brain. This kept the codebase clean, avoided CI re-runs on documentation changes, and enabled cross-project knowledge sharing without coupling.

Generation 4: The dual-layer system

The fourth project split the wiki into two distinct layers:

.context/              ← Volatile session continuity
├── 00-index.md        ← What to read for the current task
├── 01-decisions.md    ← Append-only decisions log
├── 02-mechanics.md    ← Current product/domain design
├── 03-architecture.md ← Current technical structure
├── 04-open-questions.md← Explicitly tracked unresolved questions
└── 05-session-log.md  ← One entry per session

.wiki/                 ← Stable technical reference
├── rules.md           ← Rules with Why + Consequence
├── patterns/          ← Canonical code shapes by type
├── architecture.md    ← Stable project structure
├── testing.md         ← Test conventions and gates
├── reference.md       ← Permanent gotchas and configs
├── workflow.md        ← Branch/PR/release procedures
└── gotchas.md         ← Hidden platform behaviors

The distinction is important: .context/ is updated live during a session and survives /compact. .wiki/ changes only when a decision becomes permanent. One is the agent's short-term memory. The other is its long-term knowledge base.

The decisions log (.context/01-decisions.md) is the most significant innovation. Every significant decision is recorded with:

• What was decided
• Why (the reasoning)
• What was rejected (alternatives considered)
• When it was superseded (pointer to the new decision)

This creates an audit trail that no formal Harness Engineering framework has described yet. The arXiv paper (2605.13357) explicitly lists it as an open problem. Here it was running in production, hand-written by the agent as decisions were made.

How the system maps to formal Harness Engineering

The five layers of a production harness (defined in Part 1):

Where the system stands against the formal framework

Three things it does that the academic literature has not described. The pattern routing table (patterns/index.md) tells the agent exactly which file to read before writing each component type, and the agent has to check it and mark it verified in a visible tracking block before writing the first line — no formal Harness framework has an equivalent mechanism, treating "context" as a general retrieval problem instead. The Why/Consequence structure on every rule gives three parts — what to do, why it matters, what breaks if ignored — specific enough that an agent can reason about edge cases the rule's author never enumerated, where standard engineering guidelines assume a human reader who already understands the tradeoffs. And the dual-repo architecture keeps the AI harness in a separate git boundary from the code, so code pushes don't pollute CI with documentation changes and lessons from one project reach agents on another — something the formal literature doesn't address at all, since it assumes a harness is per-project.

Measured against the same framework's H3 maturity level (adaptive harness), the gaps are just as concrete. When a gate fails, nothing traces which agent action caused it — automated failure attribution isn't present. An agent can still claim "done" without verification; victory declaration detection exists only as a convention, not a mechanical check. There's no context drift metric to measure output quality degradation near context limits, intervention logging is manual (owner corrections live in conversation, not a structured log), and the decisions log itself is hand-written by the agent rather than generated from execution traces.

What the evolution taught us

Start with consequences, not rules. A rule without a consequence is advice, and an agent follows advice only when it aligns with convenience; a rule with a consequence is a constraint, and the agent knows what breaks, which changes its behavior even under pressure.

The .context/ vs .wiki/ split, separating memory by lifetime, was the most impactful architectural decision in the whole system. When everything lives in one file, nothing is findable. When session context and permanent reference are deliberately separated, both become more reliable — and the decisions log in particular became the single most valuable file, more than rules.md, more than patterns/, serving as the agent's audit trail, the owner's review artifact, and the cross-session memory that survives context limits and compaction.

None of this is finished. Each generation added capabilities the previous one didn't know it needed — the fourth is better than the first, and the fifth will be better than the fourth. Building a harness isn't a project with a deadline; it's a discipline that evolves with the projects it supports.

The two parts of this series together tell a complete story: Harness Engineering as a formal discipline, and Harness Engineering as a practice that emerged organically from the real needs of AI-assisted development. If the formal framework survives, it will be because practitioners built these systems first and the academics named them later.

— Delaa