# We Built a Harness Before It Had a Name
Published 2026-08-19. Updated 2026-09-17.
Canonical URL https://log.delaa.dev/blog/2026-08-19-built-harness-before-it-had-name
> Four projects, four generations of evolution. The most effective AI-agent harness is the one you build from your own mistakes.
---
[The first part of this series](/blog/what-is-harness-engineering)
defined Harness Engineering as the discipline of building the
operational environment around AI models to make them reliable and
controllable. This second part is the concrete version. It is what we
actually built, empirically, over four generations of projects, before
any of the formal frameworks existed.
## The first generation started with a single CLAUDE.md
Our first attempt was a basic CLAUDE.md with project rules and
a few conventions. We then thought it would be sufficient to make the agent follow them.
As you may imagine, it did not work as expected. It did not fail catastrophically. The agent respected the rules, but
without context. One rule said never to commit directly to `main`; we
hadn't written a `→ Why:` line explaining why, or a `→ Consequence:` line explaining what breaks if the rule is violated,
so the first time a hotfix felt urgent, the agent committed straight to `main` anyway and
broke a deployment that depended on the PR review catching a missing
migration. It didn't know *why* the rule existed, so it couldn't weigh it
against the pressure of the moment, and it made the same class of mistake
more than once before we noticed the pattern.
Rules without reasoning get ignored the moment pressure shows up.
## The second generation introduced a structured wiki
We replaced the single file with a directory tree, each part of the
project's knowledge given its own place.
```
wiki/
├── index.md ← Story status, active branch, current work
├── rules.md ← Full rule reference with consequences
├── architecture.md ← Project structure and module boundaries
├── database.md ← Schema, entities, queries
├── auth.md ← Auth, roles, firewalls
├── testing.md ← Test commands, suite organization
├── reference.md ← Gotchas, configs, quick-reference tables
└── workflow.md ← Branch naming, PR process, release steps
```
Four things made the difference.
- **Single source of truth** for rules, never duplicated across files
- **`→ Consequence:` on every rule**, so the agent knows what breaks if
the rule is violated
- **Pattern routing table** (`patterns/index.md`), a registry mapping
component types to their canonical code examples, consulted before
writing the first line
- **Wiki-first discipline**, meaning the agent reads the wiki before
touching source code, inverting the natural "read the code" instinct
Two outcomes emerged. First, the agent stopped guessing project
conventions. It had an authoritative source. Second, when a new convention
was established, documenting it in one place propagated it to every future
session.
## The third generation added a shared brain and gotcha tracking
This time we moved the wiki out of any single project, into a central
location shared across all of them.
```
//
├── claude/
│ └── CLAUDE.md
├── wiki/
│ ├── rules.md
│ ├── patterns/
│ └── ...
├── context/
│ └── ...
└── memory/
└── ...
```
Each project pushed its wiki to this shared repository, making lessons from
one project visible to agents working on others. A gotcha discovered on
project A was available to project B without rediscovery.
The `.git/info/exclude` pattern was critical here. We excluded
CLAUDE.md, the wiki, and the context folder from each project's own git
repository and tracked them only on the shared brain. This kept the
codebase clean, avoided CI re-runs on documentation changes, and
let knowledge cross projects without coupling them together.
## The fourth generation split the wiki into two layers
We split it into one layer for what changes every session, and another
for what only changes when a decision actually settles.
```
.context/ ← Volatile session continuity
├── 00-index.md ← What to read for the current task
├── 01-decisions.md ← Append-only decisions log
├── 02-mechanics.md ← Current product/domain design
├── 03-architecture.md ← Current technical structure
├── 04-open-questions.md← Explicitly tracked unresolved questions
└── 05-session-log.md ← One entry per session
.wiki/ ← Stable technical reference
├── rules.md ← Rules with Why + Consequence
├── patterns/ ← Canonical code shapes by type
├── architecture.md ← Stable project structure
├── testing.md ← Test conventions and gates
├── reference.md ← Permanent gotchas and configs
├── workflow.md ← Branch/PR/release procedures
└── gotchas.md ← Hidden platform behaviors
```
The distinction matters. `.context/` is updated live during a session
and survives `/compact`. `.wiki/` changes only when a decision becomes
permanent. One is the agent's short-term memory. The other is its
long-term knowledge base.
The **decisions log** (`.context/01-decisions.md`) is the most
significant innovation. Every significant decision gets recorded the
same way, with the same four fields.
- The decision itself
- The constraint or tradeoff that forced it
- What got ruled out, and why
- The later decision that replaced this one, if any
This creates an audit trail that no formal Harness Engineering framework
has described yet. [The academic paper that first formalized this
discipline](https://arxiv.org/abs/2605.13357 "The 2026 paper that first formalized Harness Engineering as an academic discipline")
explicitly lists it as an open problem. Here it was running in production, hand-written by the agent
as decisions were made.
## How the system maps to formal Harness Engineering
Here are the five layers of a production harness, defined in Part 1,
mapped against what this wiki system actually implements.
| Layer | How the .claude/wiki/ system implements it |
|---|---|
| **Tool Orchestration** | `workflow.md` defines exact execution order: sync dev → branch → implement → gate → commit → PR. The workflow is the orchestration plan. |
| **Verification Loops** | PRE‑COMMIT GATE runs lint → typecheck → tests → build before every commit. Each failure reads the full log, not a filtered summary. |
| **Context & Memory** | Three-tier memory: `.wiki/` (semantic), `.context/` (episodic), `patterns/` (procedural). The wiki is the permanent project knowledge; the context is the session continuity thread. |
| **Guardrails** | Rules with `→ Why:` + `→ Consequence:` are the permission boundary. Dual-repo (.git/info/exclude) prevents AI files from leaking into the codebase. No agent commits or merges without explicit owner approval. |
| **Observability** | The decisions log records every choice with reasoning and rejected alternatives. Session log tracks what happened per session. Gotchas file traps platform-specific behaviors. |
### Where the system stands against the formal framework
The pattern routing table (`patterns/index.md`) is the clearest example
of something the academic literature hasn't described. It tells the
agent exactly which file to read before writing each component type,
and the agent has to check it and mark it verified in a visible
tracking block before writing the first line. No formal Harness
framework has an equivalent mechanism; the formal literature treats
"context" as a general retrieval problem instead of a routing problem
with a fixed answer per component type.
Two more gaps sit next to it. The Why/Consequence structure on every
rule gives an agent enough to reason about edge cases the rule's
author never enumerated, where standard engineering guidelines assume
a human reader who already understands the tradeoffs. And the
dual-repo architecture keeps the AI harness in a separate git boundary
from the code, so code pushes don't pollute CI with documentation
changes, and lessons from one project reach agents on another. The
formal literature doesn't address either problem at all, since it
assumes a harness is scoped to one project.
Measured against the same framework's H3 maturity level (adaptive
harness), the gaps are just as concrete. When a gate fails, nothing
traces which agent action caused it. Automated failure attribution
isn't present. An agent can still claim "done" without verification; victory
declaration detection exists only as a convention, not a mechanical check.
There's no context drift metric to measure output quality degradation
near context limits, intervention logging is manual (owner corrections
live in conversation, not a structured log), and the decisions log itself
is hand-written by the agent rather than generated from execution traces.
## What the evolution taught us
Start with consequences, not rules. A rule without a consequence is
advice, and an agent follows advice only when it aligns with convenience;
a rule with a consequence is a constraint, and the agent knows what
breaks, which changes its behavior even under pressure.
Splitting memory by lifetime, `.context/` vs `.wiki/`, was the
architectural decision that mattered most in the whole system. When
everything lives in one file, nothing is findable. When session context
and permanent reference are deliberately separated, both become more
reliable. The decisions log in particular turned out to be the single
most valuable file we ended up with, more than rules.md, more than
patterns/: it's the agent's audit trail, our review artifact, and the
cross-session memory that survives context limits and compaction.
None of this is finished. Each generation added capabilities we didn't
know we needed until the previous one broke. The fourth is better than
the first, and the fifth will be better than the fourth. Building a
harness isn't a project with a deadline for us; it's a discipline that
evolves with whatever we're working on next.
---
The two parts of this series together tell a complete story. Harness
Engineering as a formal discipline, and Harness Engineering as a
practice that emerged organically from the needs of AI-assisted
development. If the formal framework survives, it will be because
practitioners built these systems first and the academics named them
later.
---
— Delaa