R054 · Agent principles and patterns › Harness building blocks

Harness engineering: a source-code study of eleven coding agents

Read the original

agent-harnessagent-architectureretrievalagent-protocolscontext-engineeringcompetitive-analysis

Source and takeaway

Eleven production coding agents, about 4M lines between them, all write their own loop: none imports a general agent framework and none uses embeddings to search code. That is the core finding of Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents (Barbaste, Darrigol, Vu and Wiltberger; arXiv 2609.00006, cs.SE, 83 pages), the expanded second edition of an April 2026 study.

It reached us through a LinkedIn post by Dinesh Reddy (about 2026-11-10; 59 reactions, 33 comments), which compressed it to one line: the framework isn't the product, the harness is. The post cites the paper by arXiv number only and links nothing.

For a staff engineer the useful part is not the slogan. It is the paper's map of seven harness subsystems, the smallest and largest version of each it found, and 18 concrete recommendations. Read it as a design survey from source code, not as a benchmark: it measures nothing at runtime.

What the study did

The authors read the source of eleven coding harnesses and dissected each along the same seven subsystems. The result is 13 cross-cutting observations, 29 recurring design patterns and 18 recommendations. The corpus grew from eight systems in the April edition, and the retained April snapshots give a 90-day before-and-after diff of the same codebases.

The eleven: Claude Code, Codex CLI, Gemini CLI, Mistral Vibe, OpenHands, Aider, Mini-SWE-Agent, Hermes, Pi, OpenCode and OpenClaw. Omnigent, a meta-harness that coordinates other harnesses, is a contrast case outside the eleven.

SubsystemWhat it coversNotable range in the corpus
LoopThe turn cycle: call the model, run tools, decide to stopMini-SWE-Agent's ~100-line loop up to multi-mode loops with plan mode, hooks and recovery
LLM integrationProvider clients, streaming, retries, model routingSingle provider up to many providers behind one interface
ToolsBuilt-in tools, how they load, how results returnA bash tool alone up to on-demand tool loading past ~15 tools
Memory and contextWhat enters the window and how it is foundripgrep, glob, tree-sitter and Markdown context files everywhere; embeddings only for chat memory (OpenClaw)
SafetyApproval modes, sandboxing, policyFull OS sandbox plus policy-as-code down to none (Hermes and OpenCode are large but unsandboxed)
OrchestrationSub-agents, parallelism, hosting other harnessesSub-agents mostly in-process; ACP lets one harness host another
ExtensibilitySkills, MCP, hooks, plugins, SDKsSkills in 9/11, MCP in 8/11, ACP in 6; Pi rejects MCP

Key findings

The recommendations, condensed

The paper lists 18; these are the ones with a concrete design consequence.

  1. Start with a while loop and a bash tool; add subsystems only when a failure demands them.

  2. Load tools on demand once you pass about 15.

  3. No retrieval-augmented search over code unless it beats ripgrep plus tree-sitter on a held-out eval.

  4. Give developer tools three approval modes: PLAN, DEFAULT, YOLO.

  5. For enterprise use, an OS sandbox plus policy-as-code.

  6. Stay single-agent until parallelism clearly pays.

  7. Ship an ACP server so other harnesses can host yours.

  8. Skills before MCP.

  9. No LangChain-style runtimes.

  10. Cheap caps on stuck loops.

The paper closes with a 90-line minimal harness in Python as the starting scaffold.

Tradeoffs and failure modes

The grep-first loop is the right default, but it has known limits, and the post's own comment thread names them.

Caveats

Review checklist for a harness

Sources: arXiv 2609.00006; Dinesh Reddy's LinkedIn post.

Connections

See R054 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.