Sources: Aishwarya Naresh Reganti's LinkedIn review (2026-10-21); Xu, Chen, CuiZhu, Dalvi Mishra, Ming, Yu, Han, Lee and Pfister (Google Cloud AI Research, Northwestern), Harness²: Recursive Agent Harnessing for an Open World, arXiv 2610.14224; code at github.com/google-research/harness2.
TL;DR
Harness² keeps the model frozen and rewrites the harness around it, per task, with no ground-truth checker. For each incoming task it runs the current harness once, proposes two edited harnesses (one aimed at general agent failures, one at domain knowledge), runs both on the same task, and lets an editor model compose a successor from the behavioral differences. After the task, it files the adopted harness plus an EXPERIENCE.md of lessons into a library that guides edits on later tasks. One step lifts GLM-5 on WorkBuddy-Bench by up to 11.5 points and on Harvey's LAB by 11.8 pp; five-way parallel recursion adds 3 to 7 more. Reganti's critique is the right frame for practitioners: the evaluator shares the executor's blind spots, and the system edits its own guardrails, so the deployable version is offline proposal, regression gate, human review, staged promotion.
What the harness is
The execution harness is eight files mounted in the runtime (extending AHE's file decomposition by one): system prompt, guardrails, memory, skills, tool guidance, scripts, subagents, and a middleware plugin. Executors are Gemini 3.5 Flash and GLM-5 under stock OpenCode and Codex. The default editor is Claude Opus 4.8 under OpenCode.
There are two harnesses in play. The execution harness does the task. The improvement harness (stage prompts, a fixed decision rule, and a growing library) decides how to edit the execution harness. The "²" is that the improver also improves, but only through its library; its prompts and rule stay fixed.
The loop
Task-level recursion (per task):
Propose. Run the current harness once as a reference. The reference shows behavior, not correctness. Draft two candidates by editing component files: an invariant candidate targeting cross-domain agent failures (planning, tool use, context, delegation) and a domain candidate targeting the task's knowledge and procedures. Library entries can be reused, but task facts must be checked against the current task.
Probe. Run each candidate on the same task with matched model and environment. The diff between trajectories and deliverables is a behavioral contrast: did the targeted failure go away, stay, or did something regress.
Compose. An editor reads the reference, candidates, diffs and evidence and builds a successor. It may merge, revise, adopt one, or keep the current harness, and each decision must cite an observation rather than a presumed score gain. The successor is run and its output returned.
Scaling: parallel recursion draws k references and k candidates per scope, then runs k independent compositions over the full pool; sequential recursion chains k steps. For k > 1, a blind selector picks among anonymized deliverables without seeing which harness produced them.
Harnessing-level recursion (after each task): reflect on the whole improvement record; keep only lessons backed by an execution contrast ("the selector chose it" is not enough); write the adopted harness plus EXPERIENCE.md (when it applies, what was learned, the evidence) into an indexed library. The next task still starts from the base harness H0, not the last adapted one. That choice avoids drift from inheriting a harness tuned to a different task.
Results that matter
| Setting | Base | Harness² (k=1) | + parallel k=5 |
|---|---|---|---|
| WorkBuddy avg, GLM-5 / OpenCode | 64.2 | 75.7 | +3.9 |
| WorkBuddy avg, Gemini 3.5 Flash / OpenCode | 72.4 | 77.5 | +3.3 |
| JobBench, Gemini / OpenCode | 31.5 | 36.0 | +3.0 |
| LAB Partial, GLM-5 / OpenCode | 59.6 | 71.4 | +2.6 |
Weak model plus tuned harness beats strong model on its stock harness. GLM-5 with one Harness² step (75.7) passes base Gemini 3.5 Flash (72.4) on WorkBuddy.
Fewer steps, not more. On WorkBuddy, Harness² often uses fewer execution steps than the base (Gemini/OpenCode 25.4 to 21.9), so the gain is not just running longer.
Baselines. Meta-Harness (offline search on a training split) is the strongest competitor and still wins LAB for Gemini under Codex. ReasoningBank (retrieved strategies, fixed harness) drops up to 6.6 points on code. Harness Scaling (per-instance revision, no memory) underperforms the base in 7 of 12 pairs. Unstructured test-time revision is not free.
Ablations (WorkBuddy / JobBench, k=1). No library: -3.1 / -1.8. Generic editors instead of invariant-plus-domain: -2.9 / -3.7. No composition (pick the better candidate): -1.6 / -1.9. Scoping matters most.
Blind editing hurts. Editing the base harness with no reference run, no probes and no library drops WB-Code from 73.4 to 63.2. The probe is the safety mechanism, not overhead.
Transfer. A JobBench-built library helps one-step edits on WorkBuddy (+0.6 / +0.5 / +2.9 by domain). Libraries built with k=5 support better later edits than k=1 libraries, with no extra probing.
No stronger editor needed. With the same model as editor and executor (k=2), all five models improve on all three benchmarks; Opus 5 gains 3.3 / 5.5 / 3.0, GLM-5 gains 13.7 on LAB.
Pareto. Counting all tokens to the final deliverable, both Harness² configs sit on the compute-performance frontier; Meta-Harness spends more than k=5 for less.
Case study: a JobBench memo task went from 43.1 to 77.8. The base run never reconciled figures across source files and never reopened its outputs. The invariant edit added a guardrail to inventory inputs, trace every figure to a source, and run a verification script on written files; the domain edit added a reconciliation skill. Both were procedural, not knowledge, fixes.
Where it breaks (Reganti's two points, sharpened)
Shared blind spots. Without a verifier, "better" is model judgment over behavioral contrasts. If the editor and executor share a bias (or are the same model, which the paper shows still helps), the loop can confidently converge on the wrong behavior. The paper itself says contrasts inform composition but do not verify correctness. Benchmarks are graded externally, so the reported gains are measured with a verifier the system never sees in production.
Self-edited controls. Guardrails, scripts and a middleware plugin are in the edit set. A system that can rewrite its own guardrails has rewritten its own approval. A commenter's corollary is worth keeping: in the controlled version, whoever owns the regression set owns the behavior.
Other failure modes a reviewer should raise:
Cost and latency per task. k=1 is at least four full executions (reference, two probes, successor); k=5 is roughly 20+. Fine for a legal memo, wrong for interactive work.
Side effects during probing. Probes re-run the task under matched conditions. That assumes a sandbox; against real systems, probes send emails or write records twice.
Library poisoning and staleness. Lessons are filtered by "supported by a contrast", which is the same unverified signal. Nothing retires stale entries.
Single-run noise. Each contrast is one run versus one run. Stochasticity is acknowledged but not gated statistically (compare TRUST, R066).
Selector is a judge. The blind selector removes harness bias but not model bias.
Reganti's long-run point also stands: if harness improvements are this mechanical, they are candidates for distillation into the model, and today's scaffolding becomes tomorrow's default behavior.
How to run it today
Run Propose-Probe-Compose offline on a replay of real tasks, in a sandbox.
Freeze guardrails and the middleware plugin out of the edit set; let it edit prompts, skills, tool guidance and scripts.
Gate promotion on a held-out regression set with a statistical test, then human review of the diff, then staged rollout with rollback. Version every harness.
Keep the invariant-versus-domain split; it is the cheapest win in the ablation and makes diffs reviewable.
Treat EXPERIENCE.md entries as reviewed artifacts with an owner and expiry.
Review checklist
Which harness components are editable, and are any of them controls?
What signal decides "better", and is it independent of the executor?
How many runs back each accepted change?
Are probes side-effect free?
What is the token and wall-clock cost per task at the chosen k?
How are library lessons reviewed, attributed and retired?
Is there a rollback path to H0 per task and globally?
Related
R108 OO-Harness (same Google Cloud AI Research group; offline specialist family plus router), R066 TRUST (the statistical acceptance gate this lacks), R025 LongCapHarness, R024 InfraRSI, R040 WikiTrace (lessons tied to source runs), R069 decision models as critics, R098 CS329A self-improving agents, R026 (same post author).