R131 · Agent architecture › Self-improvement and learning loops

Self-Improvement Memory: carry the edit log, not just the evolved harness

Read the original

self-improvementagent-harnessmemorymodel-migrationevals

Source: Yu, Xia, Pan et al. (Google Cloud AI, Google DeepMind, Penn State), "Self-Improvement Memory Enables Continual Recursive Self-Improvement", arXiv 2611.18856, November 2026; highlighted by Hao Hoang on LinkedIn (2026-11-17).

The problem

Automated harness optimization (a proposer LLM suggests edits to an agent's context, tools and control flow; a selector keeps the ones that score better) is usually studied with one fixed policy model and one benchmark. In practice both change: models get replaced by new versions, one harness serves several models, and task distributions drift. The paper calls continuing optimization across such a change Continual Recursive Self-Improvement (CRSI).

The headline failure: a harness evolved for Claude Opus 5.5 scored below the untouched seed harness when first run with Gemini 3.8 Flash, in all four domains tested (by 0.60 to 5.48 points). Ten further rounds of optimization from that harness still finished behind a from-scratch run in three of four domains. A source-evolved harness also started below the seed after a dataset change in text classification (72.92 vs 75.72). An evolved harness on its own does not explain why its edits exist, so the next optimizer cannot tell which ones still help.

The mechanism

Self-Improvement Memory (SIM) is a structured record for the optimizer, not for the task-solving agent. Edits are grouped by a recurring harness problem (a root-cause hypothesis such as premature completion). Each attempt under a problem stores:

The proposer gets the complete SIM in its system prompt every round, and the selector reads the same record when choosing among candidates. There is no retrieval and no truncation. Each proposal must cite an existing root cause or name a new one. After selection only the chosen edit is appended, with its measured result even when it regressed. Entries with identical root-cause text merge without losing history. When optimization moves to a new model or dataset, the evolved harness and its SIM travel together, and the optimizer can retain, revise, replace or remove inherited edits with their history in view.

Evaluation design

Four domains: coding (Terminal-Bench 2.1 to 4), text classification (a 3-dataset source to an 8-dataset target suite), agentic workspace (Harvey LAB to the mean of JobBench, GDPval and APEX-Agents, LLM-judged), and engineering design (EngDesign to Frontier-Eng, frozen verifiers). Evolve and held-out splits are disjoint; only evolve-set feedback reaches the proposer, selector or memory. Each run is 10 rounds with 5 candidates per round. Four conditions share budgets: Cold Start from the seed, Harness Transfer (evolved harness, no memory), Problem-Only Memory (root causes without attempts and outcomes), and Full SIM.

Results

The case study the summary post highlights: without the record, the optimizer re-tightened a completion check it had inherited, fixed deadlocks twice, and claimed the inherited check as its own first-round work. That is a concrete picture of what an optimizer does when it can't see why code exists.

Tradeoffs

Failure modes

Review checklist for a self-improving harness

Connections

See R131 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.