R040 · Agent architecture › Memory and context

WikiTrace: agent memory that keeps the evidence behind each lesson

Read the original

agent-memoryself-improving-agentsagent-reliabilityevalsbenchmarks

Sources: DAIR.AI's LinkedIn post; paper Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents (Yukyeong Lee, Yumin Choi, Jinheon Baek, Sung Ju Hwang; KAIST and DeepAuto.ai, 2026-11-10); project page; code.

In one paragraph

Agent memory that distills past runs into short lessons can make an agent worse: it fails tasks it would have solved with no memory at all. The authors call this memory-induced degradation. The root cause is that a run carries only one label, success or failure, so whoever extracts a lesson has to guess which actions mattered. The lesson is then stored without the run, so a wrong guess can't be checked later. WikiTrace keeps three linked layers: a short insight, a compilation explaining why its source run succeeded or failed, and the raw trajectory. When an insight is vague, conflicts with another, or comes from a failed run, the agent follows the links with ordinary file tools before acting. Across eight suites in five benchmarks it averages 90.2 against 79.5 with no memory, 3.2 points above the same insights given alone. It is the only memory method tested that never scores below the no-memory agent on any suite.

Measure Gain and Regression separately

Average accuracy hides the failure mode. The paper reports Gain (tasks memory turned from failure into success) and Regression (tasks it turned from success into failure) against the majority no-memory outcome over three trials. Existing methods showed Regression comparable to, or higher than, simply rerunning the agent without memory. In tasks the agent always solved without memory but failed with it, 52% of the failures traced to a memory item the agent followed. Most often the item itself was wrong (20%); otherwise a valid item was applied outside its scope, misused, or followed while memory held a conflicting item. WikiTrace cut Regression from 6.5% to 4.5%.

Architecture

Findings worth carrying over

Limitations and failure modes

Review checklist for an agent memory design

Connections

See R040 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.