Sources: DAIR.AI's LinkedIn post; paper Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents (Yukyeong Lee, Yumin Choi, Jinheon Baek, Sung Ju Hwang; KAIST and DeepAuto.ai, 2026-11-10); project page; code.
In one paragraph
Agent memory that distills past runs into short lessons can make an agent worse: it fails tasks it would have solved with no memory at all. The authors call this memory-induced degradation. The root cause is that a run carries only one label, success or failure, so whoever extracts a lesson has to guess which actions mattered. The lesson is then stored without the run, so a wrong guess can't be checked later. WikiTrace keeps three linked layers: a short insight, a compilation explaining why its source run succeeded or failed, and the raw trajectory. When an insight is vague, conflicts with another, or comes from a failed run, the agent follows the links with ordinary file tools before acting. Across eight suites in five benchmarks it averages 90.2 against 79.5 with no memory, 3.2 points above the same insights given alone. It is the only memory method tested that never scores below the no-memory agent on any suite.
Measure Gain and Regression separately
Average accuracy hides the failure mode. The paper reports Gain (tasks memory turned from failure into success) and Regression (tasks it turned from success into failure) against the majority no-memory outcome over three trials. Existing methods showed Regression comparable to, or higher than, simply rerunning the agent without memory. In tasks the agent always solved without memory but failed with it, 52% of the failures traced to a memory item the agent followed. Most often the item itself was wrong (20%); otherwise a valid item was applied outside its scope, misused, or followed while memory held a conflicting item. WikiTrace cut Regression from 6.5% to 4.5%.
Architecture
Build, per run, offline. A compiler reads the task, the agent's trajectory and its success flag. It writes a compilation of the key decisions and supporting observations, using separate templates for successful runs (what helped) and failed runs (plausible causes). It then extracts at most three insights. Each insight says what to do, when it applies, which run it came from and whether that run succeeded.
Keep the source. The raw trajectory is kept unchanged (reasoning, action and observation per turn, plus outcome). Each insight links to both its compilation and its trajectory, and each layer has an index. Runs are processed independently, so every lesson stays tied to its evidence.
Use, at test time. The agent's first message holds all insights plus the three most similar past runs and the insights drawn from each. The prompt asks the agent to descend to the source when an insight lacks detail, comes from a failed run or disagrees with another, and again whenever an action fails. Memory is mounted as read-only files that the agent searches with read and bash for the fact that settles the question, such as an exact command format.
Setup. Benchmarks: ALFWorld, ScienceWorld, PlanCraft, AppWorld (normal and challenge splits) and tau-bench Retail. Baselines: AWM, ExpeL, ReasoningBank, Insights Only. Every method builds memory once from the same no-memory training runs, with no extra validation runs. All use the same terminal agent, with deepseek-v4-flash as both agent and memory builder. Extra build budget for the baselines did not close the gap.
Findings worth carrying over
Agents consult the linked evidence more on harder tasks, and that is where WikiTrace's lead over Insights Only is largest.
The benefit comes from evidence that changes the agent's mind, for example an insight it revises or takes back after reading its source run.
Keeping the trajectory turns a lesson from an instruction into a claim with provenance that can be checked, which is cheaper than asking an LLM to validate lessons up front.
Limitations and failure modes
Compilations of failed runs are still guesses from a binary outcome. A link lets the agent check an insight, but it doesn't verify it.
All insights go into the first message, so the prompt grows with memory. A memory too large for that becomes a retrieval problem the paper leaves open.
The degradation causes and descent labels come from an LLM judge reading the agent's stated reasoning, not from causal analysis.
It uses one model family, simulated environments and an offline build. Continual memory with rewrites, and memory shared across users, are untested.
Keeping raw trajectories raises storage and privacy questions in real systems.
Review checklist for an agent memory design
Do you report Regression (tasks memory broke) alongside average gains?
Can every stored lesson be traced to the run, document or decision it came from?
Does each lesson record when it applies and whether its source succeeded?
Is the agent told when to distrust a lesson (vague, conflicting, from a failure) and how to check it?
Is memory read-only to the agent at use time, with updates going through a separate build step?
What happens when memory no longer fits in context?
Related
R002 (the five layers of agent memory): WikiTrace adds provenance links between procedural lessons and episodic records.
R036 (Claude 101 and agent-memory evals): a small injected memory layer, with the model told what memory is for.
R005 (context as a budget): the growing first message is the cost WikiTrace accepts.
R039 (reliability layers sort answers): a second metric beside accuracy exposes what the average hides.