R057 · Agent architecture › Memory and context

SoMem: agents organizing their own memory as linked documents

Read the original

agent-memorycontext-engineeringknowledge-managementretrievalself-improving-agentsevals

· Source: Elvis Saravia (DAIR.AI) on LinkedIn, on Fu, Qi et al., "Can LLM Agents Organize Their Own Memory?", City University of Hong Kong and Tencent Jarvis Lab, arXiv 2611.11929 (2026-11-09). Code: github.com/Qi202/SoMem.

TL;DR

Most agent memory systems fix a schema up front (record types, grouping rules, fact/episode/profile buckets). SoMem drops the schema. The agent writes and revises ordinary Markdown topic documents, every paragraph footnoted back to the raw interaction it came from, and a deterministic runtime validates each edit and builds time, link and recency views on top. The design rule is "the model owns the meaning, code owns the consistency." On LoCoMo, BEAM-100K and LongMemEval-S it beats the strongest baseline by 2.0 to 3.1 points, and memory written by one model is readable by another. The catch is cost (about 34x Mem0 on a full LoCoMo run) and that the runtime guarantees structure and provenance, not that a summary still says what its source said.

The practical takeaway in the post: if your agent already reads and writes files, try letting it keep memory as linked notes before you add a vector store.

Architecture

Three stores, one validator, three derived views.

LayerWho writes itWhat it holds
SourceRuntime, append-onlyRaw interactions, immutable. The evidence.
TopicThe agentMarkdown docs it names, splits, merges and regroups. Each paragraph is a memory unit: stable ID, text, links to other paragraphs, footnote annotations with an event time (year/month/day precision as the source allows) and references to Source records.
CoreThe agent, size-cappedStanding preferences, requirements and frequently needed background. Injected into every query like a system prompt.

Validated edits. The agent stages every edit. The runtime checks that Source is untouched, the document structure parses, every evidence reference and link resolves, times are valid, paths are allowed, and Core is under its cap. Accepted edits are normalized, links broken by moves are repaired, and views are rebuilt. Rejected edits leave the previous state unchanged, so memory is transactional at the edit level.

Derived views, built by code.

Maintenance. Periodic passes revisit Topic files changed since the last pass; an optional final pass reorganizes the whole collection. An optional probe asks a factual question about each new interaction and triggers a targeted repair if memory can't answer it, which is a cheap write-time recall test.

Reading. The agent starts with Core, Recent, a file inventory and the question date, then uses read-only tools (BM25, embedding search, list, read with line ranges; or a read-only shell with ls/find/rg/grep/cat/sed). There is no fixed retrieval pipeline. The agent searches iteratively and drops to Source when it needs to verify. Turn caps: 20 on LoCoMo, 50 elsewhere.

Results

GPT-6 Luna writes and reads, no reasoning effort; GPT-4o-mini judges at temperature 0.

BenchmarkSoMemNext bestFull context
LoCoMo accuracy (1,540 Qs)88.96E-mem 86.3084.22
BEAM-100K nugget score (400)54.02Full context 50.9250.92
BEAM-100K pass rate (400)61.50A-Mem 55.7555.25
LongMemEval-S accuracy (500)76.50LightMem 74.5065.80

What the ablations actually say

Run on frozen LoCoMo memory, changing only what the reader can access:

Read plainly: the win comes from keeping the raw log and letting an agent search it iteratively, with agent-written topic notes as a modest index on top. The views are nice-to-haves on this benchmark.

Cost

This is an interactive-latency, frontier-model-budget design. It fits assistants with a few long-lived users, not high-QPS personalization.

Tradeoffs and failure modes

Review checklist for an evidence-linked memory system

  1. Is the raw log append-only and kept, not discarded after summarization? (The ablation says this is the biggest single factor.)

  2. Does every derived statement carry a resolvable pointer to its evidence, and does the runtime reject edits that break it?

  3. Are edits staged and atomic, with the previous state kept on failure?

  4. Is there any semantic check of summary against source (periodic re-verification, write-time probe questions, or sampling with a judge)?

  5. Are contradictions recorded as such, with both times, rather than overwritten?

  6. Is the always-loaded layer (Core) size-capped and reviewed for injected instructions?

  7. Can the reader verify by dropping to source, and does it within a turn budget?

  8. Are cost and latency measured per write and per read, with cache hit rate?

  9. Is memory model-portable, so writer and reader can be swapped?

  10. Are results reported per question type (temporal, multi-hop, abstention, contradiction), not just overall?

Sources

Connections

See R057 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.