Agent memory’s other half is offline “dreaming,” not more notes

Anthropic’s engineering post on agent memory opens with Kierkegaard’s line about living forward and understanding backward. The product claim underneath is concrete. Claude can write notes while it works, but the notes that matter most are the ones rewritten later against the transcript, when the session is idle. The lab calls that pass “dreaming,” after Letta and UC Berkeley’s sleep-time compute idea. Salience tagging in the moment is necessary. It is not enough, because a wrong or stale note can be trusted in every later session.

What shipped

The post defines agent memory as knowledge Claude gains from experience and keeps between sessions as readable context. It excludes weight changes, ordinary in-context learning, and compaction summaries.

On Continual Learning Bench (CL-Bench) 1.0, Claude answers database questions in fresh sessions, then sees the grader’s answer and may write memory for later. Sonnet 4.6 left untested guesses. Opus 4.7 began consolidating a reference and marking unchecked claims. Sonnet 5 and Opus 5 closed the loop with validated rules and trap warnings. Opus 5.5 and Sonnet 5.5 matched Opus 5 with memory, from a slightly higher no-memory baseline.

Pokémon Red tested what to save. Sonnet 3.7 archived almost everything (413 files across 225 folders) and earned one badge by ~10,000 steps, then spent turns rewriting “master” notes instead of moving. Fable 5 kept nine files plus global.md, earned four badges, and stored strategy the screen does not reprint. Opus 5.5 kept a small goal/next-step folder and beat the game.

Steering results cut against fixed memory rules. A bare “memory folder is available” helped stronger models. Forcing “record only failures” removed much of the memory gain for Opus 5 and Opus 4.7. Older Sonnet 4.6 nearly doubled its score when told to read memory and test theories; Sonnet 5 already read memory 9/10 times without that prompt.

Dreaming is the offline half. In one Managed Agents Pokémon run, Claude wrote that the Mt. Moon PokéCenter sat on the east side. It declared the cave done, walked a one-way ledge, and looped. Dreaming checked the note against the transcript, corrected the landmark, and Claude re-entered the cave toward Cerulean. The uncorrected and no-memory runs did not clear the mountain.

Dreaming also builds wikis: linked pages over a ~350,000-file Stainless corpus (~420M tokens). Over 128 coordinated sessions, graded criteria on 25 questions rose from 13% to 48%. With search plus the wiki, scores rose from 54% to 61% as the wiki grew, versus 53% with search alone; best wiki-only score was 48%, with about 20,000 tokens saved per question. On Harvey LAB merger questions that needed multi-document judgment, wiki use scored 87% versus 73% for search. On another LAB task, a shared contract table retained 41 of 57 graded facts until a reflection loop rewrote instructions so one agent per contract quoted terms verbatim (57/57). A second dreaming pass also repaired dangling wiki links and added an index.

Dreaming can promote notes into context injection. Claude Code’s auto memory loads only the first 200 lines or 25 KB of MEMORY.md as an index. Incorrect injected notes cost tokens in every session and can cut adherence when instruction files grow long.

Evidence on the table

Primary numbers and setups come from Anthropic’s own Managed Agents runs: CL-Bench model ladder, Pokémon folder sizes and badges, Mt. Moon correction contrast, Stainless wiki scaling (2→128 sessions), LAB multi-doc and contract-fact counts, and the MEMORY.md size cap. The post cites Letta/UC Berkeley for sleep-time compute. Independent third-party replication is not in the piece.

Steelman and gaps

Steelman. Live note-taking faces a prediction problem: the agent cannot know future needs, and environment state goes stale. Offline replay against transcripts is a natural second stage, like hippocampal replay into cortex. The Pokémon and CL-Bench ladders make the salience story measurable, not only metaphorical.

Gaps: all main evals are Anthropic-run. Dreaming needs compute and transcript access; cadence and what to re-check remain open. Wikis can balloon past the source corpus (Borges map problem) or help little when answers sit in one searchable document. Injected memory can poison every session if dreaming fails. “Dreaming” here is engineered idle compute, not biological sleep.

Who should care, and what to watch

Agent-platform and harness teams should treat memory as a two-phase system: in-session write rules plus scheduled correction/compression jobs. Eval designers should score memory quality under state change, not only recall volume. Product teams shipping “persistent memory” folders without offline audit should expect silent failure modes like the Mt. Moon loop.

Watch over six months: whether public Claude products expose dreaming schedules; whether CL-Bench-style memory loops appear outside Anthropic; whether MEMORY.md-style injection budgets shrink further as models get better at on-demand search.

Close

The post’s durable claim is not that agents need more sticky notes. It is that offline dreaming’s job is to correct and compress what in-session salience got wrong or left messy, before that junk becomes every future session’s prior. Six months on, either agent stacks ship a visible sleep pass with transcript checks, or “memory” products keep amplifying first-draft notes.