Agent Memory

Agent memory is how an AI agent keeps its working state once a task produces more history than its context window can hold: what stays in the window, what gets written outside it, and how the right pieces come back in. The window is the only memory a model has built in, and filling it hurts anyway: Chroma's 2025 context-rot study tested 18 models and found answers grew less reliable as input length grew. Most of the work is deciding what to leave out.

Three types of agent memory, three lifespans#

Sort memory by how long it has to live. Working context is whatever is in the window now: the system prompt, recent messages, the last few tool results. It is fast and complete, and it disappears when the session ends or the window fills. A scratchpad is short-term state the agent writes outside the window during one task, such as a to-do list, a progress log or a notes file, updated through tool calling. Anthropic gives Claude Code's to-do list as the example, and its Claude-plays-Pokemon agent kept accurate tallies across thousands of game steps by keeping notes no single window could hold. Episodic and long-term memory last across sessions: summaries of past runs, learned facts, user preferences, self-critiques. Reflexion-style agents write a short critique of each failed attempt into that long-term store, so the next attempt does not repeat the last failure.

The MemGPT paper (Packer et al., 2023) made the hierarchy explicit by borrowing from operating systems: the context window plays the part of RAM, external storage plays disk, and the agent moves data between them through tool calls. Letta, the company that grew out of MemGPT, keeps the split in its current docs: memory blocks pinned in the window, a search over past messages, and archival memory, a searchable store the agent can only reach by calling a search tool. As of October 2026 its docs also describe MemFS, a git-backed store that keeps memory as markdown files, with every edit committed, so a person can read and diff what the agent has learned.

Compaction: making room without losing the thread#

When the window fills in the middle of a task, the first lever is compaction: summarize the conversation so far and continue from the summary. Claude Code compacts automatically as it nears the limit, and the work is in what the summary keeps. Anthropic's guidance is to keep architectural decisions, unresolved bugs and implementation details, and to drop redundant tool outputs, which are usually most of the tokens. As of October 2026 the Claude API offers server-side compaction in beta. A lighter variant, context editing (also beta), clears old tool results while keeping the messages around them, which frees most of the space with less risk of losing a decision.

Compaction always loses something. A summary that drops one constraint ("the client requires Python 3.9") can send the next hundred steps the wrong way. That is why compaction pairs with a scratchpad: notes written to files survive a compaction word for word, so the summary only has to carry the story, not the facts. Anthropic's memory tool puts the assumption in the system prompt it injects, telling the agent that its "context window might be reset at any moment" and that progress not written to the memory directory can be lost.

Persist or recompute#

Every piece of state forces a choice: save it, or rebuild it when needed. The trade is the same as caching anywhere. A saved debugging conclusion spares the agent from repeating every tool call that produced it. A saved note also goes stale without any warning, and an agent that trusts last week's note about a module that has since been renamed will edit the wrong file. A stale note does not fail the way a missing file does. It feeds wrong context into every later step.

Persist what is expensive to rediscover and slow to change: design decisions, user preferences, hard-won diagnoses, project conventions. Recompute what changes often or costs one tool call to refresh, such as current file contents and test results. Where you can, store a pointer rather than a value, because "auth logic lives in src/auth/" stays true far longer than a copy of the code. On our prospect intelligence engine the same split shows up in the runtime: it is cache-first, so a rerun over overlapping accounts reuses work, and source recency is stored as metadata at ingestion, because a role that changed six months ago still reads as current on the page that was scraped.

Memory is a retrieval problem#

Once memory lives outside the window, getting it back is search, the same problem RAG solves, pointed at the agent's own past instead of a document set. Small memories can be loaded whole. Claude's memory tool has the agent look in a /memories directory and read the relevant files before it starts, which Anthropic calls just-in-time retrieval. Past a few dozen notes you index them like any other documents, and Letta exposes that as an archival search tool the agent calls when it decides it needs history.

A bigger context window does not remove the need for memory, because cost and answer quality still get worse with length; it only raises the point where retrieval beats pasting everything in. An agent that reloads its full history every session pays for tokens that distract the model. On the executive operations agent we built, local retrieval and planner memory carry context between runs, so the plan for a recurring routine draws on how that routine went last time. We measured plans completed with no human intervention rising from about 55% to 80% after local memory and retrieval landed.

Which tier to build first#

Start with the smallest tier that covers the work. A single-task agent needs a scratchpad file and compaction, and current agent frameworks such as Claude Code ship both. Add memory that lasts across sessions only when the same workflow comes back, and keep it as files a person can open before moving it into a search index, because a wrong note is easy to spot in a file a person opens and hard to spot inside an index. Once more than one person uses the same agent, key memory to the acting user inside the tenant, or one person's notes reach the next person's session.