Memory

A model call is stateless: the model only knows what is in the prompt for that one call. Memory is everything you put back into that prompt so the agent appears to remember — the conversation so far, notes it has written, facts fetched from a store. Manage it well and an agent can work a task that runs for days; manage it badly and it forgets what it tried an hour ago and loops.

Two kinds

• Short-term (working) memory — the current context window: the conversation, recent tool results, the task at hand. It is immediate and free to use, but bounded — it fills up, and once it does, something has to give.
• Long-term memory — a store that lives outside the context: files, a database, a vector store. The agent writes to it and reads from it on demand, so a fact survives past the context limit and across sessions. For example, a decision made on day one of a migration is useless on day three unless it was written somewhere the agent can read it back.

The whole problem is that short-term memory is finite, so the techniques are all about deciding what to keep in context and what to push out to long-term storage.

Managing a context that is always too small

1. Write important facts down, read them back when relevant. A notes file or a memory store holds what matters; the agent recalls the relevant pieces instead of carrying everything in context.
2. Compact when it grows. Summarize the running context — keep the gist, drop the verbatim transcript — so the agent keeps the thread without keeping every word.
3. Externalize state into artifacts. A plan or checklist the agent updates holds “what is done and what is left” outside the conversation, where it cannot be crowded out.

Fetching facts from long-term memory at the moment they are needed is common enough to have its own name — that is retrieval (RAG) , which is memory you read on demand.

What memory does not fix

More memory is not better. Stuffing everything back into context crowds out the room the model needs to reason and runs up the token bill, so the goal is to recall what is relevant, not everything.

There is a subtler trap: stale memory misleads. A fact written once and recalled later may no longer be true — recalled memory is “what was true when it was written,” not “what is true now.” So treat a recalled fact as a claim to check against the current state, not as ground truth, especially when it names a file, a value, or a setting that may have changed.

Where to go next

• Read from long-term memory on demand: Retrieval (RAG) .
• A plan is externalized memory: Planning .