How do AI agents remember?

A language model has no memory of its own. Each time it answers, it reads its context windowContext WindowEverything a model can see at once while it writes an answer, counted in tokens. When the window is full, something has to be left out. from the top and responds to what is there. It does not learn from your corrections, and when a conversation ends, nothing carries over by itself.

That is fine for a quick question. It is a real problem for an AgentAgentA model that uses tools in a loop, deciding each next step from what the last one returned, until the task is done. working on a task for hours, or an assistant you use every day. So "memory" in AI is not something inside the model. It is a set of tricks in the software around it that decides what goes back into the window.

Memory inside one task

An agent's window fills up fast: every file it reads and every tool result lands in it. And a fuller window is not a smarter one. As the token count grows, the model's accuracy and recall get worse.

So long-running agents tidy up as they go.

• Compaction. Older turns are replaced with a short summary. Anthropic calls this the first lever to pull.
• Clearing. Old tool results that are no longer needed are removed.
• Fresh helpers. A sub-agent takes a focused piece of work with a clean window and returns only its conclusion.
• Fetching late. Instead of loading everything at the start, the agent keeps pointers, such as file names, and reads the content when it needs it.

Memory across tasks

What must last goes outside the window altogether. An agent can keep notes in files, a progress log or a to-do list, and read them back next time. Anthropic calls this structured note-taking, or agentic memory, and offers a memory tool built for it. Search over past documents, as in RAGRAGA way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes., is another way back in.

Chat apps do a version of this for you. With Memory on, ChatGPT can remember your preferences and details from earlier chats.

What this means for you

When an assistant "remembers" you, it is reading saved notes into the window, not recalling. That has 2 practical sides. It can only remember what it chose to save, so it may hold onto an old detail and miss a new one. And those notes are stored data: check the memory settings, see what is kept, and delete what you would not want reused.

For a long task with an agent, a short written brief or notes file that it updates as it goes beats hoping it holds everything in its head.