Agent memory

What is agent memory?

Agent memory is everything an AI agent deliberately keeps between sessions, as opposed to whatever happens to be sitting in its context window right now. Close the chat, come back tomorrow, and the context window is empty. Memory is the part that survives.

The two are easy to confuse. A context window of a couple of hundred thousand tokens feels like memory while you are inside one long conversation. It is not. It is a workspace that gets wiped when the session ends. Memory is a decision somebody made about what was worth writing down.

Anthropic's memory tool puts the idea plainly: the model can create, read, update and delete files that persist between sessions, building up knowledge over time without keeping everything in the context window. Those files sit in a directory your own application controls, and the model only requests operations on them.

Short-term and long-term memory

Short-term memory is the conversation you are currently in. LangGraph calls this thread-scoped memory and describes a thread as something that organises multiple interactions in a session, the way email groups messages into one conversation. The state is written to a database by a checkpointer, so the thread can be resumed later.

Long-term memory is what crosses those boundaries. LangGraph's docs describe it as data stored across sessions and shared across conversational threads. The sharing is the entire point, and it is also where the risk lives: something learned in a Tuesday conversation surfaces in an unrelated one a month later.

Underneath all of it the model remembers nothing. OpenAI states it directly, that each text generation request is independent and stateless. Whatever looks like continuity, whether you resend the previous messages yourself or chain them with previous_response_id, is your application putting text back in front of a model that has already forgotten the lot.

How agent memory is built

Three implementations cover most of what you will meet in practice.

• A memory file. The agent reads a file at the start of a task and writes back to it as it learns. Anthropic's tool works this way, with a /memories path that your handler maps onto real storage. The appeal is that it stays readable: you can open the file, see exactly what the assistant believes about a customer, and delete a line you disagree with.

• A vector store. Facts are stored as embeddings and pulled back by meaning rather than by exact wording. This is the same machinery as a vector database used for retrieval, so teams that already run RAG tend to reach for it first.

• A structured profile. A fixed set of fields, held in an ordinary database table: preferred language, account manager, invoicing cycle, tools in use. Boring, easy to audit, and hard for the agent to corrupt because it can only fill in slots you defined.

LangGraph splits what gets stored into three kinds that are worth keeping apart in your own design: semantic memory for facts and concepts, episodic memory for past events and actions, and procedural memory for the rules used to perform tasks. "This client invoices monthly" is a fact. "We migrated their reporting in March" is an event. "Always send them a summary before the numbers" is a rule.

There is also a timing choice. Memories can be written during the run itself, which makes them available immediately, or in a background task afterwards, which keeps the user waiting less and separates memory management from the application logic.

Agent memory compared to RAG

Both put text in front of a model that was not there a second ago, so the confusion is fair. The difference is who wrote the material.

RAG retrieves from a corpus somebody curated on purpose: your policy documents, product sheets, a knowledge base. The agent reads it and does not change it. Memory retrieves from a corpus the agent wrote itself, about you, without anyone approving each line. That is why memory needs a review path and a retrieval index usually does not.

Microsoft's Agent Framework treats these as separate concerns too. Retrieval is handled by a context provider that searches a store, while conversation state lives in a session object that you serialise and resume.

What does not belong in agent memory

Memory turns an assistant into a system that stores personal data, which drags GDPR straight into a feature most teams ship as a convenience. Article 5(1)(c) requires personal data to be adequate, relevant and limited to what is necessary for the purpose. Article 5(1)(e) requires it to be kept in identifiable form no longer than that purpose needs.

Read those two next to a memory file that has been quietly accumulating notes on a client contact for eighteen months and the tension is obvious. Practical consequences:

• No credentials, tokens or connection strings. The model will usually refuse, but the guarantee has to come from validation in your handler, not from the model's good manners.

• No figures that move. Revenue, headcount and open ticket counts belong in the source system that owns them, not in a note the agent will still be quoting next quarter.

• Nothing you would not show the person it is about. That is the honest test, and it is stricter than any policy you will write.

You also need a way to delete. If a customer asks what an assistant has stored about them, "it is in the vectors somewhere" is not an answer.

What to watch out for with agent memory

Memory goes stale and nothing tells you. An agent that noted a contact person in January will keep addressing emails to someone who left in March. Nothing in the loop re-checks a written fact, so old entries are quoted with the same confidence as fresh ones. Date every entry and expire the ones nobody has touched.

A bad session poisons the good ones. Memory is a write surface, so a prompt injection that lands once does not end with that conversation. Instructions written into a memory file get read back on every future run, which turns a one-off exploit into a standing one. Treat anything the agent wrote as untrusted input on the way back in.

The privacy question is the one people skip. An assistant that remembers your customers is doing something your customers did not agree to unless somebody told them. Decide what gets kept, say so in the privacy notice, and give people a way to read and clear it. Anthropic's own guidance points the same direction, recommending that you cap file sizes and periodically delete memory files that have not been accessed in a long time.