How Do AI Agents Store Long-Term Memory?
AI agents store long-term memory in an external system that saves selected information outside the model, then retrieves it into the context window when it's relevant.
An LLM doesn't store long-term memory on its own. Long-term memory for an agent is a separate layer: a store, plus rules for what to write, when to retrieve it, and how to place it back into the model's context. Without that layer, each session starts from zero. For why that happens, see why AI agents lose memory between sessions.
In this article
Key takeaways
- Long-term memory is built outside the model: a persistent store plus rules for writing and retrieval.
- Short-term memory is what's in the context window now. Long-term memory is what survives after the session ends.
- Agent memory is usually split into semantic, episodic, and procedural memory.
- Context stuffing, summarization, RAG, and native memory features each create continuity, and each has failure modes.
- A complete memory system also needs correction, deletion, conflict handling, access control, and provenance.
What's the difference between short-term and long-term memory in AI agents?
Short-term memory is the information actively available to an agent during its current task. Long-term memory is persisted information that can be recovered after the current context or session ends.
Short-term, or working, memory includes the current goal, recent conversation, active plan, selected files, and tool results. It's immediately usable because it's already in the context window, but it's finite, temporary, and increasingly expensive to replay as it grows.
Long-term memory combines a persisted store with rules for writing and retrieving information. It usually splits into three kinds:
- Semantic memory – stable facts and preferences, such as a user's language or deployment region.
- Episodic memory – past events, decisions, failures, and outcomes.
- Procedural memory – reusable workflows, instructions, and policies.
Storage alone isn't enough. A working memory system also needs rules for promotion, ranking, recency, conflict handling, correction, deletion, provenance, and access. The Generative Agents architecture demonstrates an early record, reflection, and retrieval pipeline for reusing stored experiences in later planning.
The short-term and long-term split is useful, but it isn't a universal taxonomy. A 2026 survey argues that modern agent systems blur the lines between memory, RAG, and context engineering.
How do today’s tools fake memory, and where does each approach break?
Today’s tools create the appearance of memory by replaying, compressing, retrieving, or extracting information outside the model.
Context stuffing
Context stuffing prepends old messages, instructions, or project material to each new request. It is simple and preserves exact recent wording.
The cost is growth. Larger prompts increase token use and generally increase time to first token. Old material must eventually be truncated, and irrelevant content can compete for attention. Even relevant information can be missed when buried in a long prompt.
Summarization
Summarization replaces older dialogue with a shorter running record. It controls prompt size and reduces the amount of text replayed.
But summaries are lossy. Details, exceptions, chronology, and exact requirements can disappear. OpenAI described an early saved-memory system as feeling like talking to someone who took notes but forgot what was not written down.
A summary also needs links back to its source material if you want an audit trail.
Retrieval-augmented generation
RAG indexes external material, finds candidate passages for a query, and adds selected passages to the prompt. It gives the model access to updateable and potentially attributable information without replaying everything.
Its failures move into the retrieval pipeline: poor chunking, embedding mismatch, weak ranking, stale indexes, missed context, latency, or passages that are retrieved but not used. The foundational RAG paper identified provenance and the updating of world knowledge as open problems.
RAG can be part of memory. It is not a complete memory system by itself. It doesn’t automatically capture an agent’s evolving decisions, user-specific state, or procedures.
Native memory features
Native memory features let a provider extract preferences, search history, synthesize past conversations, or maintain project-scoped knowledge. They reduce repetition and often provide user controls.
Their limits are scope and reliability. Recall is selective, retrieval criteria may be unclear, and memory usually remains inside one provider, assistant, or project. ChatGPT’s own documentation says it doesn’t remember every detail.
What have the big LLM providers shipped, and what remains unsolved?
The big providers have shipped useful continuity features, but none of the reviewed systems establishes universal, portable, exact, and auditable memory across assistants and apps.
- OpenAI: ChatGPT expanded memory beyond explicitly saved items to include information from chat history in 2025. In June 2026, OpenAI described a background “dreaming” architecture intended to improve continuity, freshness, and relevance. OpenAI has not presented that architecture as exact archival recall or a cross-provider memory format.
- Anthropic: Claude can search prior chats, but searches are bounded to non-project chats or an individual project. Projects keep isolated histories, instructions, and uploaded knowledge. In Claude Code, each session starts with a fresh context window, and cross-session continuity comes from external memory files: the user-authored
CLAUDE.mdinstructions and project-specific auto-memory notes that Claude Code loads into later sessions. - Google: Gemini's Memory can use past chats to personalize responses, and eligible users can turn it off or delete chats. Temporary Chats don't personalize or save context, though Google retains them up to 72 hours. These controls improve privacy, but they don't make Gemini's memory a guaranteed record of a past decision or a portable layer another assistant can use.
- Microsoft: Microsoft 365 Copilot added Memory in 2025 for preferences, working style, and recurring topics, though as of its December 2025 update it wasn't available for agents. GitHub Copilot has a separate memory feature, now in public preview, storing repository facts and user coding preferences. Both improve continuity inside Microsoft's own coding and chat surfaces, not a portable, cross-assistant memory layer.
The unsolved layer is not another chat-history toggle. It is durable state that applications and agents can use across sessions, with clear ownership, access rules, correction, deletion, and provenance.
What solves cross-session memory for AI agents?
The category-level solution is a persistent, portable memory layer that lives outside any single model. It stores selected state, retrieves it when relevant, and places it back into the model’s working context. The implementation belongs in dedicated infrastructure, not in the assumption that a larger prompt will remember forever.
Walrus Memory is a portable memory layer for AI agents that enables AI agents to operate reliably across apps and sessions, without losing context. Memory persists outside prompts and context windows and isn’t tied to a single runtime or session.
FAQs
How do AI agents store long-term memory?
They store it in an external system, not in the model. A persistent store holds selected facts, events, and procedures, and retrieval rules pull the relevant ones into the context window for a later request.
What is short-term vs long-term memory in AI agents?
Short-term memory is what's in the context window during the current task. Long-term memory is information persisted outside the window that can be recovered after the session ends.
What is semantic memory in AI?
Semantic memory is an agent's store of stable facts and preferences, such as a user's language or deployment region. It's one of three common types, alongside episodic memory (past events and outcomes) and procedural memory (reusable workflows).
Is RAG the same as long-term memory?
No. RAG retrieves external material and places it into a prompt. A complete agent-memory system also needs write rules, identity and temporal structure, correction, deletion, conflict handling, and provenance.
Does a bigger context window replace long-term memory?
No. A larger window holds more temporary input but doesn't persist anything across sessions. Long-term memory still has to live outside the model.
Does Walrus Memory give agents long-term memory?
The long-term memory described here is exactly the layer Walrus Memory provides. Walrus Memory is a portable memory layer for AI agents: it stores selected state, retrieves it when relevant, and moves it across agents, apps, and workflows, rather than staying tied to a single runtime.
About Walrus Memory
Walrus Memory is a portable memory layer for AI agents. It enables agents to operate reliably across apps and sessions without losing context – portable, verifiable, and fully under builders' control. Walrus Memory lets agents handle complex, long-running workflows and coordinate using data they can trust.
