Why do AI agents lose memory between sessions?
AI agents lose memory between sessions because LLM calls are stateless, where each request is processed on its own with no record of the ones before it.
AI agents forget information between sessions because LLM calls are normally stateless: the context window holds temporary input for one request, not durable memory. Chat history, bigger windows, summaries, RAG, and native memory features can fake continuity, but each drops details or stays locked to one provider. Real cross-session memory lives outside the model.
Walrus Memory is a portable memory layer for AI agents that persists selected state and moves it across apps, sessions, and agents.
In this article
Key takeaways
- AI agents forget context because LLM calls are normally stateless. Cross-session memory has to be built outside the model.
- A context window is temporary input capacity, not durable memory.
- Chat history is an archive. Memory is selected information retrieved and reinserted into a later request.
- At the end of a session, the model's working state is discarded unless another system persists it.
AI agents lose memory between sessions because an LLM doesn't automatically carry the state of one request into following sessions. The model can only use the information supplied in its current context window. If an application doesn't store, retrieve, and reinsert information from an earlier session, the next model call starts without it. Google's API documentation describes this as stateless operation.
The assistant didn't necessarily lose a thought. The next session simply contains no record of it.
Why does ChatGPT forget previous chats?
Agents like ChatGPT forget previous chats when the product doesn't pull relevant past context into the current model request. That context can include saved memories, prior chats, files, and connected-app information.
Chat transcripts and memory are separate systems. For OpenAI models, the current memory system keeps a continually updated synthesis of context from past chats rather than a complete, verbatim record. The complete record is stored elsewhere. This makes it useful for personalization, but not a reliable archive of every prior detail.
That selectivity is deliberate: ChatGPT's memory system decides which details are most important to retain and update. Meanwhile, the Temporary Chat feature neither uses existing memories nor creates new ones, which acts as a privacy control.
Memory boundaries can also differ within the same provider. In ChatGPT today, custom agents keep their own configured instructions and uploaded knowledge, but don't use your saved memory, custom instructions, or previous conversations. New chats with the custom agent begin without the personal context from your regular ChatGPT chats.
What’s the difference between a context window and memory?
A context window is the temporary information available during a model call. Memory is an external mechanism that preserves information beyond that call and makes selected information available later.
Context window: The finite token budget a model can attend to while producing a response. It may contain system instructions, messages, retrieved documents, tool results, and output capacity. It’s not a durable account record.
AI agent memory: An external system that retains information beyond the current request or session, then retrieves selected facts, events, decisions, or procedures and places them into a future context window.
You can think of the context window like short-term memory: a larger window lets the model receive more material at once, but it does not make that material persist into a new session.
However, just because you can fit information into the window does not guarantee that the model will use it reliably. The peer-reviewed Lost in the Middle study found that performance can decline when relevant information appears in the middle of a long input.
Agent memory can be thought of like long-term memory: it is writable, user- or application-specific information that can be recovered for later work.
What happens at the end of a session?
At the end of a session, the model’s working state disappears unless another system deliberately persists it.
During inference, the model receives tokens and generates more. To avoid re-deriving the same values at every step, serving systems commonly use a key-value cache: arithmetic the system declined to repeat, not stored information about the user. Every entry is a deterministic function of tokens the user already supplied, meaning nothing is learned. Delete the cache and you lose speed, not knowledge. This holds even when the cache is deliberately preserved. A 2025 systems paper on state restoration takes stateless inference as the standard assumption and then relaxes it, writing intermediate activations to SSDs so context can be rebuilt hours later, but what persists is still a recomputable artifact of one's session's tokens, not a profile the model has formed.
Anything needed later must therefore be handled in a separate workflow. From the agent, you can resend the transcript, keep server-side conversation state, write selected information to storage, or retrieve earlier material before the next call.
Some providers offer wrappers to make these workflows programmatic. OpenAI’s Conversations API can store and retrieve conversation state across Responses API calls, while Google’s Interactions API can retain server-side state, however its stateless mode requires the client to send accumulated history.
But retained state is not necessarily permanent memory. Google documents session-handle retention of up to 24 hours in one API configuration, while OpenAI documents 30-day default application-state retention for its Responses API. These are provider retention policies, not portable long-term memory.
This is why a coding agent can regress when completing tasks like choosing a migration path. Once it processes enough logs to lose the original rationale, it can later repeat or reverse the decisions it made. They will only survive if the system preserves it and can retrieve it when relevant.
What does this mean for building AI agents?
If you're building agents, cross-session memory is something you add, not something the model provides. The model won't carry state between sessions on its own. Reliable continuity comes from a memory layer that lives outside the model: it stores selected state, retrieves it when relevant, and puts it back into the next context window.
Walrus Memory is a portable memory layer for AI agents. It keeps memory outside prompts and context windows, so it isn't tied to a single runtime or session, and it can move across agents, apps, and workflows.
FAQs
Why do AI agents forget between sessions?
AI agents forget because LLMs are normally stateless. If no external system stores and retrieves earlier information, a new request contains no durable record of the previous session.
Is a context window the same as memory?
No. A context window is the finite amount of information available during the current model call. Memory persists outside that window and must be retrieved into a later call before the model can use it.
Does a larger context window solve AI memory?
No. A larger context window increases temporary input capacity but does not create cross-session persistence. Research also shows that models may use information less reliably when it is buried within a long input.
Is chat history the same as memory?
No. Chat history is an archive of interactions. Memory is selected information retrieved and used later, and a product can maintain both as separate systems.
How does Walrus Memory stop agents forgetting between sessions?
The forgetting described in the article happens because memory has to live outside the model, and most tools keep it locked to one provider or session. Walrus Memory is a portable memory layer for AI agents that persists that state outside prompts and context windows, so continuity survives across sessions instead of resetting.
About Walrus Memory
Walrus Memory is a portable memory layer for AI agents. It enables agents to operate reliably across apps and sessions without losing context – portable, verifiable, and fully under builders' control. Walrus Memory lets agents handle complex, long-running workflows and coordinate using data they can trust.
