What Is Context Engineering?
Context engineering is the design and management of information a model sees at runtime, including instructions, state, tools, retrieval, and recalled memory.
Context engineering decides what a model sees at each step: instructions, history, tools, state, and recalled memory in one working context. This post covers what it is and how it differs from prompting. That includes what goes into an agent's context, where it sits in agentic infrastructure, and where it stops. Context engineering assembles that working context, but the durable state it draws on lives in a separate layer, like Walrus Memory.
In this article
Key takeaways
- Context engineering is the assembling of information a model needs for its next inference step.
- It manages more than prompt engineering does: retrieval, history, tools, state, recalled memory, and context size.
- Memory may be managed by a context-engineering framework, but durable state remains architecturally distinct from the temporary context view.
- Memory persistence, portability, verifiability, and control are not automatic outcomes of context engineering.
Context engineering is the practice of building the right working context for each model call.
Context engineering: the runtime design, selection, transformation, and management of the information a model sees for a specific inference step. It includes choosing what to retrieve, retain, compress, omit, or isolate so the model receives a small, relevant working set instead of every piece of available data.
The term does not yet have one universally accepted boundary. Anthropic defines it as strategies for curating and maintaining an optimal set of tokens during inference. LangChain describes filling the context window with the right information at each step of an agent’s trajectory. IBM emphasizes the deliberate design and optimization of context for accurate and relevant output.
These definitions agree on three points:
- Context engineering is broader than prompt wording.
- It happens repeatedly during an agent’s run.
- It works within a finite context window, even when that window is large.
They disagree at the edges. The Prompt Engineering Guide treats context engineering partly as a broader rebranding of prompt engineering. Anthropic calls it a natural progression. Gartner frames it as an enterprise capability covering data, workflows, environments, and persistent context. That is a disagreement about category boundaries, not the underlying mechanics.
How is context engineering different from prompt engineering?
Prompt engineering designs instructions, while context engineering designs the model’s full working situation and information supply chain.
Prompt engineering focuses on phrasing, roles, examples, output constraints, and the instructions given to a model. Context engineering includes all of that, plus:
- Which history should be included
- What external information should be retrieved
- Which tools the agent can use
- How tool results should be represented
- What task state should remain active
- Which memories should be recalled
- What should be summarized, trimmed, or isolated
Anthropic distinguishes the two by describing prompt engineering as methods for writing and organizing instructions, while context engineering manages all relevant tokens during inference, including information that arrives outside the prompt.
IBM similarly treats prompting and RAG as narrower techniques. A prompt doesn’t cover all inference-time information, while RAG doesn’t manage system instructions, tool definitions, message history, or task state.
The shift in language reflects the move from single-turn model calls to agents that work across multiple steps. The wording of the initial instruction still matters. It’s no longer the whole job.
What goes into an agent’s context?
An agent’s context contains the smallest useful set of instructions, evidence, state, tools, and memory needed for its next step.
| Component | What it contributes |
|---|---|
| Instructions and examples | Goals, rules, output schemas, tool guidance, and canonical demonstrations |
| Conversation history | Recent messages and relevant earlier exchanges |
| Retrieved information | Records selected from files, databases, search, or other knowledge sources |
| Tools and tool results | Available actions and the information returned by previous actions |
| Working state | The current plan, task status, intermediate results, and unresolved questions |
| Recalled long-term memory | Selected facts, preferences, decisions, and procedures persisted beyond the active session |
| Control information | Permissions, scope, routing, and information that must be isolated from the call |
This list is broader than “everything the model might need.” More context is not always better. Anthropic recommends finding the smallest high-signal set of tokens likely to produce the desired outcome. Google’s production guidance warns that appending everything creates rising cost and latency, stale or distracting context, and eventual window overflow. Anthropic emphasizes compaction, structured note-taking, and multi-agent architectures for long-running work. A Survey of Context Engineering for LLMs groups the field into context retrieval or generation, processing, and management.
These are different taxonomies for the same practical problem: deciding what the model should see at a given moment.
What is agentic infrastructure, and where does context engineering fit?
Agentic infrastructure is the set of runtime services and controls that let an LLM-based agent execute multi-step work reliably.
Agentic infrastructure: the models, runtime, orchestration, tools, connectors, retrieval systems, state and memory services, identity controls, observability, evaluations, and interfaces that support an AI agent while it plans, acts, and updates its state.
This is an emerging umbrella term, not a standards-defined architecture. Google’s current architecture guide lists frontends, agent-development frameworks, tools, memory, agent runtimes, AI models, and model runtimes as production components. It also identifies externalized memory and state management as requirements for production systems.
Context engineering sits between the system’s information sources and a particular model call:
- Durable and transient sources hold files, records, memory, task state, tool results, and policy.
- Context engineering retrieves, ranks, transforms, compresses, and scopes that information.
- The resulting working context is sent to the model.
- The model responds or chooses a tool.
- Orchestration updates state and decides what happens next.
Google’s ADK architecture calls the resulting context a “compiled view” over a larger stateful system. Sessions, memory, and files remain the underlying sources. The context compiler produces the temporary view used for one invocation.
Where does context engineering end and memory begin?
That compiled view is where context engineering ends: it builds the working context for one model call and can't hold anything past it, so the durable state an agent needs next session, or in another app, belongs to a separate memory layer.
Leading architectures draw this line explicitly. Google ADK separates sessions, memory, and files from the per-invocation context view. Anthropic describes structured note-taking as writing notes outside the context window and retrieving them later. The split is clean: context engineering is the compiler and traffic controller for the working context, and memory is the durable substrate it writes to and reads from.
Context engineering's own techniques don't close the gap. Replay and compaction manage the view, they don't persist it. Google warns that append-everything approaches raise cost and time to first token, bury the signal in stale tokens, and eventually overflow even large windows, while Anthropic notes that aggressive summaries drop details that matter later. Once the session ends, so does the state. Across a fleet of agents that cost compounds into agentic memory debt, the accumulating cost of agents relearning what they should have retained. For the failure modes in detail, see why AI agents lose memory between sessions.
What kind of memory layer does context engineering need?
A temporary context view and a durable record have different lifecycles: one is rebuilt on every call, the other has to outlast the runtime. So the properties of that durable layer decide how much the agent can rely on it. Four properties do the heavy lifting:
- Persistent: state survives beyond the current call and session.
- Portable: memory moves across runtimes, model providers, and apps instead of locking to one state format.
- Verifiable: writes carry provenance and an audit trail, so a recalled fact can be traced and checked.
- Controlled: access, correction, and deletion are yours to define.
A memory layer that's strong on retrieval but thin on portability, provenance, or control just pushes those problems back onto the builder. Walrus Memory is one example that meets all four: a portable memory layer for AI agents that persists outside the context window, moves across apps and runtimes, and stays under your control.
FAQs
What is context engineering in simple terms?
Context engineering is deciding what an AI model should see for its next step. It assembles instructions, history, retrieved information, tools, task state, and selected memory into a useful working context.
What is agentic context engineering?
Agentic context engineering is context engineering applied across an agent's multi-step run, assembling instructions, state, tools, retrieval, and memory fresh at each step rather than once in a single prompt.
Is context engineering the same as prompt engineering?
No. Prompt engineering focuses on writing and structuring instructions. Context engineering manages the model's full inference-time situation, including retrieval, history, state, tools, memory, compression, and isolation.
Does context engineering include memory?
Partly. Context engineering decides which memories to recall into the working context, but the durable store is a separate layer with its own persistence, governance, and retrieval. Many frameworks manage the recall; few own the durable memory. One framework can implement both.
What is agentic infrastructure?
Agentic infrastructure is the collection of runtime systems that lets an AI agent perform multi-step work. It includes models, orchestration, tools, retrieval, memory, state, security, observability, evaluations, and interfaces.
Can a larger context window replace persistent memory?
No. A larger window delays overflow but doesn't create cross-session persistence or guarantee relevance and reliable recall. Append-everything approaches also increase cost, latency, and distracting context.
How does context engineering relate to Walrus Memory?
Context engineering decides what a model sees for its current step, but the durable state it draws on has to live in a separate layer. Walrus Memory is a portable memory layer for AI agents, one example of that layer, which context engineering can write to and recall from across apps and sessions.
About Walrus Memory
Walrus Memory is a portable memory layer for AI agents. It enables agents to operate reliably across apps and sessions without losing context – portable, verifiable, and fully under builders' control. Walrus Memory lets agents handle complex, long-running workflows and coordinate using data they can trust.
