The Anatomy of a Context Window: The Seven Components

Before you can engineer context, you need to understand what a context window actually contains. It is not just the prompt. It is seven distinct components, each competing for the model attention and each consuming part of a finite budget.

Most people picture a context window as a box you put your prompt into. That mental model is wrong in a way that prevents good context engineering. A production context window is a carefully (or carelessly) assembled collection of distinct components, each a deliberate choice with costs and benefits. Understanding these components is the foundation of everything else.

The Seven Things Competing for Space

A production context window typically contains: system instructions that set the model behaviour and role; the user message or query; conversation history from prior turns; retrieved documents pulled in via RAG; tool definitions describing the functions the model can call and their outputs; memory persisted from earlier sessions; and structured outputs or schemas the model is expected to follow. Each is a lever you can pull, and the art is deciding how much of each to include, in what order, and in what format.

The Window as Attention Span

The most useful metaphor is not a storage box but an attention span. The window holds only so much, and the model attention is not uniform across it. Information at the very beginning and the very end gets disproportionate weight; information buried in the middle gets less. This is the well-documented "lost in the middle" effect, and it has a direct consequence: where you place information matters as much as whether you include it.

Critical instructions belong near the start or the end, never buried in the middle of a long context. Some production systems exploit this deliberately, rewriting the current plan at the end of context on every step so it always sits in the model highest-attention zone.

The Token Budget

Every token costs money and latency. As context grows, you pay more per call and wait longer for responses. This creates real economic pressure to keep context lean, not just a quality pressure. Thinking in terms of a token budget reframes context engineering as resource allocation: you have a finite budget of useful attention, and every element spends part of it. The question for each is whether it earns its place or wastes attention something more valuable could use.

Why Input Dwarfs Output

A critical economic fact: input tokens are far cheaper per token than output tokens, but applications send vastly more input than they receive output. A single agent step might send tens of thousands of input tokens to receive a few hundred output tokens. The result is that input dominates total cost. Messy, bloated context is not just a quality problem; it is literally where your money goes, paid on every single call.

The Practical Upshot

Understanding the anatomy changes how you build. Instead of dumping everything into one undifferentiated prompt, you treat each component as a deliberate choice: how much history to retain, which documents to retrieve, which tools to expose, what to remember. This component-level thinking is where context engineering begins, and it is what separates systems that scale from systems that break under their own accumulated weight.