Context engineering: what goes into the window, and in what order
Context engineering is choosing which tokens a model sees at each step: instructions, tools, documents, history and tool results, in an order that suits both the model and the cache. The layout that works, the 4 moves that keep an agent's context small, and a 40-turn run where they cut the tokens sent from 6.7 million to under 0.7 million.
Tecyfy · Published · 8 min read
Context engineering is deciding which tokens a model sees at each step, and in what order: instructions, tools, examples, documents, history and tool results. Anthropic's engineers put the goal in one line: find the smallest set of high-signal tokens that makes the outcome you want most likely.
For a single question it is mostly good Prompt EngineeringPrompt EngineeringWriting and testing the instructions, context and examples you give a model so it does a job reliably, and measuring each change instead of guessing.. For an AgentAgentA model that uses tools in a loop, deciding each next step from what the last one returned, until the task is done. it is a job that repeats every turn, because the agent keeps adding tool results to its own context and sends all of it again with each request. The Context EngineeringContext EngineeringChoosing what goes into a model's context at each step of a task, what to fetch only when needed, and what to clear, note down or summarise as the context grows. concept page defines the idea; this article shows the layout and the numbers.
Why less is better, even with 1 million tokens
Current Claude models hold 1 million tokens. Two things make filling them a mistake. First, context rot: as the window fills, the model gets worse at finding and using what is in it. Anthropic describes an attention budget that each token draws on; models stay capable, but lose precision. Second, cost: an agent pays for its whole context on every turn, so a 30,000-token file read at turn 3 is paid for again at turns 4, 5 and 40.
The order that works
Put what never changes first and what changes every time last:
- Tool definitions. As few as the task needs, with no 2 that overlap. If a person cannot say which tool fits, the model cannot either.
- System prompt. Clear sections (background, instructions, tool guidance, output format) at the "right altitude": concrete enough to act on, without hard-coded if-else rules.
- Examples. A few varied, typical ones, not a list of every edge case.
- Long documents. Above the question. Anthropic reports that putting the query at the end can improve answers by up to 30% on complex, multi-document inputs.
- Conversation and tool results. Kept small by the moves below.
- The question or next instruction. Last.
This order also suits Prompt CachingPrompt CachingLetting the API keep the processed start of a prompt for a few minutes, so later requests that begin the same way are cheaper and quicker.. The cache builds its prefix from tools, then system, then messages, and a change anywhere invalidates everything after it. A timestamp at the top of the system prompt, or tool JSON whose keys come out in a different order each time, means no request ever hits the cache. Our caching checklist lists the usual culprits.
The 4 moves that keep an agent's context small
- Fetch just in time. Give the agent references (file paths, stored queries, links) and tools to open them, instead of loading everything up front. RetrievalRAGA way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes. is the classic form; Claude Code reading files with grep and glob is the agentic one.
- Clear what is spent. Once a tool result has been used, replace it with a short placeholder. The Claude API does this on the server with context editing (
clear_tool_uses_20250919) once context passes a threshold you set. - Hand searches to a sub-agent. A SubagentSubagentAn agent started by another agent for one part of a task. It works in its own fresh context and returns only its findings. explores in its own fresh context, maybe tens of thousands of tokens, and returns a summary of 1,000 to 2,000.
- Take notes, then compact. The agent writes progress to a file or a memory tool and reads it back. When a conversation grows long anyway, CompactionCompactionReplacing the older part of a long conversation with a summary, so an agent can keep working when its context fills up. replaces older turns with a summary.
A worked run
An agent runs 40 turns. Every request carries 4,000 tokens of tool definitions, a 2,000-token system prompt and a 200-token task; each turn adds a 300-token reply and an 8,000-token tool result.
- No moves: request 40 is 6,200 + 39 × 8,300 = 329,900 tokens. Over the run the agent sends 6.72 million input tokens, $13.44 on Claude Sonnet 5.5 at $2 per million (before caching).
- Clear old results, keep the last 3: request 40 is 43,700 tokens; the run sends 1.43 million, $2.85.
- Clear, and search in a sub-agent that returns 1,500 tokens: request 40 is 24,200 tokens; the run sends 0.69 million, $1.37.
That is 10 times fewer tokens, and more important, the last request is 24,200 tokens instead of 330,000, far from where recall starts to slip. (The sub-agent's own tokens cost money too, but they are spent once, not re-sent 40 times.)
Largest request: 330KDeep in context-rot territory
Over 40 turns the agent sends 6.72M input tokens (about $13.44 on Sonnet 5.5 without caching); its last request is 330K tokens. Every tool result is sent again on every later turn.
A model of one run, not a measurement: tool definitions 4,000 tokens, system prompt 2,000, task 200, each reply 300. Clearing keeps the last 3 results and leaves a 50-token placeholder; a sub-agent returns 1,500 tokens; compaction leaves a 3,000-token summary. Price $2 per million input tokens.
Each move has a price
A cleared result has to be fetched again if it turns out to matter. A summary keeps what seemed important when it was written; an instruction from early in the run can be lost. So put rules that must last in the system prompt or a project file sent with every request, not in a message that compaction might summarise away. Test compaction prompts on real agent traces, aiming first to keep everything that matters, then to cut what does not.
What to do
- Order prompts static to dynamic: tools, system, examples, documents, history, question.
- Measure: log the input tokens of every request in a run. The context window page shows how much each model holds.
- Turn on tool result clearing before anything else; it is the lightest move.
- Move broad searches into sub-agents that return summaries.
- Keep rules in the system prompt; let compaction handle history only. For why Claude Code burns through limits the same way, see why Claude Code eats your limit.
Questions
- What is context engineering?
- Choosing and arranging the tokens a model sees at each step of a task: the system prompt, tool definitions, examples, documents, conversation history and tool results. Anthropic describes it as finding the smallest set of high-signal tokens that makes the result you want most likely.
- What is the difference between context engineering and prompt engineering?
- Prompt engineering is writing the instructions. Context engineering covers everything else that lands in the window too, and it repeats at every turn of an agent: what to fetch, what to keep, what to clear and what to summarise. Anthropic calls it the natural progression of prompt engineering.
- In what order should I put things in a prompt?
- Put what never changes first and what changes every time last: tool definitions, then the system prompt, then long documents, then the conversation, with the question at the end. That order matches how prompt caching builds its prefix (tools, system, messages), and Anthropic reports that putting long documents above the question can improve answers by up to 30% on complex, multi-document inputs.
- If models have a 1 million token window, why limit context?
- Because recall and accuracy fall as the window fills, an effect called context rot, and because every token is paid for on every request. An agent re-sends its whole history each turn, so a few large tool results kept around multiply the bill.
- What is context rot?
- The drop in a model's ability to find and use information as the number of tokens in its context grows. Anthropic's engineers describe it as an attention budget that every token draws on: models stay capable at long lengths but lose precision in retrieval and long-range reasoning.
Sources