Prompt engineering was the right framing when the prompt was the whole input. It isn't anymore. Here's how we think about context as a budget and what changed in our workflow.
We spent eighteen months getting good at prompt engineering. We wrote prompt style guides, ran A/B tests on phrasing, kept a private library of patterns that worked. Then sometime in mid-2025 the prompts stopped being the thing that moved the numbers. We'd tighten a system prompt and the eval scores barely budged. We'd loosen it and they barely budged. The variance had moved somewhere else.
What had moved was the context. By the time a request hit the model, the prompt was maybe ten percent of what the model was looking at. The other ninety percent was retrieved documents, tool outputs from earlier steps, conversation history, memory blobs, structured state from the application. None of that was the prompt and almost none of it was being engineered.
The shorthand we use internally: prompt engineering is what you write, context engineering is what you assemble. The prompt is one input among many. The job is to decide, for each request, what goes into the model's window and what doesn't - and to do that within a finite budget.
The budget is the part people skip. Context windows look large enough to be infinite until you start putting real work through them. A multi-turn agent with tool outputs and a retrieval step can fill a 200K window in three or four turns. Past a certain fill level model quality degrades - attention spreads, the model anchors on the wrong section, instruction-following gets brittle. The window is a budget, not a free shelf.
Four sources, ranked and pruned every turn.
1. System and policy. The fixed instructions, the role, the hard rules. Small, stable, always present.
2. Working memory. A compressed summary of what the agent has done so far in this run. Not the raw turn-by-turn history - a structured summary that gets rewritten as the run progresses. We treat this the way a database treats a materialised view. It's expensive to update but cheap to read and the read happens every turn.
3. Retrieved knowledge. Documents fetched for this specific turn, ranked and trimmed to the top N that fit the budget. Crucially, the ranker now knows the budget. If we have 8K tokens of context space left and three candidate documents at 4K each, we either pick one or summarise two. We don't truncate one mid-document and hope.
4. Tool outputs. Results from tools called earlier in the run. These are the most volatile input. A poorly bounded tool output - a database query that returns 5,000 rows when you wanted 5 - will eat the budget and crowd everything else out. Every tool now has a declared output budget and is responsible for staying within it.
The assembler runs every turn and produces a single ranked, budget-aware payload. The prompt is now the smallest piece of it.
# The shape of what we now think of as "the request"
context = {
"system": render_policy(), # ~800 tokens, fixed
"working_memory": summarise_run_so_far(state), # ~1200 tokens, rewritten
"retrieved": rank_and_fit(candidates, budget), # variable, budgeted
"tool_outputs": last_n_outputs(state, budget), # variable, budgeted
"user_turn": current_message, # small
}
assert total_tokens(context) <= MODEL_BUDGET
Three things, in order of how much they hurt.
Eval datasets had to change. Our old evals tested prompts against fixed inputs. Useless for context engineering, because the input the model actually sees depends on what the assembler produced. Our eval cases now capture the assembled context, not just the user message. When we regress, we can see whether the prompt changed, the retrieval changed, the summary changed or the budgeting changed.
Token accounting became a first-class concern. Every component that contributes to context now reports its token cost. The assembler logs the breakdown. When the model produces a bad answer, the first question is no longer "what was the prompt?" - it's "what did the context look like?" Half the time the answer is that something crowded out the part that mattered.
Prompt edits stopped being the first lever. When a new failure mode appears, our default used to be "tighten the prompt." Now the default is to look at the context first. Was the right document retrieved? Did the working-memory summary preserve the relevant fact? Did a tool dump too much output and push the policy out of attention range? Prompt edits are still useful but they're the last lever, not the first.
Prompt engineering didn't go away. The system prompt still matters, the role framing still matters, the format instructions still matter. What changed is the share of total quality variance the prompt explains. It used to be most of it. Now it's a meaningful slice but not the largest one. Treating it as the only lever is the mistake.
Context engineering isn't a rebrand of prompt engineering. It's a different discipline that takes prompt engineering as a subset. Treat the context window as a finite budget, build an assembler that ranks and prunes against it and move your evals to test what the model actually sees. The prompt is still part of the story - it just stopped being the whole story sometime in the last eighteen months.