Context Engineering: How to Feed an AI Agent the Right Information

Context engineering is the practice of deciding what information goes into an AI model's context window for a given task, and in what order. It covers the system prompt, retrieved documents, tool results, memory, and conversation history. Get it right and a mid-tier model ships good work. Get it wrong and the best model on the market produces confident nonsense. Here is how it works and how to do it.

TL;DR

• Context engineering is managing everything the model sees, not just the instruction you typed. The instruction is a small slice of it.
• The context window is a budget. Every token you spend on noise is a token the model cannot spend on your actual task.
• Four moves cover most of it: write good instructions, retrieve only relevant data, compress tool output, and prune stale history.
• "Context rot" is the slow failure mode. Long-running agents degrade as their context fills with junk. Reset before it rots.
• This is the highest-leverage skill in AI right now. Most agent failures are context failures, not model failures.

What context engineering actually is

A model does not know anything about your task except what is in its context window at the moment it responds. The window is a fixed budget of tokens. Whatever you put in it, the system prompt, the user message, retrieved files, tool results, the running conversation, is the entire universe the model reasons over. Nothing else exists to it.

Prompt engineering asks: how do I word this one instruction. Context engineering asks a bigger question: of everything I could put in the window, what should I, and in what order. For a single chat message, those questions look similar. For an agent that runs twenty steps and calls eight tools, they are not close. The wording of step one barely matters by step fifteen. What matters is whether the window still holds the right information or has filled up with the wrong information.

Think of it as packing a bag for someone who can only use what is inside it. Pack the wrong things and they fail, no matter how capable they are.

Why it beats tweaking the prompt

Most people reach for prompt tweaking when an agent misbehaves. They reword the instruction, add "be careful," add an example, and hope. Sometimes it helps. Often the real problem is upstream: the model never had the information it needed, or it had too much.

A concrete case. An agent answers customer questions about your product. It keeps getting pricing wrong. You can rewrite the prompt ten times. It will not help, because the current price is not in the context. The fix is context engineering: retrieve the live pricing page and put it in the window before the model answers. No prompt wording beats having the actual fact present.

The flip side is just as common. An agent gets worse as you give it more. You dump your entire docs site into the context "to be safe," and accuracy drops, because now the relevant paragraph is buried under forty pages the model has to wade through. Models lose details in the middle of long contexts. More is not better. Right is better.

The four moves

Most context engineering comes down to four repeatable moves. You will use all four in any serious agent.

1. Write the instructions like a contract. The system prompt sets the rules, the role, the output format, and the hard limits. Be specific. "Return JSON with keys title and summary, nothing else" beats "summarize this nicely." State what to do when the model is unsure, so it does not improvise. This is the one part of context that stays constant across every step, so it pays to get tight.

2. Retrieve only what the task needs. This is where retrieval-augmented generation (RAG) lives. Instead of stuffing all your data into the window, you fetch the few chunks relevant to the current question and insert those. The skill is in the filtering, not the fetching. A good retrieval step returns three paragraphs that answer the question. A bad one returns thirty that mention the keyword. Tune for precision over recall once you have basic recall working.

3. Compress tool output before it lands. Agents call tools, and tools return walls of text: full API responses, entire file contents, raw HTML. Most of it is noise. Pass the model a 200-token summary of what the tool returned, not the 8,000-token raw blob. A search tool should return titles and snippets, not full pages. A database query should return the three rows that matter, not the schema dump. You decide what the model sees, so trim before it sees it.

4. Prune the running history. In a long session, old turns pile up. Step twenty does not need the verbatim output of step three. Summarize completed sub-tasks into a sentence, drop dead ends entirely, and keep only what the next step depends on. Many agent frameworks now do this with a rolling summary. The principle is the same whether it is automatic or manual: the history is a resource you curate, not a log you accumulate.

Context rot and how to catch it

Here is the failure mode that surprises people. An agent works great for the first few steps, then slowly gets dumber. It repeats itself, forgets earlier decisions, calls the same tool twice, and makes choices that ignore information sitting right there in the context.

That is context rot. The window has filled with stale tool outputs, abandoned attempts, and old conversation, and the signal-to-noise ratio has collapsed. The model is technically reading everything, but the everything is now mostly junk, so its decisions degrade.

You catch it by watching for the symptoms: repeated actions, contradictions with earlier steps, and a drop in quality the longer a session runs. You fix it by resetting or compacting before the rot sets in, not after. Set a token threshold. When the context crosses it, summarize the session so far into a clean handoff, start a fresh window with that summary plus the current goal, and continue. A short, clean context beats a long, rotten one every time.

If you want the tools I lean on to build and watch agents day to day, I keep a running list in the AI daily driver stack. Several of them exist specifically to make context visible while an agent runs.

A practical workflow

Putting it together, here is the loop I use when an agent is not performing.

First, look at the actual context, not the prompt. Print the full window the model saw on the failing step. Most teams have never done this once. It is uncomfortable and it is where the answer almost always is.

Second, ask three questions of that window. Is the information the model needed present? Is information it did not need crowding it out? Is anything stale or contradictory? You will usually find at least one of the three is wrong.

Third, fix the pipeline, not the wording. Add the missing retrieval. Cut the irrelevant dump. Compress the tool output. Reset the rotted history. Then run it again and look at the new window.

Do that a few times and you develop an instinct for what a clean context looks like. That instinct is worth more than any single prompt trick, because it transfers to every agent you build after.

The bottom line

Context engineering is the difference between an agent that demos well and one that ships work. The model is rarely your bottleneck. What you feed it is. Manage the window like the scarce budget it is: precise instructions, relevant retrieval, compressed tool output, pruned history. When something breaks, read the context before you touch the prompt. The fix is almost always in there.

If you want more field notes like this, build-first and hype-free, join the newsletter. I send the patterns I am actually using, one practical idea at a time.