How to Build a Multi-Agent System (And When You Actually Need One)
To build a multi-agent system, split one job into narrow roles, give each role its own agent with a focused prompt and a small tool set, and put an orchestrator in charge of routing work to those agents and merging what they return. Start with a single agent. Add a second only when one context window can no longer hold the job reliably. Here is the full pattern, the topologies that cover most builds, and the failure modes to plan around.
TL;DR
- A multi-agent system is several single-purpose agents coordinated by an orchestrator, not one agent doing everything.
- Reach for it only when one agent's context window or tool list has grown too large to stay reliable. Most tasks still want a single agent.
- Three topologies cover almost everything: orchestrator-workers, pipeline, and debate.
- Give each sub-agent a narrow prompt, two to six tools, and a strict output schema so the orchestrator can parse the result.
- The hard part is not the agents. It is coordination: routing cleanly, merging results, and stopping runaway loops before they burn your budget.
What a multi-agent system actually is
A multi-agent system is a team of AI agents, each with one job, working under a coordinator. One agent researches. One writes. One checks the work. None of them tries to do all three.
Compare that to a single agent. A single agent runs one loop: read the task, pick a tool, act, observe the result, repeat until done. That works well until the job gets wide. When you bolt fourteen tools and four unrelated sets of instructions onto one agent, the model starts choosing the wrong tool, forgetting earlier rules, and mixing up which phase of the task it is in. Splitting the work into agents fixes that by keeping each context small and each role clear.
The coordination is what makes it a system rather than a pile of prompts. An orchestrator receives the task, decides who does what, sends each worker a clean assignment, collects the structured results, and assembles the final answer. The orchestrator never does the actual research or writing. It manages.
When you need more than one agent (and when you don't)
Default to one agent. Multi-agent is a real cost: more tokens, more latency, and a new failure point at every hand-off. Do not pay it until a single agent stops being reliable.
Three concrete signals tell you it is time:
- Tool overload. Your tool list has passed roughly a dozen and the model starts calling the wrong one. Splitting tools across specialized agents restores accuracy.
- Prompt bloat. Your system prompt has swelled with rules for tasks that never run together (invoice parsing rules sitting next to email drafting rules). Each agent should carry only the rules its job needs.
- Distinct phases. The work breaks into stages that need different tools and a different mindset, like research then draft then review. A single agent smears those together. Separate agents keep each stage focused.
If your task is a straight line of steps with one small tool set, one agent in a loop is simpler, cheaper, and easier to debug. Do not build a mesh to do a list's job.
The three topologies that cover most builds
You do not need an exotic architecture. Three shapes handle the large majority of real systems.
Orchestrator-workers. A central orchestrator fans work out to specialized workers and merges what comes back. Best when sub-tasks are independent and can run in parallel, like researching five sources at once. This is the default and the one to learn first.
Pipeline. Agents run in a fixed sequence, each one's output feeding the next: extract, then transform, then format. Best when the work has clear stages that must happen in order. Simple to reason about and easy to debug because the flow is linear.
Debate or critic. One agent produces, a second critiques, and they iterate until the output passes a bar. Best when quality matters more than speed and a fresh set of eyes catches mistakes the author cannot. The key is that the critic is a separate agent with its own prompt, so it is not just the writer grading its own homework.
Most production systems are a blend. A common one: an orchestrator dispatches research to parallel workers (orchestrator-workers), pipes the merged findings into a writer, then routes the draft through a critic loop (debate) before returning it.
How to build one: a worked example
Say you want a system that produces a researched, edited brief on any topic. Build it in four moves.
1. Name the roles. Researcher, writer, critic. Three agents plus an orchestrator. Do not invent a role that has no distinct job.
2. Write each worker's prompt narrow. The researcher's system prompt is short: "You gather facts. Given a topic, run searches, return the five most relevant findings as JSON with a source URL for each. Do not write prose." The writer only writes. The critic only scores and flags. Each prompt names the job, the tools, and the exact output shape.
3. Give each agent a strict output schema. The researcher returns { "findings": [ { "claim": "...", "source": "..." } ] }. The writer returns { "draft": "..." }. The critic returns { "score": 0-10, "issues": ["..."] }. The orchestrator can parse these without guessing. If you are new to forcing clean JSON out of a model, the mechanics are in How to Get Reliable Structured Output From an LLM.
4. Write the orchestrator's logic. In plain code, not a giant prompt: call the researcher, pass its findings to the writer, send the draft to the critic, and if the critic's score is below 7, send the issues back to the writer for one revision. Then return the final draft. The control flow lives in code you can read and test, which is what keeps a multi-agent system debuggable.
Notice the orchestrator is mostly ordinary code with a few model calls inside it. That is the point. The agents supply the intelligence, and your code supplies the structure. If you want a real, shipped version of this pattern to study, Claude Finance Agents breaks one financial analysis into coordinated specialist agents you can lift into your own build.
How agents talk to each other
Through structured messages, never loose conversation. Treat every hand-off like an API call: defined inputs, defined outputs, and a validation check before the next agent consumes the result.
Two rules save you the most pain:
- Pass the minimum. Do not forward one agent's entire transcript to the next. Send only what the next agent needs to do its job. The critic needs the draft, not the researcher's raw search logs. Trimming the hand-off keeps each context small and each agent accurate. This is the same discipline as context engineering applied across agents instead of within one.
- Validate every return. When the researcher hands back JSON, check that it parses and matches the schema before the writer touches it. If it does not, retry that one agent rather than letting a malformed result poison everything downstream. One bad hand-off caught early is a retry. Caught late it is a wrong final answer with no obvious cause.
The failure modes nobody warns you about
Multi-agent systems fail in ways single agents do not. Plan for these three.
- Runaway loops. A critic that never approves and a writer that never satisfies it will ping-pong until your token budget is gone. Cap every loop. Two or three revisions, then ship the best draft and move on. Never let a loop run unbounded.
- Lost-in-the-handoff drift. Each hand-off is a chance to drop a constraint. The user asked for a 200-word brief, the researcher ignored the length, the writer never saw the constraint, and the output is 600 words. Carry the key constraints through every message, and have the orchestrator check them at the end.
- Cost creep. Five agents passing messages can cost several times a single agent for the same task. Track cost per successful task, not per call. If the multi-agent version is not clearly better on quality or reliability, collapse it back to fewer agents. Complexity you cannot justify with a number is complexity to remove.
The through-line: the agents are the easy part. Coordination is where systems break, so put your engineering effort into clean routing, tight hand-offs, hard loop caps, and honest cost tracking.
The bottom line
A multi-agent system is not a smarter agent, it is a better-organized one. Split a job into narrow roles only when a single agent's context or tool list has outgrown reliability. Give each worker a focused prompt, a few tools, and a strict output schema. Keep the orchestrator thin and put the control flow in code you can test. Then watch cost per successful task to be sure the extra machinery is earning its keep. Start with one agent, prove you need the second, and grow from there.
Want the patterns I actually ship, sent when they are ready to steal? Join the newsletter.
What is a multi-agent system?
A multi-agent system is several single-purpose AI agents that coordinate to finish one job, instead of one agent trying to do everything. Each sub-agent has its own focused prompt, a small set of tools, and a narrow role such as researcher, writer, or critic. An orchestrator sits above them: it decides which agent handles which piece of the task, hands each one a clear assignment, and merges the results into a single answer. The value is separation of concerns. A researcher agent with three search tools stays sharper than one agent juggling fourteen tools and four conflicting instructions in the same context window.
When should I use multiple agents instead of one?
Use one agent by default and add more only when a single context window can no longer hold the job reliably. The signals are concrete: the tool list has grown past roughly a dozen and the model starts picking the wrong tool, the system prompt has swelled with rules for tasks that never run together, or the work has distinct phases that each need different tools and a different mindset. If your task is a straight line of steps, one agent in a loop is simpler and cheaper. Multi-agent earns its complexity when the work genuinely branches into parallel roles or needs an independent reviewer that a single agent cannot give itself.
What is an orchestrator agent?
The orchestrator is the agent in charge of coordination. It does not do the underlying work itself. It reads the incoming task, breaks it into sub-tasks, routes each one to the right worker agent, waits for structured results, and assembles them into the final output. Think of it as a manager who never touches the keyboard. Keeping the orchestrator thin matters: if it starts doing research or writing on its own, you have rebuilt the single overloaded agent you were trying to escape, just with extra latency and cost.
How do agents pass information to each other?
Through structured messages, not free-form chat. The orchestrator sends each worker a small, explicit assignment (the task, the constraints, the inputs it needs) and each worker returns a strict output, usually JSON that matches a schema the orchestrator can parse. Avoid letting agents converse in loose natural language, because ambiguity compounds at every hop and you lose the ability to validate what came back. Treat every hand-off like an API call: defined inputs, defined outputs, and a check that the result is well formed before the next agent consumes it.
Are multi-agent systems worth the extra cost?
Sometimes, and you should measure it rather than assume. Every extra agent adds token cost, latency, and a new place for the workflow to fail, so the reliability or quality gain has to be real. Multi-agent pays off when parallel roles genuinely improve the output (a separate critic catching errors the writer cannot see) or when splitting the work is the only way to keep each context small enough to stay accurate. It does not pay off when you have split a simple linear task into five agents that now spend most of their tokens passing messages around. Start with one agent, prove you need the second, and let cost per successful task be the judge.
OpusJake is Jake Schincariol's operating system for building with AI: agents, workflows, prompts, and the free resources behind them. Get the next move every week.