Multi-agent system

What is a multi-agent system?

A multi-agent system is a setup where several AI agents work on the same job at the same time. A lead agent takes the request apart, hands the pieces to other agents, and puts the results back together into one answer. Each of those other agents has its own context window, its own tools and its own piece of the problem.

The reason people build these is simpler than it sounds. A single agent works through a task step by step, and everything it has learned so far has to stay in one context window. On a broad question, that window fills up and the agent starts losing the thread. If you split the work across several agents, then each one keeps a clean context for its own piece, and they can also work at the same time instead of one after the other.

Anthropic put the underlying point plainly when they wrote about the research system behind their own product. On this kind of work, how much thinking you can spend on the problem largely determines how good the answer is, and a multi-agent setup is a way of spending far more of it in the same wall-clock time.

The orchestrator-worker pattern

Almost every multi-agent system in production uses the same shape, and it is worth knowing by name.

A lead agent, sometimes called the orchestrator, reads the request and works out an approach. It then starts up a handful of subagents, each with a narrow assignment and a description of what to bring back. Those subagents run at the same time, each doing their own searching or calling their own tools. The lead agent reads what comes back, decides whether it has enough, and either sends out another round or writes the final answer.

In Anthropic's system the lead agent typically starts three to five subagents in parallel, and those subagents use several tools in parallel themselves. That layer of parallelism is where most of the speed comes from on complex questions.

What makes this pattern work is that the subagents do not need to know about each other. Each one gets an assignment and gives back a result. All the coordination sits with the lead agent, which means you have exactly one place to look when the division of labour goes wrong.

What it costs

This is the part that gets skipped in most demos. Multi-agent systems are expensive, and the expense is structural rather than incidental.

Anthropic measured it on their own workload. A single agent already uses roughly four times as many tokens as a normal chat conversation, because it goes round a loop, calls tools and reads results. A multi-agent system uses roughly fifteen times as many tokens as a chat conversation. Those are their numbers on their task, so treat them as an order of magnitude rather than a rate card, but the shape holds anywhere: every subagent has its own prompt, its own context and its own output.

That has a direct consequence for when this is worth doing. If the task matters enough that spending fifteen times the tokens is fine, and open-ended enough that you cannot write the steps down in advance, then a multi-agent system earns its keep. Research, broad investigation and wide searches fit that description. A monthly report that always follows the same six steps does not, and you should build that as a plain workflow with one model call per step.

Where it gets hard in production

State that has to survive. These agents run for a long time and build up state across many tool calls. If something fails at minute twelve, you cannot just start over, because you would throw away everything that already worked and pay for it again. You need to be able to pick up where the agent was.

Errors that compound. A subagent that misunderstands its assignment does not fail loudly. It comes back with a confident, useless result, and the lead agent folds that into the answer. The failure only becomes visible at the end, in the final text.

Deploying while agents are running. Because a run can take a long time, you cannot just replace the code underneath it. Anthropic describes using gradual rollouts so that running agents are not disrupted mid-flight.

Coordination that stays synchronous. In most current setups the lead agent waits for its subagents. That is simpler to reason about, and it also means the whole thing runs at the speed of the slowest subagent in each round.

Evaluating something that is different every time. Two runs of the same question can take different paths and both be correct. That breaks the usual approach of comparing output to an expected answer, and it is why teams building these keep humans in the review loop to find the edge cases the automated checks miss.

What to watch out for with a multi-agent system

Check first whether one agent is enough. Multi-agent is the answer to a breadth problem. If your task is deep rather than broad, or if the steps are known in advance, then splitting it up adds cost and failure modes without adding much.

Put a hard ceiling on the loop. Cap the number of subagents, the number of rounds and the token budget. Without those limits a lead agent that is not making progress will keep sending out more work.

Write the assignments as if for a stranger. A subagent knows nothing except what you put in its instruction. Vague assignments are the most common cause of two subagents doing the same thing while a third does nothing useful.

Log every leg separately. When the answer is wrong, you want to see which subagent got which assignment and what it brought back. Without that, you are debugging a single block of text.

Keep the human at the moment of action. Reading and searching in parallel is low risk. Sending an email, making a payment or changing a record is not. Put the approval on the action itself rather than trusting the loop to stop in time.