Multi-Agent Orchestration

Multi-agent orchestration is what you reach for when one agent's context window and attention stop being enough. Instead of a single model grinding through a broad task sequentially, a coordinating agent splits the work, hands each piece to a specialist agent with its own context and tools, and merges the results. It is the same instinct as staffing a hard project with a team instead of one heroic engineer: separation of concerns, applied to LLMs.

When several agents beat one#

Two forces justify the extra machinery. The first is context isolation. A single AI agent doing broad research fills its window with search results, page dumps, and dead ends; long before the window is technically full, the signal drowns in the noise. A subagent runs in its own fresh window, can burn 100,000 tokens exploring its slice, and returns only a 1,000-token summary to the coordinator. The second is separation of concerns: each specialist gets a focused prompt, its own tools, and exactly one job, so it digs deep without being distracted by the other nine things the overall task involves.

The effect is measurable. In Anthropic's research system, a lead agent running Claude Opus 4 with Claude Sonnet 4 subagents beat a single Opus 4 agent by 90.2 percent on their internal research eval. The gains concentrate on breadth-first work, queries with many independent directions to pursue at once. Their canonical example is "find every board member of the IT companies in the S&P 500": a single agent grinds through it with slow sequential searches and fails, while parallel subagents each take a slice of companies and finish together. Tellingly, token usage alone explained about 80 percent of the performance variance in their analysis. Multi-agent systems win largely because they bring more total context to bear on the problem than any one window can hold.

The supervisor pattern#

The dominant structure is a supervisor, also called orchestrator-worker or lead agent. The supervisor owns the plan: it analyzes the goal, decomposes it, spawns subagents with scoped subtasks, waits for their results, and synthesizes the final answer. Subagents never talk to each other; everything flows up through the supervisor. LangChain's framing of the same idea is "subagents as tools": each specialist is exposed to the coordinator through tool calling, so delegation is just another tool use the model decides to make.

The craft is in the task description. Anthropic found that vague delegations like "research the semiconductor shortage" cause subagents to duplicate each other's work or leave gaps; reliable supervisors spell out the objective, the output format, the tools to use, and the boundaries of the slice. Concretely, a competitive analysis of eight vendors becomes eight subagent calls, each told "profile vendor X's pricing and enterprise features, return a 300-word summary with sources, do not cover the other vendors," followed by one synthesis pass over the eight summaries.

Handoffs and shared state#

The supervisor pattern keeps one agent in charge for the whole run. The alternative is a handoff, where control itself moves between peer agents. In the OpenAI Agents SDK, a handoff is literally a tool: an agent named Refund Agent appears to the model as a tool called transfer_to_refund_agent, and invoking it transfers the conversation to that specialist, which takes over from there. This decentralized shape fits triage flows. A support front door classifies the request, then hands the whole session to billing, orders, or refunds rather than relaying messages on their behalf.

Either way, the hard design question is shared state: what crosses the boundary between agents. Pass the full conversation and you lose the context isolation you came for; pass a one-line summary and the specialist re-derives or contradicts decisions already made, a telephone-game failure. Production systems maintain explicit shared artifacts instead: a plan the supervisor keeps updated, a scratchpad or filesystem that subagents write outputs to (a form of agent memory), and structured result formats so the merge step is mechanical rather than interpretive. Anthropic's system has subagents write large findings to files and return only lightweight references, keeping bulky outputs out of the coordination channel entirely.

What it costs#

Multi-agent systems are token furnaces. Anthropic measured single agents using about 4 times the tokens of a chat interaction, and multi-agent systems about 15 times. A question that costs a cent as a plain completion can cost dollars as an orchestrated run, so the economics only work when task value clearly exceeds token cost: research that saves a person-day, due diligence, large code audits. Latency is the friendlier side of the ledger. Because subagents run in parallel, a ten-direction research task finishes in little more than the time of its slowest branch, where a single agent would work the directions back to back. One common cost optimization follows directly: run the supervisor on a frontier model and the subagents on a model a tier cheaper, since scoped subtasks need less judgment than planning and synthesis do.

Failure propagation, and the case against#

More agents also means more ways to fail. The MAST taxonomy, built from over 1,600 annotated execution traces across seven popular frameworks, sorts multi-agent failures into three buckets: system design issues (bad specifications, missing termination conditions), inter-agent misalignment (one agent ignoring or contradicting another's work), and weak task verification (nobody checks the merged answer). Errors also compound vertically: a wrong claim from one subagent flows into the synthesis unchallenged unless the supervisor verifies it, so a 5 percent error rate per specialist quietly becomes a much higher error rate per final answer.

Cognition's "Don't Build Multi-Agents" essay presses the counterargument: actions carry implicit decisions, and parallel agents that cannot see each other's traces make conflicting decisions that no merge step can reconcile. That is why sequential, tightly coupled work, most coding included, still belongs in a single agent with continuous context. Treat the architecture as a hypothesis and run end-to-end evals before believing an orchestration win.

Practical takeaways#

Reach for orchestration when the task is breadth-first, parallelizable, tool-heavy, and valuable enough to pay 15x tokens; keep one agent when steps are sequential or every decision depends on the same shared context. Make the supervisor write explicit, scoped task descriptions with output formats and boundaries. Pass state through structured artifacts and files rather than chat summaries. Verify subagent outputs before synthesis, and evaluate the system end to end, because per-agent quality says little about the quality of the merged answer.