Multi-Agent or One Agent With Tools?

In This Issue

• Why a single agent with tools should usually be the baseline.

• The conditions that can justify a multi-agent design.

• What coordination adds to cost, latency, reliability, and evaluation.

• A decision matrix for choosing the smallest architecture that works.

The signal

Multi-agent frameworks are becoming easier to build. Tooling has lowered the implementation barrier. Runtime coordination still carries cost.

The architecture decision comes down to one question: does dividing this workflow across agents buy something a single agent with tools cannot deliver reliably?

OpenAI's current guidance is explicit about the default. A single agent can absorb many capabilities through tools, and teams should generally maximize that design before moving to multiple agents. It recommends splitting when prompt logic becomes difficult to manage or when tool selection remains unreliable even after the tools and instructions have been improved. Microsoft gives similar guidance: use the lowest level of complexity that reliably meets the requirement, with a tool-using single agent as a common default.

Separate roles are not automatically good runtime boundaries. They can often remain inside one agent as instructions, policies, or tool permissions. Microsoft specifically warns against assuming that distinct roles require distinct agents.

What it is

A single-agent system with tools has one reasoning loop. The agent chooses among APIs, search, retrieval, code execution, databases, or other capabilities as the task unfolds. The same loop keeps the working context and owns the final response.

A multi-agent system divides execution across two or more agent loops. A manager may delegate to specialist agents, or agents may hand control to one another. OpenAI describes both manager and decentralized handoff patterns. Anthropic's Research system uses an orchestrator-worker model in which a lead agent creates parallel subagents and later synthesizes their findings.

The distinction is whether the workflow has one decision-making loop or several loops that must exchange work, context, and control.

Where multi-agent can earn its cost

The strongest case is independent parallel work. If a task separates into branches with little dependency between them, separate agents can increase search breadth or reduce elapsed time.

Anthropic's production research system is a useful example. Its internal evaluation found that a lead Claude Opus 4 agent with Claude Sonnet 4 subagents outperformed a single Claude Opus 4 agent by 90.2% on its research evaluation. Anthropic says the advantage is strongest on breadth-first queries that require several independent directions to be explored at once. It also reported up to a 90% reduction in research time after increasing parallelism. These are system-specific results, but they show a workload where coordination produced measurable value.

A second case is context separation. Independent agents can work with separate context windows, then return compressed findings to a coordinator. That can help when one loop would otherwise carry too much information or when different branches need different source material.

A third case is tool or domain separation. OpenAI recommends considering multiple agents when complicated logic becomes hard to scale or when overlapping tools cause persistent selection errors. Microsoft also points to cross-domain workflows and distinct security boundaries as reasons separation may be justified.

The separation should fix a measured limitation. Organizational neatness is not enough.

What changes when you split the loop

Once a workflow is split, every agent boundary becomes an interface.

The system now has to decide what context crosses the boundary, what the receiving agent is expected to return, who owns shared state, how retries behave, and how a failed branch affects the rest of the run. The more agents that can call one another, the larger the space of possible execution paths becomes.

Anthropic describes this directly. Its early multi-agent research agents spawned excessive subagents, duplicated work, left gaps, and distracted one another with unnecessary updates. It also notes that asynchronous multi-agent execution creates added challenges in result coordination, state consistency, and error propagation.

The cost also shows up in inference. Anthropic reports that its agents use roughly four times as many tokens as normal chat interactions, while its multi-agent systems use about fifteen times as many. The comparison is to chat, not to every possible single-agent implementation, so the number should not be treated as a universal multiplier. It still makes the economic constraint visible: the extra performance has to be worth the extra work.

Evaluation gets harder too. A single agent already has nondeterministic paths through tools. Multi-agent execution adds routing and handoff decisions. Anthropic found that evaluating its system required judging end outcomes and process quality rather than expecting one fixed sequence of steps.

A practical decision matrix

This matrix pushes the decision toward evidence. Several responsibilities are not enough. A recurring single-agent failure that improves after the split is.

The build sequence

Start with the single-agent version unless the workflow has a hard separation requirement from day one.

Give that agent the required tools. Keep tool names, descriptions, inputs, and outputs distinct. Add tracing. Build a small evaluation set from real tasks. Measure task success, latency, model and tool cost, and the failure mode you care about.

Then split only the responsibility associated with a recurring failure. Run the same evaluation again.

This creates a useful threshold. The multi-agent version should earn its extra architecture with a result you can observe: higher completion quality, materially lower elapsed time, cleaner security separation, or a failure rate the single loop cannot reach after reasonable optimization.

If the split changes nothing measurable, collapse it.

What this means for builders

Decide the agent count after mapping the workflow.

Stage 2 is where the boundary should be decided. Map the workflow first. Mark which steps share state, which can run independently, which tools overlap, and where control or permission boundaries are real. That tells you whether you have one loop with several capabilities or several loops that need orchestration.

For most early products, a single agent gives you a cleaner baseline before routing behavior makes failures harder to attribute.

Operator takeaway

A multi-agent architecture adds an operating surface around the prompts.

Tracing has to show which agent acted, what it received, which tool it called, what it returned, and where the run failed. State and retries need explicit ownership. Cost controls need to account for fan-out. Evaluation should cover both end-state correctness and coordination failures.

Only keep that operating burden when the workflow needs the separation. If the same work can stay inside one observable loop, the extra coordination has no clear return.

Takeaways

• Use one agent with tools as the baseline unless separation is a hard requirement.

• Split when independent parallel work, context isolation, tool complexity, or security boundaries create measurable value.

• Define every agent boundary as an interface with explicit context, outputs, state ownership, and failure behavior.

• Compare architectures on the same eval set. Measure quality, latency, cost, and operational failure modes.

• Keep the multi-agent design only if it earns the coordination cost.

Worth reading

Anthropic's engineering write-up on its Research system is useful because it publishes both sides of the trade-off: large gains on parallel research tasks, along with the token, delegation, evaluation, and reliability costs that appeared in production. Anthropic: How we built our multi-agent research system

If this helped you, leave a comment or your reaction. I’d like to hear where you landed.

INVENEW exists to help tech builders, operators, founders, and leaders turn AI from experiments into working systems.

In partnership with AWS

Sponsored link – Webinar: Redefining application and API security in the age of AI. See how to embed security early, align with compliance expectations, and discover AWS Partner solutions in AWS Marketplace.

Note: Third-party company and product names belong to their respective owners and are used for identification and illustrative reference only.