How does parallel tool calling work, and what do you have to get right when you use it?

The model can ask for several tool calls in one response when they do not depend on each other, for example looking up the weather in three cities. Your code runs them, ideally at the same time, and sends all the results back together in one message, each matched to its call by id. Done right, it cuts the number of model turns and the total time. Done wrong, it breaks the conversation format or runs actions that should have been sequential.

How it works on the Claude API

1. The response contains several tool_use blocks, each with its own id.
2. Your code executes them, in parallel where safe.
3. The next user message holds one tool_result for each, with the matching tool_use_id.
4. The model continues with all the results in view.

Anthropic's parallel tool use guide stresses the formatting rule: every result goes back in a single user message. Splitting results across messages teaches the model, through the conversation history, to stop calling tools in parallel.

Turning it off

Parallel calls are on by default. Setting disable_parallel_tool_use: true inside tool_choice limits the model to at most one tool call per response (with tool_choice: auto). Use it when calls must happen strictly one at a time.

What to get right

• Independence. Read-only lookups are safe to run together. Actions that change state (create an order, then charge it) must not run in parallel. Enforce ordering in code, not only in the prompt.
• Partial failure. If one of four calls fails, return the three results and an error result for the fourth. The model can retry just that one.
• Concurrency limits. Ten parallel calls to a rate-limited API may all fail. Use a pool with a sensible limit.
• Idempotency. If a parallel batch is retried, calls that already succeeded should not run twice. Use idempotency keys for writes.
• Ordering of results. Match by id, not by position.

When it pays

Research and lookup steps: fetch several documents, query several systems, check several conditions. Each turn of the agent loop resends the whole conversation, so collapsing four turns into one saves input tokens as well as time.

A good example to give

"Our travel agent checked flights, hotels and visa rules in one turn instead of three. Latency per request fell by roughly half, and we disabled parallel calls only for the booking step, which must run in order."