Parallel Tool Calling

Master concurrent tool execution to build faster, more efficient AI agents

Code It Right

Now let's see how to implement parallel tool calling in real code. We'll cover basic patterns, error handling, timeouts, and batching strategies.

Interactive: Code Pattern Explorer

parallel_tools.pyBasic Parallel Execution
import asyncio
from typing import List, Any

async def call_tool(tool_name: str, args: dict) -> Any:
    """Simulate calling a tool"""
    await asyncio.sleep(1)  # Simulate I/O delay
    return f"{tool_name} result with {args}"

async def parallel_tool_calls(tools: List[dict]) -> List[Any]:
    """Execute multiple tools in parallel"""
    # Create tasks for all tools
    tasks = [
        call_tool(tool['name'], tool['args']) 
        for tool in tools
    ]
    
    # Run all tasks concurrently
    results = await asyncio.gather(*tasks)
    return results

# Usage
tools = [
    {'name': 'weather_api', 'args': {'city': 'NYC'}},
    {'name': 'news_api', 'args': {'topic': 'tech'}},
    {'name': 'stock_api', 'args': {'symbol': 'AAPL'}}
]

results = await parallel_tool_calls(tools)
# All 3 tools complete in ~1 second instead of 3!

Key Implementation Concepts

Async/Await

Use async/await syntax for concurrent I/O operations without blocking

🎯

Promise.all / gather

Wait for all promises/tasks to complete before proceeding

🛡️

Error Isolation

Handle errors individually so one failure doesn't break everything

⏱️

Timeouts

Prevent slow tools from blocking your entire system indefinitely

Production Best Practices

Always Add Timeouts
Never wait forever. Set reasonable timeouts for all tools.
Limit Concurrency
Use batching to avoid overwhelming external APIs (respect rate limits).
Handle Partial Failures
Design systems to work even if some tools fail.
Log Everything
Track which tools succeed/fail and how long they take.