Batch Operation | Patterns for Agentic Tools

arcade.dev/patterns

Context

Operations that need to be performed on multiple items.

Problem

N items requiring N tool calls is slow and token-expensive.

Solution

Support batch inputs:
- Accept arrays: items: list[Item]
- Return per-item results: Success/failure for each
- Optimize internally: Batch API calls
- Handle partial failures: either return partial results or fail all, depending on the use case

Examples

Python

@tool
def send_emails_batch(
    emails: list[EmailRequest]
) -> list[SendResult]:
    """Send multiple emails in one call.
    
    More efficient than calling send_email multiple times.
    Returns result for each email (success or failure).
    """
    results = []
    for email in emails:
        try:
            results.append(send_single(email))
        except Exception as e:
            results.append(SendResult(error=str(e)))
    return results

Considerations

Set reasonable batch size limits
Return status for each item, not just overall
Consider async for large batches in I/O bound operations
Consider a reasonable retry strategy within the batch operation, minding for idempotency and retry limits

Related Patterns

More in Tool Composition

Abstraction Ladder

Provide tools at multiple levels of granularity for the same capability.

Task Bundle

Combine multiple operations into a single tool.

Operation Mode

Provide different modes for different access patterns.

LLM-Optimized View

# Batch Operation

Process multiple items in a single tool call.

## Category
Tool Composition

## Context
Operations that need to be performed on multiple items.

## Problem
N items requiring N tool calls is slow and token-expensive.

## Solution
Support batch inputs:
- **Accept arrays**: items: list[Item]
- **Return per-item results**: Success/failure for each
- **Optimize internally**: Batch API calls
- **Handle partial failures**: either return partial results or fail all, depending on the use case

## Example (Python)
```python
@tool
def send_emails_batch(
    emails: list[EmailRequest]
) -> list[SendResult]:
    """Send multiple emails in one call.
    
    More efficient than calling send_email multiple times.
    Returns result for each email (success or failure).
    """
    results = []
    for email in emails:
        try:
            results.append(send_single(email))
        except Exception as e:
            results.append(SendResult(error=str(e)))
    return results
```

## Considerations
- Set reasonable batch size limits
- Return status for each item, not just overall
- Consider async for large batches in I/O bound operations
- Consider a reasonable retry strategy within the batch operation, minding for idempotency and retry limits

## Related Patterns
- partial-success
- async-job
- performance-hint

---
Source: https://arcade.dev/patterns/batch-operation