Context
Tools that resolve natural identifiers.
Problem
Always requiring confirmation is slow; never confirming risks mistakes.
Solution
Use confidence thresholds:
- High confidence (>90%): Auto-accept
- Medium (50-90%): Return options
- Low (<50%): Ask for different input
- Configurable: Let users set threshold
- High confidence (>90%): Auto-accept
- Medium (50-90%): Return options
- Low (<50%): Ask for different input
- Configurable: Let users set threshold
Examples
Python
@tool
def find_user(
name: str,
auto_accept_threshold: float = 0.9
) -> User | ConfirmationRequest:
"""Find a user by name.
Auto-accepts matches above 90% confidence.
Returns options for lower confidence matches.
"""
matches = fuzzy_search_users(name)
if not matches:
return ConfirmationRequest(
message="No users found",
suggestions=["Try a different spelling", "Use email instead"]
)
best_match = matches[0]
if best_match.confidence >= auto_accept_threshold:
return best_match.user # Auto-accept
return ConfirmationRequest(
message=f"Multiple possible matches for '{name}'",
options=[{
"user_id": m.user.id,
"name": m.user.name,
"confidence": f"{m.confidence:.0%}"
} for m in matches[:5]]
) Considerations
- Document the threshold behavior
- Allow threshold customization
- Log auto-accepted matches for audit
- When threshold is not met, return a clear error message with potential matches for confirmation