Tool Calling

Deep workflow for LLM tool/function calling—schema design, validation, permissions, errors, idempotency, testing, and safe orchestration with agents. Use when wiring models to APIs, databases, or internal tools.

Tool Calling (Deep Workflow)

Tool calling is contract design between a probabilistic planner (the model) and deterministic systems. Failures are usually schema, permissions, or ambiguity—not the LLM “being dumb.”

When to Offer This Workflow

Trigger conditions:

• Designing OpenAI/Anthropic-style functions, MCP tools, or internal JSON tool protocols
• Debugging wrong arguments, hallucinated parameters, or unsafe side effects
• Building agents with many tools—selection and routing problems

Initial offer:

Use six stages: (1) define tool surface, (2) schema & validation, (3) authz & safety, (4) execution semantics, (5) errors & observability, (6) evaluation & regression. Confirm side-effect class (read-only vs write).

---

Stage 1: Define Tool Surface

Goal: Minimize tools; maximize clarity per tool.

Principles

• One action per tool when possible—avoid mega-tools with mode flags unless necessary
• Names descriptive: search_orders not do_stuff
• Prefer idempotent operations where writes exist; separate read vs write clearly

Anti-patterns

• Exposing raw SQL or shell to the model
• Too many overlapping tools → routing errors

Exit condition: Tool list with purpose, inputs, outputs, side effects table.

---

Stage 2: Schema & Validation

Goal: Arguments are typed, constrained, and machine-validated before execution.

Practices

• JSON Schema: enums, min/max, patterns, required fields
• Normalize dates, IDs, currencies server-side—never trust model formatting alone
• Default behaviors explicit in description + schema

Descriptions

• Tool and parameter docstrings seen by model—precise language; examples of valid args

Exit condition: Validator rejects invalid args with actionable errors back to model or orchestrator.

---

Stage 3: Authorization & Safety

Goal: Every tool call runs as some principal with least privilege.

Patterns

• User-scoped credentials carried from session; tool implementation re-checks ownership (e.g., order_id belongs to user)
• Admin tools behind explicit allowlists and human approval when needed
• Rate limits per user + global circuit breakers

Data exfiltration

• Tools that read sensitive data need output filtering and logging policies

Exit condition: Threat brief: “What if model is tricked into calling tool X?” answered.

---

Stage 4: Execution Semantics

Goal: Clear transactionality, retries, and idempotency.

Design

• Idempotency keys for writes; dedupe window
• Timeouts and cancellation propagation
• Ordering: parallel safe vs must be serial

Long operations

• Async jobs with poll tool vs blocking calls—prefer non-blocking for UX and cost

Exit condition: Semantics documented for retry behavior (at-least-once delivery common).

---

Stage 5: Errors & Observability

Goal: Model (or orchestrator) can recover from failures without leaking internals.

Error messages

• Structured error codes: ORDER_NOT_FOUND, PERMISSION_DENIED
• Hints for model on how to fix—without stack traces to end users

Observability

• Trace IDs across tool calls; audit log for write tools (who/when/args hash)

Exit condition: Dashboards/alerts on tool error rate, latency, denials.

---

Stage 6: Evaluation & Regression

Goal: Tool changes are tested like APIs.

Harness

• Golden conversations with expected tool calls (args normalized)
• Adversarial prompts attempting privilege escalation
• Version tools; deprecate with compatibility window

Exit condition: CI or manual eval suite before deploying new tools/schemas.

---

Final Review Checklist

• Minimal orthogonal tool set
• Strict schema validation on server
• AuthZ enforced per call; sensitive reads controlled
• Idempotency and timeouts defined for writes
• Structured errors + observability + eval harness

Tips for Effective Guidance

• Treat tool descriptions as API docs the model reads—iterate wording like UX copy.
• Recommend two-step patterns for dangerous ops: propose → confirm (human or policy).
• When using MCP, same discipline—server must validate everything.

Handling Deviations

• Read-only RAG: fewer semantic risks—still validate query args and injection into search backends.
• Local tools (filesystem): sandbox, path allowlists, size limits.