How Function Calling Works — And Why It Changes Everything About Tool Use

Function calling is not the same as asking an LLM to return JSON. It is a first-class API feature where the model is specifically trained to decide when and how to call tools. The difference matters enormously in production.

Many AI agent builders have a misconception about how their agents use tools. They believe the LLM "executes" the tool. It does not. The LLM decides which tool to call and what arguments to pass — your application executes the actual tool. This is not a subtle distinction. It is the foundation of why function calling is safe, controllable and production-ready.

The Three-Step Flow

Step 1 — You define tools in JSON schemas and pass them to the LLM in the API request. Each schema describes what the tool does, when to use it, and what parameters it accepts. The LLM receives this description and uses it to reason about when tool use is appropriate.

Step 2 — The LLM outputs a structured tool call when it determines a tool is needed. Not prose. Not a request. A structured JSON object specifying exactly which function to invoke and with what arguments. This is deterministic and parseable — your application does not need to interpret ambiguous text output.

Step 3 — Your application executes the tool and returns the result to the LLM in the next message. The LLM incorporates the result into its reasoning and either calls another tool or produces its final response. The LLM never has direct access to your systems — it only sees what you choose to return to it.

Writing Tool Schemas That Actually Work

The description field is the single most important part of any tool schema. It tells the model three things: what the tool does, when to call it versus when not to, and what it returns. A one-line description produces unreliable tool use. A three-sentence description that answers all three questions produces consistent, correct tool selection.

Include: what the tool does in plain English, when the model should prefer this tool over alternatives, what the tool returns and in what format, and any important limitations (rate limits, data freshness, scope restrictions).

Parallel Tool Calling — The Performance Multiplier

GPT-4o and Claude 3.5 both support parallel tool calling — calling multiple tools simultaneously in a single model response. When a task requires multiple independent data lookups, parallel calling reduces latency by 5-8x compared to sequential calls.

A research agent comparing three companies can call nine tools simultaneously — revenue data, employee count and recent news for each company — and assemble the complete comparison in one round trip instead of nine sequential ones. This is not a minor optimisation. For data-intensive tasks, it is transformative.