How Does Tool Calling Work?

The model never runs anything. It emits a structured request, your code executes it, and the result goes back as text. Everything follows from that.

The most important fact about tool calling is what does not happen: the model never executes anything.

It emits a structured request. Your code decides whether to run it, runs it, and hands the result back. The model’s only capabilities are reading and writing text — tool calling is a protocol layered on top of that, not a new power.

Once this is clear, the security model and the failure modes both follow directly.

The exchange

You describe available tools in the request. The model, instead of answering, may emit a tool call.

1 · You declare tools, each with a name, a description, and a parameter schema:

{
  "name": "get_weather",
  "description": "Get current weather for a city",
  "parameters": {
    "type": "object",
    "properties": {
      "city": {"type": "string"},
      "unit": {"enum": ["celsius", "fahrenheit"]}
    },
    "required": ["city"]
  }
}

2 · The model requests a call rather than producing text:

{"tool": "get_weather", "arguments": {"city": "Tokyo", "unit": "celsius"}}

3 · Your code executes it. This is entirely on your side — the model is waiting.

4 · You return the result as a message in the conversation.

5 · The model continues, now with the result in context. It may answer, or call another tool.

Why it is reliable

Tool arguments come out well-formed far more consistently than free-text JSON, and there are two reasons.

Models are trained on this format specifically, so the pattern is deeply reinforced. And providers apply schema-constrained generation — the sampling step is masked so only tokens valid under your schema can be chosen. Malformed arguments become structurally difficult rather than merely unlikely.

This is why defining a “tool” purely to receive structured data is a legitimate pattern even with no function behind it.

The tool description is a prompt

The single highest-leverage thing to get right.

The model chooses tools based on their descriptions, and nothing else. A vague description produces wrong tool selection, and this is the most common source of agent misbehavior — more common than reasoning failures.

Descriptions must state what the tool does, when to use it, when not to, and what it returns. Two tools with overlapping descriptions will be confused with each other. See Writing Good Tool Definitions.

Parallel and sequential

Models can request several tools at once when the calls are independent — three cities’ weather in one turn, executed concurrently. Worth supporting, since it cuts latency substantially.

Dependent calls must be sequential: search, read the result, then fetch the specific document it named. The model cannot construct the second call before seeing the first result, which is exactly why the loop exists.

Security

Because your code does the executing, every security decision is yours. The model is an untrusted source of function arguments.

Validate arguments independently. Schema conformance is not authorization. A well-formed request to delete a record is still a request to delete a record.

Enforce permissions at execution. Check what the user may do, not what the model asked for. The model has no notion of who is asking.

Assume prompt injection is possible. Tool results enter the context as text. A retrieved document or fetched web page may contain text shaped like instructions. If a tool reads untrusted content, treat everything downstream as potentially influenced — this is the central unsolved problem in agent security, and there is no complete mitigation. Partial defenses: keep destructive tools out of reach of untrusted input, and require confirmation for consequential actions.

Gate irreversible operations. Sending, deleting, paying, deploying. Read-only tools are safe to let run freely; anything else warrants a human in the path.

Practical notes

Return errors as normal results. A failed tool should return {"error": "City not found"}, not raise. The model can then correct course — try a different spelling, ask the user. Raising the exception ends the loop and wastes the recovery ability you are paying for.

Keep results compact. Tool output goes into the context window every subsequent turn. A tool that dumps 10,000 tokens of JSON poisons the rest of the run. Return the fields that matter.

Fewer tools is better. Beyond roughly a dozen similar tools, selection accuracy degrades. Consolidate, or route to a subset based on the task.

Make them idempotent where possible. Retries happen.

What to remember

• The model emits a request; your code executes. It never runs anything itself.
• Schema-constrained generation makes tool arguments far more reliable than free-text JSON.
• The description is a prompt — vague or overlapping descriptions cause wrong tool selection, the most common agent failure.
• All security is yours: validate independently, enforce user permissions, gate irreversible actions.
• Tool results are untrusted text and can carry prompt injection.
• Return errors as results so the model can recover; keep output compact.

Next: The Agent Loop