Tool use (function calling)

What is tool use (function calling)?

Tool use, which most model vendors call function calling, lets a language model do more than produce text. You give the model a set of tools it is allowed to call, each one a function or an API in your own systems, and the model decides on its own when one is needed and which one fits.

The part people get wrong: the model does not run anything. It reads your tool descriptions and returns a structured request that names the tool it wants and the arguments to pass. Your code executes that request and hands the result back, and the model carries on with the result in context. The LLM decides, your application does the work and holds the permissions.

A tool call is a form of structured output: instead of free text, the model returns JSON that matches a schema you defined. You describe the tools in the same request as your system prompt and any prompt template.

How the tool-use loop works

A single round of tool use has four moves, and in a real conversation the loop repeats until the model has enough to answer.

First you describe each tool: a name, a plain-language description of what it does and when to use it, and a schema for the arguments it expects. That description is what the model reads to decide when a tool fits, so vague names and thin descriptions are where things go wrong. A tool definition is usually a small piece of JSON:

{
  "name": "get_order_status",
  "description": "Look up the current status of a customer order by its order number. Returns the shipping state and expected delivery date.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_number": {
        "type": "string",
        "description": "The order number, for example ORD-10432"
      }
    },
    "required": ["order_number"]
  }
}

Next the model chooses. Given a question like "where is order ORD-10432?", it returns a call to get_order_status with order_number set to ORD-10432, rather than a text reply. Your code catches that call, runs the real work (a database query, a call to your order API), and gets back something like "shipped, arriving tomorrow". You feed that result back into the conversation, and the model writes the final answer. It never touched your database, it only asked.

When one result leads to another, the loop runs again: the model might see the order is late, then call a second tool to draft an apology email. Each pass, your code runs the tool and returns the result, until the model stops calling tools and answers.

Parallel and multi-step tool calls

A model is not limited to one tool at a time. Ask for the weather and local time in San Francisco, Tokyo, and Paris, with a get_weather tool and a get_current_time tool available, and the model can return six calls in a single turn: three for weather, three for time. Your code runs them, returns all six results, and the model composes one answer. Running independent calls together is faster than forcing them one after another.

Multi-step tasks chain calls instead. The model calls one tool, reads the result, decides the next step, and calls again. That loop, a model calling tools and reacting to what comes back, is the engine inside every AI agent. Without tool use a model is a text generator; with it, the model can pull live data and drive a task from start to finish.

Function calling, MCP and RAG

Every model vendor has its own notation for tool use, so a tool you wire up for one model has to be redefined for another. MCP (Model Context Protocol) puts a standard layer on top: you describe a tool once on an MCP server, and any model that speaks MCP can use it. Function calling is still what happens underneath, it is how the model asks to run a tool; MCP is how the tool is exposed, so every model sees the same definition.

RAG (Retrieval-Augmented Generation) fits the same picture. You can treat its retrieval step as one tool, a search over a knowledge base. The difference is who decides: classic RAG runs that search on every question, while with function calling the model decides whether a search is needed at all.

What to watch out for with function calling

The model decides which tool to call and what to pass it, and it can get both wrong. A few things to guard against:

• Bad tool choice. The model picks a tool from its name and description, so if two tools look alike or a description is vague, it can reach for the wrong one. Clear names and detailed descriptions matter more than almost anything else.

• Hallucinated arguments. If a question does not contain everything a tool needs, the model may invent a plausible value rather than ask for it. Validate every argument before you execute: check types and ranges, and confirm the record exists before you act on it.

• Destructive actions. The model only sends a request; your code decides what happens. A tool that writes data, sends a message, or takes a payment belongs behind scoped permissions and, for anything you cannot undo, a human confirmation step before it runs.

• Too many tools. Every tool you offer takes up space in the context window and gives the model one more option to confuse. Keep the set per task small and group tools by use case.

• Untrusted tool output. A tool that returns text from email, web pages, or documents can carry hidden instructions aimed at the model. This is a prompt injection risk: treat outside content as data, not as commands.