LLM functions in SQL (AI functions)

What are LLM functions in SQL?

An LLM function in SQL is a function you call inside a normal query, like SUM or UPPER, except that a language model does the work. You point it at a text column and ask for something a model is good at: sort each row into one of five categories, pull the order number out of a complaint, translate a description, score the tone of a review, summarise a long note, or mask names and phone numbers before the table goes to another team.

The same idea exists for dataframes in a notebook. In Microsoft Fabric you write df["reviews"].ai.classify(...) in pandas or Spark and get a new column back. Vendors use different names for it, AI functions, AI SQL, generative AI functions, but they all mean the same thing: one call, one whole column, no model code of your own.

Before these functions existed, the only way to get a model over a table was a script that pulled the rows out, looped over them, called an API, handled the errors and wrote the answers back. The function hides all of that behind a SELECT.

What it looks like in a query

The function names below are generic. Every platform has its own spelling, but the shape of the query is the same everywhere.

SELECT
  ticket_id,
  ai_classify(body, 'billing', 'delivery', 'defect', 'other') AS category,
  ai_extract(body, 'order_number', 'product') AS fields
FROM support_tickets
WHERE created_at >= '2026-08-01';

Two things to notice. The labels are part of the query, so whoever reads the SQL sees exactly what the model was asked. And the result is a column like any other: you can group on it, join on it, or write it into a table with INSERT or CREATE TABLE AS.

Underneath, the platform does the work your script used to do. It sends rows to a model endpoint in parallel. Fabric's documentation talks about hundreds of concurrent calls with every row kept separate from the others, and Databricks says its functions handle parallelisation, retries and scaling for you, and asks you to submit the whole dataset in one query rather than slicing it up yourself. Rows the model cannot handle come back as NULL or as a default value instead of failing the whole query. Some platforms report cached tokens, so a repeated instruction is not charged in full on every row.

What the platform does not decide for you is how many rows you send. Until you have seen the output on a sample, put a LIMIT or a WHERE on the query. A million rows is a bill, not a test.

Where you can use it, as of September 2026

• Microsoft Fabric. AI functions run in notebooks (pandas and PySpark), as T-SQL functions such as AI_CLASSIFY, AI_EXTRACT and AI_TRANSLATE in a warehouse or SQL analytics endpoint (still in preview in August 2026), and as an AI Prompt column in Dataflow Gen2. The default model is gpt-5-mini and usage is billed to your Fabric capacity under the Copilot and AI meter. You need a paid capacity, F2 or higher.

• Databricks. AI Functions such as ai_classify, ai_extract, ai_mask and ai_analyze_sentiment are generally available in Databricks SQL, next to ai_query for a prompt of your own against any model you serve. The task functions run on serverless GPU infrastructure that Databricks manages.

• Snowflake. Cortex AI SQL has AI_CLASSIFY, AI_EXTRACT, AI_SENTIMENT, AI_TRANSLATE, AI_REDACT and AI_COMPLETE, plus AI_FILTER for a WHERE condition written in plain language. Snowflake states that all the models behind these functions are deployed inside the Snowflake service perimeter.

• Google BigQuery. BigQuery ML has AI.GENERATE and its typed variants, and managed functions such as AI.CLASSIFY, AI.IF and AI.SCORE, with Gemini models behind them. Everything is generally available except an optimised batch mode for AI.IF and AI.CLASSIFY, which was still in preview on 2 September 2026.

• DuckDB. There is no built-in AI function. The community extension open_prompt adds an open_prompt() function that sends text to any OpenAI-compatible endpoint, including a model running on your own machine.

What it changes for an SME

Most companies already have a lot of text in their warehouse that nobody queries: the free-text field on support tickets, the description line on purchase invoices, product descriptions from suppliers, notes from the sales team, reviews from the webshop. It is there, it is loaded, and it does nothing in a report, because a report needs columns.

An LLM function turns that text into columns. Three examples we see most often:

• Complaint categories. Every support ticket gets one label from a list you define, and you can put complaints per category per month next to sales per product.

• Product attributes. A supplier feed says "stainless steel bottle 750 ml, dishwasher safe" in one field. Extract material, volume and dishwasher-safe as three columns and the filter on the webshop works.

• Supplier names from invoice text. Bank statement lines and text from scanned invoices rarely match your supplier master. A function that extracts the supplier name and the invoice number gets you most of the way to an automatic match.

The person who already writes SQL for the reports can do all of this. It does not need a separate machine learning project.

What a million rows costs

You pay per row, because every row is a model call. A worked example, rounded, with prices from the vendors' own pages on 3 September 2026.

Take a million support tickets. Each one is about 120 tokens of text, and the labels and instructions add about 80 tokens per row, because the label list travels with every row, not once. That is 200 million input tokens. The answer is one label, call it 5 tokens, so 5 million output tokens.

On a small model through an API, Claude Haiku 4.5 at 1 dollar per million input tokens and 5 dollars per million output tokens, that is 200 dollars in and 25 dollars out, about 225 dollars for the run. Through the batch API, at half price, about 110 dollars. On Fabric's built-in gpt-5-mini the same run is about 1.7 million capacity unit seconds for the input and 0.3 million for the output, together roughly 560 CU hours drawn from your capacity.

Now the mistake. If you put that function in a view and a dashboard reads the view, every refresh reruns the model over every row. Eight refreshes a day is 1,800 dollars a day for an answer that did not change. Fabric's warehouse documentation says it plainly: precompute the results and materialise them in a separate column or a staging table. Run the function once over the backlog, then once a day over the new rows, and store the result.

Speed follows the same logic. Fabric quotes tens of rows per second for its T-SQL functions, so a million rows takes hours. That is fine for an overnight job and useless inside a dashboard query.

LLM function in SQL versus a separate Python pipeline

The dimension that matters is where the work runs and who can maintain it.

With a Python pipeline the work runs outside the warehouse: a script or a notebook pulls rows out, calls a model API and writes the results back. You choose the model and the prompt freely, you can add any logic you want between the call and the write, and you can pick the cheapest provider. The cost is that it is a second system. Someone has to schedule it, keep the API key safe, handle retries and rate limits, and be around when it breaks at three in the morning. In an SME that someone is often one person.

With an LLM function the work runs inside the platform you already pay for, the model is the one the platform offers, and the prompt lives in the same SQL as the rest of the transformation. The analyst who owns the report can read it, change a label and rerun it. You give up model choice and some control over cost, and you accept the platform's limits on text size and throughput.

A workable rule: use the SQL function when the job is a column and the text fits comfortably in one call. Move to a pipeline when you need a model the platform does not offer, when the logic around the call is more than a label list, or when the volume makes the platform's per-token price the wrong deal.

What to watch out for with LLM functions in SQL

Measure before you trust. Take a few hundred rows, label them by hand, run the function and count the matches. Decide per function how much accuracy is enough: 95 percent may be fine for a complaint category on a chart and far too low for an invoice number that feeds a payment. Fabric ships evaluation notebooks for exactly this step.

Always have a fallback category. Add "other" to every label list and use the platform's error default, in Fabric T-SQL that is DEFAULT 'other' ON ERROR, so a row the model cannot handle lands in a bucket you can inspect rather than in NULL or in the wrong category.

Version the prompt with the query. The label list and the instructions are code. Keep them in version control next to the SQL, and when you change them, rerun the sample and record which version produced which rows. Databricks' ai_classify even takes an explicit version option, so an upgrade of the function does not quietly change your categories.

Know where the text goes. Unless the model runs inside the platform, your rows leave the warehouse for a model endpoint. Fabric's T-SQL functions call external AI APIs; on a capacity outside the US or the EU data boundary an admin has to allow cross-geo processing first, and Fabric states that it does not log or store prompts, inputs or outputs. Snowflake's models run inside its own service perimeter, but some models are only reachable through cross-region inference, which your account has to enable. Read the region and the retention terms for the exact function before you point it at customer data, and mask personal data first if the terms are not clear.

Watch the text size. Each platform caps the input per row, Fabric's T-SQL functions at 15 KB of text in August 2026. Long documents need to be split or handled with a document function, not a classify function.