Grounding

What is grounding?

Grounding is the property of an AI answer being tied to source material you supply, so every claim can be traced back to a specific document, database row, or tool result rather than coming from the model's own memory. When an answer is grounded, you can point at the passage it rests on.

Without grounding, a large language model (LLM) answers from parametric memory, the patterns it absorbed during training. That is fine for general explanation, but weak for anything that changes: prices, stock levels, last week's policy. Grounding makes the model work from the facts in front of it instead of what it half-remembers.

How grounding works in practice

Grounding is usually three moves stitched together.

1. Retrieve the right material. The application fetches context relevant to the question, from a search index, a vector database, a SQL query, or a live API call, and passes it to the model with the question.

2. Constrain the answer. The prompt tells the model to answer only from that material and to say so when the material does not cover the question. Azure OpenAI On Your Data exposes this as a "Limit responses to your data" setting: with it on, the model relies on your documents instead of adding its own knowledge, and gives an "I don't know" style reply when the documents fall short.

3. Return citations. The answer carries references back to the passages it used. Anthropic's citations feature returns the exact sentences that support each claim, while OpenAI file search and Google's Gemini attach the filename or link and the character span behind each statement, so a reader can check the source.

Deciding what to retrieve and how to frame it is the work of context engineering, and the answer-only-from-the-source instruction usually lives in a reusable prompt template.

Grounding versus RAG

These two get used as synonyms, but they sit at different levels. RAG (retrieval-augmented generation) is an architecture: it retrieves relevant chunks, often from a vector database, and injects them into the prompt. Grounding is the property that the answer is actually anchored in those chunks.

A RAG pipeline is the most common way to ground an answer, but it is not the only one. A single API call to your ERP or a SQL result from your data warehouse grounds an answer just as well. And the reverse: you can build a full RAG system and still get an ungrounded reply if the model ignores the retrieved text and answers from memory. RAG is the plumbing; grounding is whether the water comes through.

What to watch out for with grounding

Grounding lowers the hallucination rate, but it does not remove it. The model can still misread a retrieved passage, attribute a fact to the wrong source, or blend two snippets into a claim neither one makes. A citation proves the passage was retrieved, not that the sentence built on it is faithful.

Two things follow. First, source quality decides answer quality: ground a model on an outdated procedure and it will answer, confidently and with a citation, from the wrong procedure. Second, for high-stakes answers you still want a human to check the cited source before it reaches a customer. Grounding narrows where the model can go wrong; it does not close the door.