Foundation model

What is a foundation model?

A foundation model is a large AI model trained on a broad sweep of general data, then adapted to handle many different tasks. You do not train a fresh model for every problem. You train, or more often reuse, one general base and steer it toward translation, summarising, classification, image captioning, code, or whatever you need.

The base models behind today's chat assistants are foundation models. A Large Language Model like the ones from OpenAI, Anthropic, Google, or Mistral is a foundation model trained on text. Vision models trained on huge image collections are foundation models too. The same base can answer questions, draft copy, pull fields out of an invoice, and sort support tickets, without a new model for each job.

The name came from Stanford's Center for Research on Foundation Models in 2021. They picked the word on purpose: the model is the layer everything else gets built on, not the finished product. What reaches your users usually sits on top of that base.

How a foundation model is built

Two phases matter. The first is pretraining: the model works through an enormous amount of general data and learns to predict the next piece. For a text model that means predicting the next word, over and over, across books, code, and web pages. Nobody hand-labels this data. The text is its own answer key, which is why the approach scales to volumes no labelling team could ever tag by hand. This is called self-supervised learning.

After pretraining, the model has broad skill but no particular job. The second phase is adaptation: you point that general skill at a specific task.

How you adapt a foundation model

There are three common routes, cheapest first.

1. Prompting. You describe the task in the prompt, sometimes with a few examples. Nothing gets retrained. This is in-context learning, and it covers a surprising amount of ground.

2. Retrieval. You fetch relevant documents and place them in the prompt so the model answers from your own data. This is the pattern behind retrieval-augmented generation.

3. Fine-tuning. You continue training the base model on your own examples so it adjusts its weights for your task, tone, or format. This is transfer learning at work: the broad pretraining is the starting point, and you carry that knowledge into a narrower use.

Foundation model versus a task-specific model

A task-specific model is trained from scratch for one job: a spam classifier, a demand forecaster, a model that reads one type of form. It can be small, fast, and cheap to run, and for a narrow, stable job it is often the better pick.

A foundation model starts general and gets nudged toward the task. You trade some efficiency for flexibility: one base covers many jobs, and you can stand up a new use case with a prompt instead of a new training project. The cost is size. Foundation models are large, so each call costs more and often runs through an external provider's API.

Common types

Foundation models now cover several kinds of data. Text models power writing and question answering. Image models generate or interpret pictures. Speech models handle audio. Multimodal models take more than one kind of input at once, for example reading an image and answering a question about it. What they share is the same recipe: broad pretraining first, then adaptation.

What to watch out for with foundation models

You inherit the training data. Whatever gaps, dated facts, or bias sit in the pretraining data come along for the ride. Bias in the source data becomes bias in the base, so a general model is only as balanced as what it read.

General is not the same as correct. A fluent answer can still be wrong. Foundation models can state made-up facts with full confidence, so anything that matters needs checking or grounding in your own data.

Cost and dependency. Running a large model per call adds up, and leaning on one provider's API ties you to their pricing, limits, and roadmap.

Data leaving the building. Sending prompts to an external model means sending your data there too. For personal or confidential data, check where it goes and whether you need a self-hosted or open-weight option instead.

Regulation. The EU AI Act sets specific obligations for general-purpose AI models and for how they are used in higher-risk settings. If you build on a foundation model, work out early which duties fall on the provider and which fall on you.