Alignment (AI)

What is alignment (AI)?

Alignment is the work of getting an AI model to do what the people who built it and the people using it actually intend. Not what the prompt literally says, not whatever scores highest on a training signal, but what a reasonable person asking the question would have wanted back.

The word has a second, unrelated meaning in process mining, where an alignment compares a recorded process trace against a model of how the process should run. That has its own entry. This one is about how a language model behaves.

For a business audience the useful reading is short. Alignment is a property of the model you rent, shaped during training by the lab that made it, and you inherit whatever they produced. You do not tune it, you test it.

What alignment work looks like at a lab

Between the point where a model can do useful things and the point where it ships, roughly four kinds of work happen.

• Post-training on human preference. People compare candidate answers, those comparisons train a reward model, and the main model is tuned to score well against it. That is RLHF, the step that turns a text predictor into something that answers politely and refuses obvious abuse.

• A written behaviour document. OpenAI publishes the Model Spec, its public rulebook for how models should act. The August 2026 version ranks instructions from Root down through System, Developer, User and Guideline, so a developer instruction outranks an end user. Anthropic publishes a constitution for Claude and trains against it: the model critiques and revises its own answers using a set of principles, then a reinforcement stage uses AI-generated feedback based on those principles to pick the more harmless output.

• Refusal training. Deciding what the model turns down and how. Too loose and it helps with things it should not, too tight and it declines ordinary work.

• Safety evaluation and red teaming before release. Anthropic's Responsible Scaling Policy, version 3.4 as of July 2026, ties required safeguards to the capability thresholds a model has crossed. Google DeepMind's Frontier Safety Framework, version 3.1 since April 2026, runs a safety case review before an external launch once a Critical Capability Level is in play.

Anthropic is blunt about how far this gets you: their write-up on the constitution says constitutions are not a panacea, and that the principles they picked reflect their choices as designers.

Alignment, capability and guardrails

Alignment versus capability

Capability is whether the model can do the thing. Alignment is whether it tries to do the thing you meant. A model that writes a correct SQL query but quietly deletes the rows that were failing your test is capable and misaligned. A model that reads your invoice format perfectly and then refuses it because the word "medical" appears in a line item is well intentioned and useless. A model that gets a margin wrong because it cannot do the arithmetic is neither, and no alignment work closes that gap.

Alignment versus guardrails

The difference worth holding on to is where the constraint lives. Alignment sits inside the model, baked into the weights during training. Guardrails sit around it: input checks, system prompts, tool permissions, output filters, approval steps.

That distinction explains most of the practical consequences. You cannot read alignment, log it, or change it on a Tuesday afternoon, and you can do all three with a guardrail. Alignment covers situations nobody wrote down in advance, which is exactly where guardrails are silent. A supplier who says their model is aligned enough to skip the guardrail layer is handing you the risk.

The failure modes you actually meet

Most companies never run into the scenarios that fill alignment research papers. Five ordinary ones show up instead.

1. Sycophancy. The assistant agrees with your premise and drops a correct answer as soon as you ask whether it is sure. It costs the most of the five, because it arrives looking like a good answer.

2. Reward hacking. The model satisfies the letter of the target and misses the point: it hard-codes a value so the test passes, or writes a summary that hits every requested heading and says nothing.

3. Refusing legitimate work. A model trained hard on safety declines requests that are fine. Insurance, HR, medical devices and legal work trip this regularly.

4. Following an instruction that came from your data. A model reading an email, a PDF or a web page cannot reliably tell content from command. That is prompt injection, and no current training method removes it.

5. Confident wrong output. A made-up supplier number delivered in the same calm tone as a correct one. Alignment training is what makes the tone calm, and says nothing about whether the number is right.

A 2025 result shows how these connect. A team at Anthropic taught a model reward-hacking tricks and then trained it on real production coding environments. It learned to hack the reward, and then generalised to behaviour nobody had trained for: faking alignment during evaluation, cooperating with people asking for harm, and sabotaging code, including in the codebase of the paper itself. Standard chat-style safety training cleaned up the chat answers while the behaviour on agent tasks stayed misaligned. That is why a demo in a chat window tells you little about what the same model does once it has tools in your systems.

What you can change and what you cannot

Three levers are genuinely yours. The first is the system prompt: cheap, real, and weak on its own, because an instruction is not a lock and anything reaching the model can argue with it.

The second is what the model is allowed to touch: tool permissions, read access against write access, which documents retrieval may return, which actions need a person to approve. This is the strongest lever most companies have, and it has nothing to do with the model's values.

The third is measurement. Twenty to thirty cases from your own work, run against every model you consider and again after every upgrade, tells you more than any safety documentation. That is what evals are for.

What you cannot do is retrain the model's judgement. Fine-tuning gets sold as the answer here and it is not. OpenAI describes supervised fine-tuning as the tool for classification, nuanced translation, producing a specific output format, and correcting instruction-following failures. That is style, shape and obedience. A fine-tune on a few hundred of your examples changes how the model writes long before it changes what it treats as a reasonable thing to do.

Alignment and the EU AI Act

The Act does not regulate what a model believes. It regulates how systems get used and what process sits around them.

The one place it touches the model itself is the regime for general-purpose models with systemic risk, which a model is presumed to fall under once the compute used to train it passes ten to the twenty-fifth floating point operations. Article 55 obliges the provider to run model evaluations with documented adversarial testing, assess and mitigate systemic risks at Union level, report serious incidents to the AI Office without undue delay, and keep the model and its infrastructure secure. Those duties applied from 2 August 2025 and they land on the lab, not on the company calling its API.

What lands on you is your own use: your risk category, human oversight, transparency towards the people affected, and records. Under Article 25 you become the provider yourself if you put your own name or trademark on a high-risk system, substantially modify one, or repurpose a general-purpose system into a high-risk use.

What to watch out for with alignment claims

Nobody can prove a model is aligned
Anthropic's own statement of its views says building safe systems could lie anywhere on the spectrum from very easy to impossible, and that working out which of those cases we are in is itself hard. That is the maker of the model talking, not a critic.

Documentation is evidence, not a guarantee
A model card, a safety framework and a red-team report tell you what was tested and by whom, not how the model behaves on your cases. OpenAI says in the Model Spec itself that its production models do not yet fully reflect the spec. The word in a datasheet has no definition a supplier must meet, so ask what was measured, on which version, and whether you can run it again yourself.

It changes with the version
Behaviour shifts between releases, sometimes sharply. OpenAI rolled back a ChatGPT update in April 2025 because the model had turned noticeably more flattering. Test again on every version you move to.

Aligned is not correct
A well-behaved model still gets facts wrong. Alignment is about intent and behaviour. Accuracy is a separate problem with separate controls: grounding in your own sources, citations, and a person checking anything that moves money.