System prompt

What is a system prompt?

A system prompt is the instruction an application hands a language model before anyone types a message. It sets the assistant's role, scope, tone, and the rules every answer has to follow. The person chatting rarely sees it, yet it shapes every reply.

It also stays fixed for the whole conversation. User messages come and go while the same system prompt sits underneath all of them. The user decides what a reply is about; the system prompt decides how the assistant answers it. Every LLM chat product has one, and when you build your own AI agent, the instruction field you fill in is its system prompt.

How the system prompt reaches the model

To the model, the system prompt is just more text in its context window, tagged with a role label. What sets it apart is that models are trained to weight system instructions above the user's, so a rule set once keeps applying without being repeated.

Vendors expose it differently. Anthropic's Messages API takes it as a dedicated system parameter, separate from the user and assistant messages, and advises keeping the role there and task detail in the user turn. OpenAI's newer models take the same instructions under a developer role instead of a system role, following the chain of command in its model spec, where the system role now belongs to OpenAI itself. Writing that instruction well is the craft of prompt engineering, alongside prompt templates, structured output and context engineering.

What goes into one

Microsoft's guidance for Azure OpenAI groups a system prompt into a few parts:

• Role and task. Who the assistant is and what it is responsible for.

• Tone and audience. Formal or casual, short or thorough, and which language to answer in.

• Scope and boundaries. What it handles, what it refuses or hands off, and what to say when it cannot help.

• Tools and data. Which systems it may call, and what it never does without confirmation.

• Output format. Free text, a fixed structure, or JSON a downstream system can parse.

Filled in, one reads like a staff briefing: "You are the support assistant for Fietsdelen.be. Answer in Dutch, in four sentences at most. Handle only orders, delivery and returns; send billing questions to accounting. Never promise a delivery date that is not in the order data." Vague rules leave the model to guess, differently each time, so concrete instructions win.

Why it is not a security boundary

The model weights the system prompt above the user, but that preference lives in the training, not in a lock. A user, or a hidden instruction planted in a document the assistant reads, can push the model into ignoring its own rules or repeating them back verbatim. This is prompt injection, and it is why the system prompt is the cheapest of your guardrails and also the weakest.

OWASP is blunt about it: a system prompt should not be treated as a secret or used as a security control. Keep credentials, API keys, connection strings and internal customer data out of it, and enforce who may do what in systems the model cannot reach. For an internal assistant that only writes text, the prompt alone is often enough; once tools, customer data or public users can touch it, you add real controls on top.