Chain-of-thought prompting

What is chain-of-thought prompting?

Chain-of-thought prompting, often shortened to CoT, is a prompting technique where you ask a language model to reason through intermediate steps before giving the final answer. Instead of asking only for the result, you ask for a worked path to the result.

The well-known zero-shot version is a short instruction such as think step by step or work this out before answering. The model then produces intermediate steps and usually arrives at a more reliable answer on tasks that need arithmetic, symbolic reasoning, or several dependent decisions.

The original research in 2022 showed large improvements on reasoning benchmarks when large models were given worked examples or even a simple step-by-step cue. The important caveat is that the effect depends on the model and the task. It is useful for reasoning; it is usually wasted on simple lookup or classification.

Zero-shot and few-shot CoT

Zero-shot chain-of-thought
You give the model the task and add a short instruction to reason step by step. No examples are included. This is the lightest version and is often enough for everyday use.

Few-shot chain-of-thought
You give the model a few examples that include the question, the reasoning pattern, and the answer. The model imitates the structure on the real question. This is useful when you want a specific checklist, calculation style, or decision policy.

Few-shot CoT costs more tokens and takes more care. Start with zero-shot. Add examples only when you need the model to follow a specific reasoning shape.

When chain-of-thought helps

• Calculations. Margin, VAT, capacity, time windows, or any question where one arithmetic slip changes the result.

• Multi-step decisions. For example: classify a customer request, check exceptions, choose the next action, and explain the handoff.

• Code and debugging. Walking through state changes can expose where a function breaks.

• Policy or compliance checks. A visible checklist can make it easier to audit whether the answer used the right criteria.

It helps most when the answer depends on a sequence. It helps least when the model only needs to retrieve a fact already present in the context.

When not to use it

Do not add chain-of-thought to every prompt. It increases output length, latency, and cost. It can also make a simple answer harder to read.

For simple classification, extraction, or retrieval, ask directly for the result and maybe a short reason. For example: Return the category and one sentence explaining why. That gives enough checkability without forcing a long reasoning trace.

For customer-facing chatbots, raw step-by-step reasoning is rarely what the user wants. A concise explanation or cited source is usually better.

Chain-of-thought and modern reasoning models

Newer reasoning models often spend internal tokens on reasoning before producing the final answer. Some platforms expose summaries or configurable reasoning effort, but they do not necessarily expose raw hidden reasoning. In practice, that changes how you prompt.

Instead of asking a reasoning model to reveal every internal step, ask for the output format you need: a final answer, a short rationale, assumptions, checks performed, or a concise audit trail. That keeps the answer useful without depending on raw chain-of-thought text.

For standard non-reasoning models, CoT-style instructions can still help. For reasoning models, use the model's reasoning controls where available and ask for a clear summary of the result.

What to watch out for with chain-of-thought prompting

Long reasoning is not proof
A detailed explanation can still be wrong. Check calculations, sources, and assumptions.

It can expose irrelevant detours
Some models ramble when asked to think step by step. Ask for concise steps or a checklist if length matters.

It may leak sensitive context
In regulated or customer-facing settings, be careful about displaying internal policy reasoning, confidential facts, or private data.

Use the right level of explanation
For an analyst, a calculation trail is useful. For a customer, a short answer with the source may be better.