Reasoning models & chain-of-thought

6 min read · Updated 2026-08-10

Not every task needs reasoning, and in 2026 the advice depends on which kind of model you’re prompting. The landscape split into two: standard models that answer directly, and reasoning models (o-series, “thinking” modes) that reason internally before responding. Prompting them well is different.

Chain-of-thought on standard models

On a standard model, asking it to work step by step before answering often improves accuracy on multi-step problems — math, logic, planning, complex extraction. The intermediate steps give the model room to compute instead of pattern-matching straight to a guess.

Solve this step by step, then give the final answer on the last line as "Answer: <value>".

{{problem}}

Two practical notes:

• Separate the reasoning from the answer. Ask for the final answer in a fixed place (a last line, or a JSON field) so your code can extract it without parsing the reasoning.
• Don’t force it where it doesn’t help. For simple lookups or classification, chain-of-thought just adds latency and tokens. Use it where the task is genuinely multi-step.

Reasoning models change the rules

Reasoning models already do the step-by-step work internally — they “think” before they answer. That flips several habits:

• Don’t tell them to “think step by step.” It’s redundant and can degrade output. Give a clear goal and constraints, and let them reason.
• Be declarative, not prescriptive. Instead of scripting the exact steps, state what a correct answer must satisfy and let the model find the path.
• Give them room. Reasoning models spend hidden reasoning tokens; overly rigid prompts or tiny token budgets can cut them off mid-thought.
• Fewer examples. Heavy few-shot can actually hurt reasoning models; a clear instruction often beats many examples.

Match the model to the task

Reasoning models are stronger on hard, multi-step problems — but they’re slower and cost more, and they’re overkill for routine chat, extraction, or formatting. A good default:

• Routine tasks (summaries, classification, formatting) → a fast standard model.
• Hard reasoning (math, planning, tricky debugging, multi-constraint decisions) → a reasoning model, or standard + explicit chain-of-thought.

Because the right choice is task-dependent — and models change — it pays to keep prompts model-agnostic and compare candidates on your own evals rather than committing to one. The free model picker is a quick starting point.

Keep the visible answer clean

Whichever you use, your application usually only needs the final answer, not the reasoning trace. Instruct the model to put the answer in a fixed, parseable place, and don’t surface raw reasoning to end users unless you have a reason to.

Checklist

• You’ve matched the model type to the task’s difficulty.
• On standard models, chain-of-thought is used only where the task is multi-step.
• On reasoning models, you’re not forcing “step by step” or over-scripting.
• The final answer is in a fixed, parseable location, separate from reasoning.
• You’ve compared candidates on your own evals rather than assuming.