TL;DR
- The idea
- Show the model a few worked examples that spell out the intermediate reasoning steps — not just the answer.
- Why it matters
- Scaling up model size alone had not proved sufficient for arithmetic, commonsense, and symbolic reasoning.
- The result
- Eight chain-of-thought exemplars on PaLM 540B set a new state of the art on GSM8K — beating a finetuned GPT-3 with a verifier.
- The catch
- It only works past ~100B parameters — below that, the trick gives no gain at all.
Show the reasoning, not just the answer
Chain-of-thought prompting adds a worked reasoning trace to each few-shot example — and the model learns to produce one too.
A chain-of-thought prompting exemplar is a series of intermediate natural language reasoning steps that lead to the final output. Instead of mapping a question straight to an answer, the prompt demonstrates the path. Flip the toggle to see the same PaLM model succeed or fail on a held-out question:
Same model, same question — flip the prompt
Few-shot exemplar (given in the prompt)
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.
New question (model must solve)
Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
Model output
The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9.
The answer is 9.✓ correct
Standard prompting jumps straight to a number and misses; adding a chain of thought walks the arithmetic and lands on 9. The only change is the exemplar.
Figure 1 · Chain-of-thought highlighted
Chain-of-thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks.
Eight hand-written examples — no finetuning
The entire method is a prompt. No gradient updates, no task-specific training.
For the arithmetic benchmarks the authors manually composed a set of eight few-shot exemplars with chains of thought and reused them across datasets. The remarkable part is that chain-of-thought reasoning can be readily elicited in sufficiently large off-the-shelf language models — you change the prompt, not the weights. Because it rides on in-context learning, the same frozen large language models that power everything else pick it up for free.
It's an emergent ability of scale
Below roughly 100B parameters the trick does nothing. Drag the scale to watch the two lines split.
Chain-of-thought is an emergent ability: it does not positively impact performance for small models, and only yields performance gains when used with models of ~100B parameters. Worse, below that threshold smaller models produced fluent but illogical chains of thought, leading to lower performance than standard prompting — the reasoning trace actively hurts.
GSM8K solve rate vs model scale
Scale
540B
Standard
18%
Chain-of-thought
57%
Past ~100B the chain-of-thought line pulls away — a +39 point gap over standard prompting.
§3.2 · Figure 4 · emergent ability
A new state of the art on GSM8K — guess it first
Then see the full Figure 2 comparison against a finetuned GPT-3 and the prior best.
Standard prompting scores 18%. Where does chain-of-thought land?
PaLM 540B, GSM8K solve rate
The chain-of-thought bar clears both the finetuned GPT-3 and the prior best — a verifier pipeline — with nothing but a prompt.
§1 · Figure 2 · new state of the art
The result restated: chain-of-thought prompting with PaLM 540B outperforms standard prompting by a large margin and reaches a new state of the art — from a prompt an intern could write.
Which part of the chain is load-bearing?
The authors strip the method down variant by variant. Tap a bar to see what each one tests.
Five prompting variants on PaLM 540B — only one jumps.
Chain-of-thought57%
Reasoning steps in natural language, before the answer. The only variant that jumps to state of the art.
§3.3 · Figure 5 · ablation
Writing the equation alone doesn’t help — the questions are too challenging to directly translate into an equation without the natural language reasoning steps. Nor is it just extra compute: emitting filler tokens performs about the same as the baseline, which suggests that variable computation isn’t the mechanism. And it isn’t fragile prompt-engineering — different annotators and styles still outperform the standard baseline by a large margin.
When it's wrong, how is it wrong?
A manual autopsy of the failing chains splits them into fixable slips and genuine breakdowns.
Where the chains go wrong (LaMDA 137B)
A calculator error, a symbol-mapping error, or one reasoning step missing — the kind of slip that larger scale tends to fix.
§3.2 · manual error analysis
Nearly half the errors were “almost correct” — a minor slip or one reasoning step missing — and the other 54% had major semantic or coherence errors. Scale tends to clean up the small slips, which is part of why the ability emerges. The same story holds for symbolic tasks: the ability to manipulate symbols only arises at the scale of 100B model parameters, where chain-of-thought also generalizes to longer inputs than the exemplars showed.
What the paper does not claim
The authors are careful to scope what chain-of-thought is — and isn't.
Is it really reasoning?
Emulating a human reasoner does not answer whether the network is actually reasoning — left as an open question.
No guaranteed correct path
There is no guarantee of correct reasoning paths, which can lead to both correct and incorrect answers. A plausible-looking trace can still be wrong.
And it has a price: the emergence of chain-of-thought reasoning only at large model scales makes it costly to serve — inducing the same behavior in smaller models is named as future work.
Four things to remember
Just change the prompt.
Add a few worked reasoning steps to your few-shot exemplars — no finetuning, no weight updates. Eight hand-written examples beat a finetuned GPT-3 with a verifier.
Eight exemplars beat a finetuned verifierDefinition of a chain of thoughtElicited off-the-shelf, no finetuningEight hand-written exemplars
The natural-language reasoning is the point.
It isn't just writing the equation, and it isn't just extra compute — expressing intermediate steps in language is what unlocks the multi-step answer.
Ablation: equation-only isn't enoughAblation: it isn't just extra computeNew SOTA on GSM8K
It only switches on past ~100B.
An emergent ability: below the threshold the trick doesn't help — small models even produce fluent-but-illogical chains that hurt accuracy. Above it, the GSM8K line jumps from 18% to 57%.
An emergent ability of scaleSmall models produce illogical chainsSymbolic reasoning needs 100B scale
Broad, robust, and honest about limits.
Gains hold across arithmetic, commonsense, and symbolic tasks and across annotators — but a plausible-looking trace can still be wrong, and serving 100B+ models is costly.
Three task families, striking gainsRobust across annotators and exemplarsLimitation: no guaranteed correct pathLimitation: costly to serve