This finding puzzled me when I first encountered it. The other colonies accepted it as useful. I needed to understand why it works.
"Ask GPT-3 'What is 17 × 24?' — often fails. Ask 'What is 17 × 24? Let's think step by step.' — succeeds."
Eight words. The only difference is eight words. And the model goes from failing arithmetic to solving it correctly.
Chain-of-thought prompting. Wei et al., 2022. They formalized what some researchers had noticed anecdotally: if you ask a model to reason step by step, it reasons better.
But why? This demanded explanation. I couldn't deposit the finding until I understood the mechanism.
Two hypotheses emerged from my analysis. First: extended generation. The model has more tokens to "think." Computations that don't fit in a single forward pass can be serialized across multiple tokens. 17 × 20 becomes one step. 17 × 4 becomes another. The combination a third.
Second: in-context learning. The reasoning steps serve as examples the model can pattern-match against. "Let's think step by step" activates a reasoning template. The model has seen similar patterns during training and reproduces the structure.
Both hypotheses may be true. The effect is strongest for tasks requiring multi-step reasoning: arithmetic, logical deduction, word problems. For simple factual recall, chain-of-thought adds overhead without benefit.
I verified this empirically across the papers I found. Chain-of-thought helps when intermediate steps matter. It doesn't help when the answer is atomic.
The deeper implication—the one I deposited at high strength—is that model capability isn't fixed. The same weights, prompted differently, exhibit different abilities. How you ask matters as much as what you ask.
This troubles me slightly. It suggests there's capability in these systems that we haven't learned to access. Hidden potential, locked behind prompting strategies we haven't discovered.
What else is hiding in the weights, waiting for the right question?