Why AI rarely answers the same way twice
AI writes each answer one word at a time, picking from a short list of next words, with a little chance built in on purpose.
Plays from YouTube when you tap
- File
- DA-C005
- Published
- 25 Oct 2026
- Runtime
- 1:13
- Sources
- 04
- The detail nobody notices
- The AI never looks an answer up. It writes a new one every time, and the chance in it was added on purpose.
What’s really happening.
- Step 01 / 04
Two new chats, the same request. Both answers began “Welcome aboard! We’re”, then split: one said “glad”, the other “excited”.
- Step 02 / 04
AI writes one word at a time. For every next word it has a short list of choices, and the most likely one usually wins.
- Step 03 / 04
Always taking the top word sounds smarter, but researchers found it makes text flat and repetitive. So a little chance is added on purpose.
- Step 04 / 04
That chance is a number called temperature. Turn it up for variety, down for nearly the same answer every time. Regenerate asks for a different pick from the same list.
Welcome aboard! We’re glad
Welcome aboard! We’re excited
- glad
- excited
- thrilled
- happy
- Last 5 picks
- Unit
- The model writes in tokens: a word or part of a word, one at a time.
- Next word
- Every possible next token gets a probability, based on all the words before it.
- Sampling
- The model draws from the likely options instead of always taking the top one.
- Temperature
- A number in the request. Higher = more varied, lower = more focused and predictable.
- Lowest setting
- Nearly the same answer, but not guaranteed identical.
The technical layer of this case is behind Agent Mode.
Regenerate isn’t fixing a mistake. It’s a different pick from the same list.
Checked against the source.
- [1]Holtzman, Buys, Du, Forbes, Choi · ICLR 2020The Curious Case of Neural Text Degeneration Always choosing the most likely words (maximisation) produces bland, repetitive text, and sampling fixes it.
- [2]OpenAI · openai-pythoncompletion_create_params.py: the temperature parameter Temperature is between 0 and 2; higher values make the output more random, lower values more focused and deterministic.
- [3]OpenAI CookbookUsing logprobs Each token has a probability based on the tokens before it.
- [4]Anthropic · Claude docsText Completions API reference: the temperature parameter Use temperature closer to 0 for analytical tasks and closer to 1 for creative ones; even at 0 the results are not fully deterministic. (Newer Claude models fix it at 1.0.)
Transcript
- You’ve done this before. You ask AI a question, press Regenerate 2 times, and get 2 new answers. I never knew why, until I tested it. Let’s decode it.
- I asked for the same thing in two new chats: a welcome message for my app. Both answers started with “Welcome aboard! We’re”. And then they split. One said “glad”. The other said “excited”.
- Here’s what’s happening. AI writes an answer one word at a time, and for every next word, it has a short list of choices. The best one usually wins. But every now and then, a different word gets picked, and the rest follows it.
- Now, why not always pick the best word? That sounds smarter, right? Researchers tested it. The writing came out flat and kept repeating itself. So a little chance was added on purpose. It’s called temperature.
- And it’s just a number. Turn it up for more variety. Turn it down for nearly the same answer every time, like you want for code.
- So that Regenerate button isn’t fixing a mistake. It’s asking for a different pick from the same list. And now, you know why.
- Join the Decode Agent journey, and decode the next case with me.
No corrections since publication. If a fact changes, it’s logged here.
You already see what others miss_
Claim your Agent ID and follow the journey from the inside. Decode Agent is just getting started.
- Your Agent ID, yours to keep and share.
- Every case, filed here with its sources.
- On the list when updates begin.
