Same prompt, different answers
Sampling and temperature: why a reply can change when nothing else did.
The same request can get different replies. In lesson 2's loop, the model doesn't have to take the top token. It Sampling Picking each next token at random, in proportion to the model's odds.: it picks at random, weighted by the odds, and one different early pick sends the rest down another path. The dial sets Temperature A setting that sharpens or flattens the odds before sampling.: how concentrated those odds are. It shows lesson 2's six likeliest tokens, 80 percent of the recorded odds, stretched to fill the chart. At temperature 1, delayed fills about 55 percent of the chart, from recorded odds of 45 percent.
Low temperature puts the odds on one word; high temperature spreads them out
The most common complaint from our coffee subscribers this week was about the
Shares among these 6 tokens at temperature 1.0 Shares among these 6 tokens at temperature 1.0
- ·delayed 55%
- ·incons 12%
- ·quality 10%
- ·delay 8%
- ·delivery 8%
- ·inconsistent 7%
Every other token is left out, so these 6 add up to 100%. · marks a space. Computed from GPT-4.1 mini’s odds, recorded on 9 Nov 2026.
At 1, the six keep their recorded ratios.
We asked GPT-4.1 mini for an email subject line for our made-up coffee company, giving it the week's main counts without saying whether to use them. The request shortens our late or missing deliveries to late deliveries.
The same request, five samples at temperature 0 and five at 1. Look for which samples put the numbers in the line.
You
Write a subject line for an email to Pinewick Coffee's leadership team summarizing this week's customer feedback. The main points: 11 messages about late deliveries, 7 about billing, and one report of mold on sealed beans. Reply with the subject line only.
Temperature 0
Recorded Recorded from GPT-4.1 mini · 9 Nov 2026
Weekly Customer Feedback: Delivery Delays, Billing Issues & Mold Report
Temperature 1
Recorded Recorded from GPT-4.1 mini · 9 Nov 2026
Weekly Customer Feedback Summary: Delivery Delays, Billing Issues & Mold Report
At 0, all five samples were the same line, with no counts. At 1, there were three different lines, and two of the five used the counts. The request left that choice to the model, and at 0 it made the same choice every time.
At temperature 0, five separate requests with no chat history all gave the same subject line. What does that show?
Anthropic's current API version doesn't accept a temperature setting, and its newest models use only the default. So if a reply must include something, like the week's counts, ask for it.
What to remember
The model Sampling Picking each next token at random, in proportion to the model's odds. each token from its odds, so different answers to the same request are normal.
Temperature A setting that sharpens or flattens the odds before sampling. changes how varied the picks are, and many current models don't let you set it. If a reply must include something, ask for it in the Prompt The instructions and material you give a model., then check the reply has it.
Every lesson’s takeaways are on the course cheat sheet.