Ask an AI chatbot the same question twice and you'll often get two different answers — sometimes slightly, sometimes completely. It's one of the first things people find unnerving about the technology: shouldn't a computer be consistent? The answer reveals something fundamental about how these systems actually work, and it's controlled by a single dial with a slightly misleading name: temperature.

The model doesn't pick a word — it picks the odds

When a language model generates text, it doesn't decide on the next word directly. At every step it produces a probability distribution: a ranked list of all the words it could say next, each with a percentage. After "The capital of France is", the word "Paris" might get 92%, "a" 3%, "located" 2%, and a long tail of others sharing the rest.

What happens next is the crucial part. The system samples from that list — it rolls a weighted dice. Most of the time it lands on "Paris", but not always, and across a whole paragraph those small rolls compound into genuinely different responses. The variation isn't a bug or the model "changing its mind." It's the model doing exactly what it's designed to do: drawing from a distribution rather than reading out a fixed script.

What temperature actually does

Temperature is the dial that controls how adventurous that dice-roll is. Technically, it reshapes the probability distribution before sampling.

• Low temperature (near 0) sharpens the distribution — the most likely word gets even more likely, the rest get squeezed toward zero. At a temperature of 0 the model becomes (almost) deterministic: it just takes the single highest-probability word every time. Same question, same answer.
• High temperature (above 1) flattens the distribution — the gap between the likely and unlikely words narrows, so the model takes more chances on surprising choices. The output gets more varied, more "creative", and also more prone to going off the rails.

A useful mental image: low temperature is a careful person who always says the obvious, safe thing; high temperature is someone brainstorming out loud, throwing out ideas without filtering. Neither is smarter — they're tuned for different jobs.

Two other dials you'll sometimes meet

Temperature usually works alongside two cousins that limit which words are even eligible. Top-k sampling says "only consider the k most likely words, ignore everything else." Top-p (or "nucleus") sampling is subtler: "consider only the smallest set of top words whose probabilities add up to p percent." Both are ways of cutting off the weird long tail so the model can be varied without occasionally blurting out something nonsensical. In practice they're often used together with temperature to shape the trade-off between reliable and interesting.

Why it matters in practice

This isn't just trivia — it's the difference between using the tool well and being baffled by it.

• For anything where you want accuracy and consistency — factual answers, code, data extraction, following instructions — you want low temperature. You're not after creativity; you're after the single most-likely-correct continuation, the same way every time.
• For creative or exploratory work — brainstorming names, drafting fiction, generating varied options — higher temperature is the point. The randomness is the feature.

And one honest caveat worth holding onto: a higher temperature does not make a model more intelligent or more insightful. It makes it less predictable. The "creativity" you get from cranking it up is just the model wandering further from its safest guesses — which sometimes lands on something fresh and useful, and sometimes lands on a confident mistake. The variation you see isn't a window into deeper thought. It's a dice-roll you're allowed to weight.

The takeaway

The reason an AI gives you different answers each time is not mystery or malfunction — it's deliberate, controllable randomness, baked into how it turns probabilities into words. Once you know the dial exists, the behaviour stops being unnerving and becomes useful: turn it down when you want the AI to be a reliable clerk, turn it up when you want it to be a restless brainstormer. Same machine, different mood — and now you know which knob sets it.