If you have used an AI writing tool with any of its settings exposed, you have probably seen a slider or a number labelled temperature, usually running from 0 to about 2. It is one of the most useful dials to understand, because it controls something fundamental: how much randomness the model uses when it writes. And it explains a puzzle every chatbot user notices sooner or later — why the same prompt, asked twice, can come back with two different answers.

How a model actually chooses its words

To see what temperature does, you first have to know how a language model generates text. It does not write a whole sentence and hand it over. It produces one small chunk at a time — a "token", roughly a word or a piece of one — and for each token it calculates a probability for every possible next option in its vocabulary. After "The capital of France is", the token "Paris" gets a very high probability, and thousands of other tokens share what is left. The model then samples from that distribution: it draws a token, weighted by those probabilities. Repeat that, token after token, and a sentence appears.

What temperature changes

Temperature reshapes that probability distribution just before the model samples from it. Turn it down toward 0 and the distribution sharpens: the single most likely token dominates, and the model becomes almost deterministic, nearly always choosing the safest, highest-probability continuation. Turn it up and the distribution flattens: less likely tokens get a real chance of being picked, so the output grows more varied, more surprising, and less predictable.

That is the whole trick. Low temperature is focused and repetitive; high temperature is diverse and adventurous. At very low settings the same prompt tends to return the same answer every time; at higher settings it lands somewhere new on each run.

When to turn it down, and when to turn it up

The practical rule follows directly. For anything with a right answer — a factual question, a piece of code, a data extraction — you generally want temperature low, so the model holds to its most confident path and gives consistent, repeatable output. For anything where variety is the point — brainstorming, opening lines, alternative phrasings — a higher temperature is what stops every attempt from reading the same.

Temperature is not the only knob. You will sometimes see top-p (also called nucleus sampling) and top-k sitting alongside it. Those work a little differently — they limit which tokens are even eligible to be chosen, rather than reshaping the whole distribution — but temperature is the one that most directly turns the dial between "careful" and "creative".

The honest caveat

Two things are worth keeping straight. First, temperature does not make a model more creative in the way a person is; it does not have better ideas at high heat. It just widens the lottery of which token gets drawn, trading coherence for variety, and pushed far enough the text stops making sense. Second, and more important: setting the temperature to 0 makes the output consistent, not correct. A model that is confidently wrong will be deterministically, repeatably wrong at temperature 0. Low temperature buys you reliability of behaviour, never a guarantee of truth — a distinction worth holding onto every time someone suggests that turning the temperature down to zero is what makes an AI trustworthy.