What Is a Temperature Setting in AI?
Ask an AI the same question twice and you'll almost always get two different answers.
Not wildly different, usually. But different in word choice, structure, emphasis, sometimes in substance. If you're used to software that produces deterministic outputs, the same input always producing the same output, this variability can seem like a bug. It isn't. It's a feature, and it's controlled by a parameter called temperature.
To understand temperature, it helps to understand how language models generate text. At each step, the model doesn't pick the single most likely next word. It produces a probability distribution over its entire vocabulary, assigning a likelihood to every possible next token. Temperature is a parameter that shapes that distribution before the model samples from it.
At a temperature of zero, or close to it, the model becomes deterministic. It always selects the highest-probability token at each step, producing the same output every time for the same input. The output is maximally predictable and maximally conservative. It reflects the model's best single guess at each step, with no variation.
As temperature increases, the probability distribution flattens. Lower-probability tokens become more likely to be selected. The model takes more risks, reaching for less obvious word choices and generating more varied, sometimes more creative output. At very high temperatures, the distribution becomes so flat that the model essentially samples randomly, producing outputs that are often incoherent.
The practical range most applications operate in is somewhere between 0 and 1, with the right setting depending on what the output is for. Factual question answering, code generation, and structured data extraction generally want low temperature. The goal is accuracy and consistency, not variation. Creative writing, brainstorming, and tasks where diversity of output is valuable generally want higher temperature. A temperature of around 0.7 is a common default for general-purpose use, representing a balance between coherence and variation that works reasonably well across many tasks without being optimized for any particular one.
Temperature interacts with another related parameter called top-p, sometimes called nucleus sampling. Where temperature reshapes the entire probability distribution, top-p restricts sampling to a subset of the most likely tokens, specifically the smallest set whose combined probability exceeds a threshold p. Setting top-p to 0.9 means the model only samples from tokens that together account for 90% of the probability mass, cutting off the long tail of very unlikely options. The two parameters are often used together, and the combination gives fairly fine-grained control over output behavior without requiring deep technical knowledge to apply in practice.
Understanding temperature also helps explain some common AI behaviors that otherwise seem mysterious. The reason an AI writing assistant produces different variations of a paragraph each time you ask is temperature. The reason a code generation tool set to low temperature produces the same function implementation reliably is temperature. The reason the same model seems more creative in some applications and more precise in others is often temperature, set differently by whoever built the application on top of the model.
For most people using AI tools through a consumer interface, temperature is set by whoever built the product and isn't directly accessible. But for anyone building on top of AI models through an API, temperature is one of the first parameters worth understanding and experimenting with. It's one of the more direct levers available for shaping model behavior, and knowing what it does makes the difference between adjusting it deliberately and adjusting it at random.