confidence has a temperature

This is a physicist’s companion to what Jev is selling, for anyone who wants the why. That post turns an honest probability into a threshold on a refund queue and shows what a cooled model costs there. This one is why a model’s confidence usually isn’t a probability in the first place, why the knob that bends it is called temperature, and why preference tuning can turn it down.

The short version: two models can rank every item in exactly the same order and still need different thresholds to make the same decisions, because a model’s confidence is a number of a particular kind, and most of the time it isn’t the kind it looks like.

---

what kind of number

In 1946 the psychologist S. S. Stevens sorted measurements by what you can do to the numbers without changing what they mean. An ordinal scale (finishing places in a race, star ratings) survives anything that keeps the order, so you can sort the numbers but not meaningfully average them. Interval and ratio scales (Celsius, weight) survive less and so support more arithmetic. A probability sits past the end of his list. It survives no transformation at all, because 0.8 has to mean four times in five, and anything that moves it to 0.9 has changed what it claims. That’s what lets you compare it to a threshold worked out from prices, as the refund queue does, and it’s what makes it fragile, because most of what models report as confidence can only be trusted as an ordinal score, a number that ranks things correctly and means nothing else, written with a percent sign.

None of this is new. Weather forecasters have been calibrated for decades, since a forecast gets checked every day, and their tools came from their own journals: Glenn Brier’s score in 1950, Allan Murphy’s decomposition of it in 1973. Machine learning came to it late and sideways. Niculescu-Mizil and Caruana’s 2005 benchmark found boosted trees and SVMs squashing their outputs toward the middle, naive Bayes flinging them toward the corners, and logistic regression calibrated almost by accident, because log loss is a proper scoring rule. Then Guo and colleagues (ICML 2017) found that bigger, more accurate deep networks had quietly become less honest, and proposed a single number called temperature as the fix.

---

temperature

Every classifier that hands you a probability over options, from a 2005 neural net to whatever you called this morning, ends in the same operation. It gives each option a score, called a logit, which can be any real number, and then turns the scores into probabilities by exponentiating each one and dividing by the total, so that bigger scores become bigger probabilities, everything comes out positive, and the whole thing sums to one. Machine learning calls this softmax. Physics has called it the Boltzmann distribution since the 1870s, when Ludwig Boltzmann showed that a molecule in a gas at temperature $T$ sits in a state of energy $E$ with probability proportional to $e^{-E/k_B T}$, so that low-energy states are likelier and temperature sets how much likelier. Read the logit as a negative energy and softmax is Boltzmann at $T = 1$, and the temperature slider in every LLM API is the same $T$, dividing the logits before the exponential:

$$p_i = \frac{e^{z_i/T}}{\sum_j e^{z_j/T}}$$

What physics gives you for free is an intuition for what temperature does. Heat a system and the differences between energies stop mattering, so it wanders through every state almost evenly; cool it and it settles into its lowest state and stays there. Here are the same three logits, 2, 1 and 0, at three temperatures:

Nothing about the options changed, and neither did their order, since the favorite is the favorite at every temperature. What changed is how loudly the distribution announces its favorite, and if that distribution is also what the model reports as its confidence, temperature is a dial on how sure the model sounds that has no connection to how much it knows. For a yes/no question, here is what cooling and heating do to a model’s stated probability as a function of what it actually believes:

Read up from 0.8 on the horizontal axis and three models that believe the same thing report 0.94, 0.80 and 0.64. The curves bend, but none of them ever turns back on itself, so any item ranked above another stays above it, and cooling changes every number while changing no ordering. That’s the thread back to Stevens: cooling is a transformation an ordinal scale shrugs off and a probability can’t survive.

Now put it back in the refund queue, where approving an abusive request costs five dollars, turning away a customer costs twenty, and the threshold works out to 0.2. Cool a probability of 0.3 to $T = 0.5$ and it becomes about 0.155, which lands on the other side of 0.2. Run a cooled copy of the fake model described further down through that queue at 0.2 and it costs $1.59 a ticket against the honest model’s $1.44, and a heated one ($T = 2.5$) costs $1.88. Neither knows any less than the honest model. The cooled one’s own cheapest threshold is down near 0.04, where it costs $1.43, but nothing in the arithmetic would send you there, and only labelled examples could.

Anything that depends only on the ordering survives cooling: which answer ranks first, a human preferring one answer to another, and for a yes/no question the AUC and accuracy at 0.5, since cooling never moves a probability across one half. So the metrics most people check first can’t see the thing that just cost about eleven percent.

---

why models run cool

This matters because of what happens to most models after pretraining. The step is called preference tuning, usually some variant of RLHF, and the vocabulary it borrows from reinforcement learning makes it sound more exotic than it is. The model’s distribution over possible answers gets called a policy. Human raters look at pairs of answers and pick the better one, and a second model, the reward model, learns to predict their picks, so that it can give any answer a score. Then the original model is nudged toward answers that score well, with one restraint: it pays a penalty for drifting too far from the model it started as, measured by KL divergence (Kullback–Leibler), which is a standard yardstick for how different two probability distributions are, and a single number, $\beta$, sets how heavy that penalty is.

That optimization has an exact solution,

$$\text{new}(y) \;\propto\; \text{old}(y)\, e^{\,\text{reward}(y)/\beta},$$

and to a physicist it’s another Boltzmann factor, with reward playing the part of negative energy and $\beta$ playing temperature, multiplied onto the distribution the model already had. Now suppose raters tend to reward answers that sound confident, and push that to its limit, where the reward simply is the model’s own log-probability. Substitute and the solution becomes

$$\text{new}(y) \;\propto\; \text{old}(y)^{1 + 1/\beta},$$

and raising a Boltzmann distribution to a power is the same as dividing its temperature, so the tuned model is the old model cooled to $T = \beta/(1+\beta)$, which is below one for any positive $\beta$. It’s a caricature, and the takeaway is narrow: this toy reward shows one way preference optimization can sharpen probabilities without making the predictions any better. Real preference tuning also changes which answers a model prefers, so it isn’t just a temperature. Whether tuned models end up overconfident in practice is an empirical question, and the measurements say they often do: Tian and colleagues found the token probabilities of preference-tuned models poorly calibrated where the pretrained ones had been well calibrated.

The same closed form is the starting point of Rafailov and colleagues’ DPO paper (NeurIPS 2023), where it appears without the thermodynamics.

---

measuring it

To watch this happen at scale I needed a model whose honesty I could set exactly, which no real model offers, so I built a fake one that generates things in the reverse of the natural order. A real model meets an item whose answer is already fixed and forms a belief about it. The mock draws the belief first, then flips a coin weighted by that belief to decide what’s true:

const belief = beta(rand, 0.45, 0.45);
const actual = rand() < belief;

That makes it calibrated by construction, since among the items where it believed 0.8 the event happens 80% of the time, because that’s how the event was drawn. It has exactly two knobs. The beta(0.45, 0.45) decides where its beliefs come from, which amounts to deciding how smart it is: both parameters below one bend the distribution into a U, a model that is usually confident one way or the other with a thin band of genuine toss-ups, which is roughly what a competent classifier looks like from the inside. The temperature decides how honestly it says what it believes.

Those two knobs map one-to-one onto the two measures that carry through the rest of this series, both from Allan Murphy’s split of the Brier score. ReliabilityHow far a model's stated probabilities sit from how often things actually happen. Zero is perfectly honest. is how honest the forecasts are, and the temperature sets it. ResolutionHow much the outcomes differ between the items a model scores differently. A model that gives everything the same score has none. is how informative they are, and the beta sets it.

For a calibrated forecaster, resolution works out to the variance of its beliefs, and a symmetric beta with both parameters equal to $a$ has variance $1/\big(4(2a+1)\big)$, which for $a = 0.45$ is 0.1316. The simulation below measured 0.1317.

Turn only the temperature, leaving every hidden truth where it was, and run twenty thousand items on the same seed:

ECE, expected calibration error, is the plainest of these: sort the predictions into ten buckets by what they claimed, check how often the thing actually happened in each bucket, and average the gaps, weighting each bucket by how many predictions landed in it.

temperature	ECE	reliability	resolution
1.0, honest	0.0065	0.00007	0.1317
0.5, cooled	0.0739	0.00638	0.1290
2.5, heated	0.1280	0.01924	0.1308

Reliability gets about ninety times worse cooled and nearly three hundred times worse heated, and resolution stays within two percent of the value the beta promised, because the temperature never touched what the model believed, only what it claimed.

---

where this goes

A word on the metaphor running through these posts. When I say a model is honest, I mean calibrated: its stated probabilities match how often things happen, over a population of items. That’s an empirical relationship between numbers and outcomes, not a claim about what the model believes or whether it means what it says.

A model whose confidence runs cool or hot still ranks your items as well as it ever did, so nothing that measures ranking will flag it, and every threshold you set on its numbers is quietly in the wrong place. What Jev is selling runs three bent models through a refund queue and repairs them with a few hundred labels or with none, and a probability you can count on is the loop that keeps a model honest in production.