Probability for AI

Models do not output answers, they output probability distributions. Understanding distributions, softmax, and confidence explains how AI systems actually behave.

Almost every modern AI model gives you a probability, not an answer. A classifier does not say “cat” — it says “88% cat, 9% dog, 3% fox.” An LLM does not pick the next word — it produces a probability for every possible next token. Once you see outputs as distributions rather than decisions, a lot of otherwise puzzling model behavior becomes obvious.

A distribution is a spread of belief

A probability distribution assigns a number to each possible outcome, where the numbers are all non-negative and sum to 1. That constraint — summing to 1 — is what makes it a distribution rather than an arbitrary list of scores: it forces the outcomes to compete, since giving more to one means giving less to the others.

“88% cat, 9% dog, 3% fox” is a distribution over three outcomes. So is an LLM’s output over its entire vocabulary — tens of thousands of tokens, each with a probability, all summing to 1. The shape of the distribution carries meaning. A peaked distribution (one outcome near 1) means the model is committing. A flat one (probability spread across many outcomes) means the model sees several plausibilities. That shape is the raw material every sampling decision works from.

From raw scores to probabilities: softmax

A network’s final layer does not naturally emit numbers that sum to 1. The forward pass produces a raw score for each outcome — these are called logits — and they can be any value, positive or negative. Logits of [3.2, 1.1, -0.5] are not a distribution; they do not sum to 1 and one is negative.

Softmax is the function that converts logits into a valid distribution. It does two things: exponentiate each score so all become positive and larger scores are amplified, then divide each by the total so they sum to 1. The result is a proper distribution where the largest logit gets the largest probability, but the smaller ones still get a share.

The details of this conversion — and why exponentiating rather than just normalizing matters — are covered in logits and softmax. What matters here is the role: softmax is the standard bridge from a network’s raw output to a probability distribution you can act on. Every multi-class classifier and every LLM ends this way.

Temperature: sharpening or flattening

Because softmax exponentiates, you can tune how peaked the resulting distribution is with a single knob before applying it. That knob is temperature.

• Low temperature sharpens the distribution toward its top outcome — the model becomes more confident and deterministic.
• High temperature flattens it, spreading probability toward less likely outcomes — more variety, more risk.

This is the exact mechanism behind the temperature setting you meet in sampling parameters. It does not change what the model believes is most likely; it changes how sharply that belief is expressed before an outcome is drawn. Temperature 0 collapses to always taking the single most probable token; higher values let genuine randomness in.

Sampling: turning a distribution into one output

A distribution is a spread of possibilities, but eventually you need one concrete output. You sample from the distribution — draw an outcome with probability equal to its share. A token with 0.7 probability gets picked roughly 70% of the time, one with 0.01 rarely.

This is why the same prompt can give different answers on different runs, and it is not a bug. The model produced a distribution; sampling drew different tokens from it. If you always took the single most probable token you would get repetitive, often worse text, which is why some randomness is usually kept. The whole family of choices here — how much of the distribution to sample from, how to cut off the unlikely tail — lives in sampling parameters.

Confidence is not correctness

The most important and most misunderstood point. A model’s probability is a statement about its internal patterns, not about truth. A distribution peaked at 0.97 means the model’s training produced a strong pattern pointing at that outcome. It does not mean the outcome is 97% likely to be correct in the world.

Models are often miscalibrated: a model that says 90% might be right only 70% of the time, or a confidently wrong answer might carry a very peaked distribution. This is a root cause of hallucination — fluent, high-probability output that is simply false. The distribution reflects statistical confidence, and statistical confidence and factual accuracy are different things that only sometimes agree. Treat a model’s certainty as a signal to weigh, never as a guarantee.

Why this underpins everything downstream

Probability is the native language of these systems. Training minimizes a loss defined over probabilities — cross-entropy rewards putting high probability on the right answer. Generation samples from probability distributions. Evaluation asks whether those probabilities are calibrated and accurate. When you understand that a model outputs a distribution and that softmax with temperature shapes it, the behavior of every generative system — its variety, its confidence, its failures — stops being mysterious.

What to remember

• Models output probability distributions — numbers over all outcomes that sum to 1 — not single answers.
• Softmax converts a network’s raw scores (logits) into a valid distribution; it is how classifiers and LLMs end.
• Temperature sharpens or flattens the distribution, controlling how confident and varied the output is.
• Sampling draws one outcome from the distribution, which is why the same prompt can yield different results.
• Confidence is not correctness: a peaked distribution reflects a strong internal pattern, not verified truth — the root of many hallucinations.

Next: Logits and Softmax — the exact conversion from a model’s raw scores to next-token probabilities.