LLM Parameters Explained: Temperature, Top-P, and the Dials That Still Work

LLM parameters are the settings you send with an API call to control how the model picks each token and when it stops. Sampling parameters (temperature, top-p, top-k) decide how much randomness gets into the choice. Control parameters (max_tokens, stop sequences, seed) decide length, cutoff, and repeatability. In 2026 there is a twist: the frontier models have started rejecting the sampling group entirely.

TL;DR

• Temperature rescales, top-p deletes. Temperature reshapes the probability distribution without removing options. Top-p cuts the tail off before the draw. They are not two versions of the same dial.
• Move one, not both. Both OpenAI and Anthropic document the same rule. Two overlapping controls make the result impossible to attribute.
• The dials are being removed. Anthropic's Messages API reference now marks temperature, top_p, and top_k deprecated: models released after Claude Opus 4.6 reject any non-default value with a 400. OpenAI's GPT-5 and o-series do the same.
• What replaced them: reasoning_effort, verbosity, structured output schemas, and prompt-level control. The steering moved up the stack.
• Temperature 0 is not determinism. Anthropic states plainly that results will not be fully deterministic even at 0.0.
• The dials were never the big lever anyway. A bad prompt at temperature 0.2 is still a bad prompt.

What the sampling parameters actually do

Every token the model produces starts as a vector of raw scores called logits, one per token in the vocabulary. On its own that vector is not a probability distribution and it is not an answer. Three steps turn it into a single chosen token, and each sampling parameter owns exactly one of them.

Temperature divides every logit by the value you set, then runs softmax. Divide by 0.5 and the gaps between scores double, so the leading token pulls further ahead and the output gets more predictable. Divide by 1.5 and the gaps compress, so unlikely tokens get a real chance. Set it to 0 and the sampling step collapses to greedy selection: always take the top-scoring token. Nothing is ever removed from the running, the odds are just redistributed.

Top-k takes the K highest-probability tokens and throws the rest away. At top_k=40, token number 41 has a zero percent chance no matter how close it was.

Top-p, or nucleus sampling, sorts tokens by probability, walks down the list adding them up, and stops once the running total reaches p. At top_p=0.9 the bottom 10 percent of probability mass is deleted. The size of the surviving set moves with the model's confidence, which is the useful part: on a token the model is sure about, top-p might keep two candidates, and on a genuinely open choice it might keep two hundred.

That difference is the whole reason both providers tell you not to move temperature and top-p at the same time. One reshapes the distribution, the other truncates it, and stacking them means you can no longer say which one caused the behavior you are looking at.

The 2026 change: the dials are being taken away

Here is the part most parameter guides have not caught up to. On the newest frontier models, you cannot set these at all.

Anthropic's Messages API reference now carries a deprecation notice on all three sampling parameters. Models released after Claude Opus 4.6 do not support setting temperature: a value of 1.0 is accepted for backwards compatibility, and every other value is rejected with a 400 error. top_p accepts values at or above 0.99 and rejects the rest. top_k is rejected outright.

OpenAI's reasoning line landed in the same place. GPT-5 and the o-series return a 400 on temperature, with an error stating that only the default value of 1 is supported, and the same applies to top_p, frequency_penalty, presence_penalty, logit_bias, and logprobs.

The reasoning is straightforward once you see it. A reasoning model does not generate one pass of tokens and hand it to you. It thinks, checks, and revises inside a process the lab tuned as a unit. Letting a caller reach in and flatten the distribution mid-process breaks the calibration that makes the output good. So the labs closed the panel.

The practical consequence is a migration bug that has hit a lot of codebases: a client library that always sends temperature: 0 starts throwing 400s the moment someone swaps in a newer model. If you maintain a model router, strip the sampling parameters per model rather than sending a fixed set to everything.

The parameters that still work everywhere

The sampling group got restricted. The control group did not, and it is the more useful half in production anyway.

max_tokens caps generation length. It is a hard stop, not a target: the model does not plan a shorter answer to fit, it gets cut off mid-sentence. Set it as a safety ceiling well above your expected output, and control real length in the prompt. It is also your defense against a runaway loop billing you for 8,000 tokens of repetition. Anthropic's API has one useful edge case here: max_tokens: 0 populates the prompt cache without generating a response.

Stop sequences end generation when a specific string appears. They are the cleanest way to keep a model from writing past the artifact you asked for, and the matched sequence is reported back so you know why it stopped rather than guessing.

Seed is OpenAI-side and best-effort. Pass the same seed and the same parameters and you get a mostly repeatable sample, with a system_fingerprint that tells you when the backend changed underneath you. Anthropic's Messages API does not expose one. Treat a seed as a debugging aid, not a contract.

reasoning_effort and verbosity are the replacements on the reasoning models. Effort trades latency and token spend for depth on hard problems. Verbosity controls how much the model says at a given effort. These are coarser than a temperature dial and that is deliberate.

Structured output schemas do more for output stability than any sampling parameter ever did. Constraining the response to a JSON schema removes the entire class of variation you were trying to suppress with temperature 0, because the shape is enforced rather than requested.

Settings by job, on models that still accept them

Plenty of production traffic still runs on older models where these knobs work. A defensible starting grid:

• Classification, extraction, routing: temperature 0. Same input, same label. Anything else makes your eval set unreadable, because you cannot tell a real regression from a resample.
• Code generation: temperature 0.2 to 0.3. Low enough to stay on known-good patterns, not so low that it repeats a bad first line forever.
• Drafting, summaries, internal docs: temperature 0.5 to 0.7. This is where the default of 1.0 is usually too loose and 0 is flat.
• Marketing copy and idea generation: temperature 0.8 to 1.0, and generate several candidates rather than one. Variance is the point.
• Candidate generation for a judge or a filter: above 1.0 is legitimate, because a downstream scorer is doing the selecting. On its own, it is just noise.

Two rules that matter more than the numbers. First, change one parameter at a time and re-run the same eval set, or you are guessing. Second, log the parameters with every call. When quality moves next quarter you want to know whether the model changed, the prompt changed, or someone nudged a dial in a config file.

What to change instead of the dials

The uncomfortable truth about the whole sampling panel is that it was always a small lever. On a scored eval set, moving temperature from 0.7 to 0.2 typically buys you consistency, not correctness. If the model is getting the answer wrong, a narrower distribution just makes it wrong more reliably.

The levers that actually move quality, roughly in order of return:

1. Give it the right context. Most failures are missing information, not excess randomness.
2. Constrain the output shape. A schema beats an instruction, every time.
3. Show two or three examples. Few-shot examples pin down format and edge-case handling faster than any parameter.
4. Split the job. One call that extracts, one that decides. Each is easier to evaluate and cheaper to fix.
5. Run it as a loop, not a single shot. Generate, check against a rule you can write down, revise. That structure is what turns a good model into a reliable system, and it is the pattern I wrote up in Write Loops, Not Prompts.

Once those five are in place, the sampling parameters become what they should have been all along: a small final adjustment, not the thing you reach for first. I send the working versions of these patterns, with the failure modes that came with them, in the newsletter.

The bottom line

Learn what the parameters do, because you will meet them in older models, open weights, and every self-hosted stack. Temperature rescales the distribution, top-p truncates it, top-k truncates it more bluntly, and you move one of them at a time or you learn nothing.

Then check whether your model still accepts them. On the frontier models of 2026 the answer is increasingly no, and a hardcoded temperature: 0 is now a 400 waiting to happen. The controls that survived, max_tokens, stop sequences, output schemas, and reasoning effort, are the ones worth wiring into your code properly.

The dial board made a good picture. It was never where the quality came from.