Control determinism where you can

You cannot make an LLM fully deterministic, but lowering temperature, pinning versions, and caching make behavior far more repeatable.

Turn variance down for tasks that need it

For tasks with a right answer, high temperature buys you nothing but inconsistency. Lower it. Where the platform exposes a seed, use it. You will not get bit-for-bit determinism, but you will narrow the spread of behavior considerably.

Pin the things that silently change

Model versions, prompts, and tool schemas all drift, and each drift can move behavior. Pin model versions explicitly and version prompts, so a change in reliability is something you chose rather than something that happened to you.

Cache stable results

When the same input should always yield the same output, cache it. Caching removes both cost and a source of variance, and it makes the deterministic parts of your system genuinely deterministic.

pythonfrom functools import lru_cache

@lru_cache(maxsize=10000)  # identical input skips the model entirely
def classify(text):
    resp = client.chat.completions.create(
        model="gpt-4o-2024-08-06",  # pin the version, not a floating alias
        temperature=0,               # no creativity for a task with one right answer
        seed=42,                     # same seed narrows the remaining spread
        messages=[{"role": "user", "content": text}],
    )
    return resp.choices[0].message.content