Why LLMs Aren't Deterministic (Even at Temperature 0)

Temperature 0 gives you more reproducibility, not complete reproducibility. This video breaks down why, and what practitioners actually do about it in production systems where reproducibility isn't optional.

What's Covered

• Floating-point non-determinism: why GPU parallel operations produce different results at the bit level
• Infrastructure non-determinism: cloud LLM APIs run on dynamic hardware configurations
• Model version drift: providers silently update models without changing endpoint names
• Context window effects: attention and KV cache implementations introduce variation

The Regulated-Industry Problem

For most applications, occasional non-determinism at temperature 0 is a minor annoyance. For regulated applications — financial analysis, medical decision support, legal document review — it's a structural problem. "We ran this analysis and got this result" needs to mean something reproducible when an auditor asks.

What You Can Do

The companion post Why LLMs Aren't Deterministic (Even at Temperature 0) — And How to Fix It covers the practical mitigations in more depth: seed control, output fingerprinting, deterministic components for critical calculations, and designing for auditability rather than determinism.