Charles Frye

Turned out the problem was routing -- consistent-hashing based on session IDs wasn't enough to solve over-dispersion of load. So we added stateful, load+KV-aware routing, and dispersion dropped (Fano Factor ~4 -> ~0.5), and TTFT tails along with it.