Charles Frye

After we had optimized single-replica performance, we saw that it didn't transfer directly to aggregate performance in prod. We saw occasional big spikes in TTFT and generally lower per-replica throughput.