Charles Frye

We've written before about how much speculative decoding matters for interactivity, and that's not changed. New this time, at least for what we've shared publicly: tiered KV caching and smarter routing.