Rohan Paul
@rohanpaul_ai
A model’s context window does not have to be an agent’s workspace limit.
KVMEM makes million-token agent memory practical by paging old KV state, cutting recall cost while beating compaction on task success.
Long-running agents eventually fill their context. Today, they usually compress old history into summaries or fetch old text again. Summaries can forget details. Fetching text makes the model process the same history again.
KVMEM keeps the work the model already did. Old context stays as reusable KV cache in GPU memory, RAM, or NVMe, and the system pulls back only the pieces needed for the current step.
On DeepSWE with Qwen3.8-27B, Pass@1 rose from 43.8% to 48.4% versus compaction-only. Across controlled benchmarks, recovery was 11.4–53.8× faster than Compact+RAG.
On a laptop with a 24 GB RTX 5090 GPU, KVMEM supported a 1M-token workspace at about 50 tokens/s.
This is not a 1M-token active prompt. Each step still sees only a bounded slice.
