PACE: Keeping the right tokens in the KV cache is not enough for long-horizon LLM agents
Read the original at arxiv.org→arXiv:2611.10240v1 Announce Type: new Abstract: Long-horizon LLM agents accumulate sessions of up to millions of tokens, far more key-value (KV) cache than one GPU holds, so serving systems keep a small budget of...
Original headline: "PACE: Keeping the Right Tokens in the KV Cache Is Not Enough for Long-Horizon LLM Agents"
Coverage timeline
- Nov 10, 05:00 UTC arXiv cs.LG lead source PACE: Keeping the Right Tokens in the KV Cache Is Not Enough for Long-Horizon LLM Agents