AbridgeKV: compressing each completed block of generated text into the KV entries of learned pause tokens
Read the original at arxiv.org→arXiv:2610.20641v1 Announce Type: new Abstract: Long chain-of-thought decoding is memory-bound: the KV cache of the generated tokens grows linearly with the test-time compute being spent. Evicting cache entries does...
Original headline: "AbridgeKV: Compressing Long Reasoning into the KV Cache of Learned Pause Tokens"
Coverage timeline
- Oct 16, 04:00 UTC arXiv cs.CL lead source AbridgeKV: Compressing Long Reasoning into the KV Cache of Learned Pause Tokens