Aggressive decoding-time KV eviction: EMA temporal aggregation preserves ranking under aggressive KV compression
Read the original at arxiv.org→arXiv:2609.03515v1 Announce Type: new Abstract: Decoding-time KV cache compression research focuses heavily on designing better token scoring functions, while the temporal rule that aggregates scores across decode...
Original headline: "What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation"
Coverage timeline
- Sep 4, 04:00 UTC arXiv cs.AI lead source What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation