Low-RAM: Low-Rank keys and amortised values for composable KV cache compression at test-time
Read the original at arxiv.org→arXiv:2611.07025v1 Announce Type: new Abstract: Autoregressive decoding relies on the KV cache, whose memory footprint grows linearly with context length and caps batch size and throughput. Existing post-training...
Original headline: "LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time"
Coverage timeline
- Nov 6, 05:00 UTC arXiv cs.LG lead source LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time