To keep or not to keep: learning KV cache retention in disaggregated LLM serving systems
Read the original at arxiv.org→arXiv:2610.28637v1 Announce Type: new Abstract: Disaggregated \textsc{LLM} serving separates prefill and decode across node pools, placing a network fabric between \textsc{KV}-cache computation and consumption. This...
Original headline: "To Keep or Not to Keep: Learning KV Cache Retention in Disaggregated LLM Serving Systems"
Coverage timeline
- Oct 22, 04:00 UTC arXiv cs.LG lead source To Keep or Not to Keep: Learning KV Cache Retention in Disaggregated LLM Serving Systems