Distributed LLM inference via communication-efficient speculative decoding
Read the original at arxiv.org→arXiv:2610.35957v1 Announce Type: new Abstract: Distributed speculative decoding has emerged as a promising approach for device--server collaborative large language model (LLM) inference, but its decoding speedup is...
Original headline: "Distributed LLM Inference via Communication-Efficient Speculative Decoding"
Coverage timeline
- Oct 28, 04:00 UTC arXiv cs.LG lead source Distributed LLM Inference via Communication-Efficient Speculative Decoding