Multi-layer indexer: overlapping selection and attention in DeepSeek sparse attention
Read the original at arxiv.org→arXiv:2611.03768v1 Announce Type: new Abstract: Sparse attention cuts the cost of long-context inference by making the attention matrix sparse: each query attends only a subset of the sequence, chosen by a fixed...
Original headline: "Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention"
Coverage timeline
- Nov 2, 22:24 UTC r/LocalLLaMA New sparse attention method for ultra-long context
- Nov 4, 05:00 UTC arXiv cs.LG lead source Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention