Sparse attention is matrix approximation, not choosing from a bag of values
Read the original at arxiv.org→arXiv:2610.10871v1 Announce Type: new Abstract: Large Language Models (LLMs) achieve strong performance across many domains, but their efficiency is limited by the quadratic cost of attention with respect to prompt...
Original headline: "Sparse Attention Is Matrix Approximation, Not Choosing from a Bag of Values"
Coverage timeline
- Oct 9, 04:00 UTC arXiv cs.CL lead source Sparse Attention Is Matrix Approximation, Not Choosing from a Bag of Values