Parameter-free adaptive sparse attention via compression-based content selection
Read the original at arxiv.org→arXiv:2607.21752v1 Announce Type: new Abstract: Data-adaptive sparse attention masks substantially outperform fixed patterns (e.g., BigBird and Longformer) and can even exceed dense attention on long sequences....
Original headline: "Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection"
Coverage timeline
- Jul 27, 04:00 UTC arXiv cs.LG lead source Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection