SMat-Attention: structured matrix attention for long-context sequence modeling
Read the original at arxiv.org→arXiv:2609.36062v1 Announce Type: new Abstract: Long-context sequence models face a fundamental tradeoff: softmax attention uses flexible token-level interactions at quadratic cost, whereas linear attention obtains...
Original headline: "SMat-Attention: Structured Long-Context Sequence Modeling"
Coverage timeline
- Sep 30, 04:00 UTC arXiv cs.AI lead source SMat-Attention: Structured Long-Context Sequence Modeling