HyScale: scaling long-context training for learned sparse attention
Read the original at arxiv.org→arXiv:2610.32057v1 Announce Type: new Abstract: We present HyScale, a long-context training system for learned sparse attention. Learned sparse attention is a promising architecture for million-token contexts,...
Original headline: "HyScale: Scaling Long-Context Training for Learned Sparse Attention"
Coverage timeline
- Oct 26, 04:00 UTC arXiv cs.AI lead source HyScale: Scaling Long-Context Training for Learned Sparse Attention
- Oct 26, 04:00 UTC arXiv cs.CL LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning