Learning what matters: supervising sparse attention routing with causal evidence sets
Read the original at arxiv.org→arXiv:2607.21692v1 Announce Type: new Abstract: Sparse attention reduces the cost of long contexts by allowing each query to read only selected parts of the input. These selectors are often trained by distilling the...
Original headline: "Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets"
Coverage timeline
- Jul 27, 04:00 UTC arXiv cs.LG lead source Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets