Query-derived erase direction (QED) added to mitigate interference in linear attention reads
Read the original at arxiv.org→arXiv:2608.13668v1 Announce Type: new Abstract: Linear attention keeps a state of fixed size. At long context, many stored items share this state, and interference between them degrades retrieval. Gated DeltaNet-2...
Original headline: "The Query Knows What to Forget: A Second Erase Direction for Linear Attention"
Coverage timeline
- Aug 17, 04:00 UTC arXiv cs.LG lead source The Query Knows What to Forget: A Second Erase Direction for Linear Attention