Maglev: Sliding Recurrent Memory uses a recurrent Transformer with fixed-size memory to generalize sliding-window attention; introduces a two-model architecture with a prefiller and memory model
Read the original at arxiv.org→arXiv:2608.02870v1 Announce Type: new Abstract: We introduce \ours{}, a recurrent Transformer architecture with fixed-size memory that generalizes sliding-window attention while remaining parallelizable during...
Original headline: "Maglev: Sliding Recurrent Memory"
Coverage timeline
- Aug 5, 04:00 UTC arXiv cs.LG lead source Maglev: Sliding Recurrent Memory