LSTM-style recurrent transformer shares a key–value cache across its core, reducing parameters by 40.4% at width 1024 and lowering persistent cache sets from fifteen to four
Read the original at arxiv.org→arXiv:2610.33360v1 Announce Type: new Abstract: Recurrent transformers reuse weights to perform deep computation with fewer parameters. We study an LSTM-style transformer that also shares one key--value cache across...
Original headline: "LSTM-Style Looped Transformers with Shared KV Caches"
Coverage timeline
- Oct 27, 04:00 UTC arXiv cs.LG lead source LSTM-Style Looped Transformers with Shared KV Caches