LSTM-UT and Recurrent-Depth Transformers on cellular automata: LSTM-UT with bounded memory extrapolates depth more reliably than CoTFormer on Rule 30, for BUT carries only current hidden state
Read the original at arxiv.org→arXiv:2609.19521v1 Announce Type: new Abstract: Recurrent-depth Transformers apply shared computation repeatedly, but differ in how they retain information across steps. We compare a Block Universal Transformer...
Original headline: "LSTM-UT and Recurrent-Depth Transformers on Cellular Automata"
Coverage timeline
- Sep 18, 04:00 UTC arXiv cs.LG lead source LSTM-UT and Recurrent-Depth Transformers on Cellular Automata