Retrofitting recurrent depth into a pretrained language model improves iterative latent transitions at two parameter budgets; Qwen2.5-0.5B-Instruct uses a recurrent block with an identity-preserving loop.
Read the original at arxiv.org→arXiv:2608.11233v1 Announce Type: new Abstract: A dense, pretrained language model can be retrofitted with recurrent depth and learn an iterative latent transition that persists after outcome-only annealing....
Original headline: "Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets"
Coverage timeline
- Aug 13, 04:00 UTC arXiv cs.CL lead source Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets