Retrofitting unbounded context onto stock language models; the paper shows two stock models can retain and query across all prior context by manipulating geometry through Rotary attention
Read the original at arxiv.org→arXiv:2611.10141v1 Announce Type: new Abstract: A language model that never had to forget would keep every document it has read and every turn it has spoken, and answer from any of it. This paper shows that two...
Original headline: "Retrofitting Unbounded Context onto Stock Language Models"
Coverage timeline
- Nov 10, 05:00 UTC arXiv cs.CL lead source Retrofitting Unbounded Context onto Stock Language Models