A conditional no-hair theorem for transformers shows that a transformer’s memory of its prompt decays as a power law with token position.
Read the original at arxiv.org→arXiv:2610.24133v1 Announce Type: new Abstract: We ask how much an autoregressive transformer retains of its prompt after generating $t$ tokens. Consider two different prompts followed by the same continuation text,...
Original headline: "A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt"
Coverage timeline
- Oct 20, 04:00 UTC arXiv cs.AI lead source A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt