Transformers learn two competing sorting algorithms, and only one length-generalizes
Read the original at arxiv.org→arXiv:2610.26974v1 Announce Type: new Abstract: We show that two-layer transformers trained to near-zero loss on a sorting task converge to one of two mechanistically distinct solution classes depending on their...
Original headline: "Transformers Learn Two Competing Sorting Algorithms, and Only One Length-Generalizes"
Coverage timeline
- Oct 21, 04:00 UTC arXiv cs.LG lead source Transformers Learn Two Competing Sorting Algorithms, and Only One Length-Generalizes