Flat loss, evolving features: grokking in modular addition

arXiv:2610.22593v1 Announce Type: new Abstract: Can a network keep learning while its training cross-entropy stays nearly flat at a positive level? We show that it can. On modular addition, a standard testbed for...