Muon optimizer reaches grokking threshold on modular arithmetic faster than AdamW; ablation shows speedup from Newton-Schulz orthogonalization
Read the original at arxiv.org→arXiv:2607.20512v1 Announce Type: new Abstract: The Muon optimizer reaches the grokking threshold on modular arithmetic faster than AdamW. Prior work attributes this to "spectral-norm constraints plus orthogonalized...
Original headline: "The Active Ingredient in Muon's Grokking"
Coverage timeline
- Jul 24, 04:00 UTC arXiv cs.LG lead source The Active Ingredient in Muon's Grokking