Muon update polarizes matrix parameters via Orthogonalized Muon update; study analyzes PolarGD on matrix quadratics and compares line-search directions to mitigate learning-rate effects
Read the original at arxiv.org→arXiv:2610.30280v1 Announce Type: new Abstract: Muon updates matrix-valued parameters along approximately orthogonalized directions and often trains large language models faster than AdamW, yet it remains unclear...
Original headline: "A Quadratic Lens on Muon's Polar Update: Benefits, Pitfalls, and the Role of Newton-Schulz"
Coverage timeline
- Oct 23, 04:00 UTC arXiv cs.LG lead source A Quadratic Lens on Muon's Polar Update: Benefits, Pitfalls, and the Role of Newton-Schulz