Muon with finite Newton-Schulz: the smoothing benefit in nonsmooth nonconvex optimization
Read the original at arxiv.org→arXiv:2608.26288v1 Announce Type: new Abstract: Muon has emerged as a strong optimizer for the matrix-valued parameters in large language model pretraining, approximately orthogonalizing its momentum with a few...
Original headline: "Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization"
Coverage timeline
- Aug 28, 04:00 UTC arXiv cs.LG lead source Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization