Adam converges under heavy-tailed noise in the plain vector-form setting, with guarantees for stochastic gradients having bounded p-th moment (p in (1,2]).
Read the original at arxiv.org→arXiv:2607.27383v1 Announce Type: new Abstract: We establish the first convergence guarantees for the plain vector-form \emph{Adam} optimizer under heavy-tailed stochastic noise. While several Adam variants are...
Original headline: "The Convergence Behavior of Adam under Heavy-Tailed Noise"
Coverage timeline
- Jul 31, 04:00 UTC arXiv cs.LG lead source The Convergence Behavior of Adam under Heavy-Tailed Noise