Adam’s full update rule is analyzed as a diagonal empirical Fisher approximation to measure its geometric deviation from natural gradient descent across losses.
Read the original at arxiv.org→arXiv:2610.00004v1 Announce Type: new Abstract: Adam is the standard optimizer in deep learning, yet its geometric relationship to natural gradient descent (NGD) contains unresolved questions. We study Adam's full...
Original headline: "How Far is Adam from Natural Gradient Descent?"
Coverage timeline
- Oct 2, 04:00 UTC arXiv cs.LG lead source How Far is Adam from Natural Gradient Descent?