The best optimizer depends on batch size.
Read the original at arxiv.org→arXiv:2610.08975v1 Announce Type: new Abstract: A plethora of new adaptive optimizers are designed to efficiently estimate and use minibatch gradient statistics to shape parameter updates, but they are typically...
Original headline: "The Best Optimizer Depends on Batch Size"
Coverage timeline
- Oct 8, 04:00 UTC arXiv cs.LG lead source The Best Optimizer Depends on Batch Size