Quadratic model can be surprisingly predictive for optimization in large-language models with 150M parameters
Read the original at arxiv.org→arXiv:2607.21716v1 Announce Type: new Abstract: Due to the complexity of neural network loss landscapes, optimization theory is forced to rely on idealized models, and there is generally a tradeoff between how...
Original headline: "A Defense of the Quadratic Model"
Coverage timeline
- Jul 27, 04:00 UTC arXiv cs.LG lead source A Defense of the Quadratic Model