Data predictability shapes Weibull weight-scale growth in transformer training
Read the original at arxiv.org→arXiv:2608.23573v1 Announce Type: new Abstract: A trained transformer's weight magnitudes can be summarized by a two-parameter Weibull distribution whose shape $k \approx 1.2$ is stable across layers and models, so...
Original headline: "Data Predictability Shapes Weibull Weight-Scale Growth in Transformer Training"
Coverage timeline
- Aug 26, 04:00 UTC arXiv cs.LG lead source Data Predictability Shapes Weibull Weight-Scale Growth in Transformer Training