LoRA rank does not control capacity under per-factor norm budgets, as updates remain within the rank-at-most-r ball inside a nuclear-norm constraint.
Read the original at arxiv.org→arXiv:2609.32002v1 Announce Type: new Abstract: The rank $r$ in LoRA is widely treated as a capacity control: a smaller rank is assumed to yield a simpler model that generalizes better. We show that, under hard...
Original headline: "What Does the Rank Buy? A Spectral and Distributional Analysis of Low-Rank Adaptation"
Coverage timeline
- Sep 29, 04:00 UTC arXiv cs.AI lead source What Does the Rank Buy? A Spectral and Distributional Analysis of Low-Rank Adaptation