Between gradient and natural gradient: a continuum of LoRA initializations
Read the original at arxiv.org→arXiv:2607.26247v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) fine-tunes large pretrained models at a fraction of the cost of full fine-tuning, but its performance depends strongly on how the adapters...
Original headline: "Between Gradient and Natural Gradient: A Continuum of LoRA Initializations"
Coverage timeline
- Jul 30, 04:00 UTC arXiv cs.LG lead source Between Gradient and Natural Gradient: A Continuum of LoRA Initializations