ProLand: a new infeasible manifold optimization method for LoRA fine-tuning exploits temporary infeasibility to explore near the Stiefel manifold and progressively restore orthogonality.
Read the original at arxiv.org→arXiv:2611.00236v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) enables parameter-efficient fine-tuning but can suffer from redundant basis directions. Constraining the basis factor to the Stiefel...
Original headline: "ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning"
Coverage timeline
- Nov 3, 05:00 UTC arXiv cs.LG lead source ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning