Rank measures of fine-tuning updates are only marginally above noise when compared with random matrices and baseline references, the study finds.
Read the original at arxiv.org→arXiv:2610.22438v1 Announce Type: new Abstract: Studies of what fine-tuning does to a language model often summarise the weight update $\Delta W$ by its rank. We compare the rank measures used for this against the...
Original headline: "Rank Measures of Fine-Tuning Updates Need a Noise Baseline"
Coverage timeline
- Oct 19, 04:00 UTC arXiv cs.LG lead source Rank Measures of Fine-Tuning Updates Need a Noise Baseline