Deletion Tuning: zeroing under 0.05% of LLM weights matches full fine-tuning across post-training regimes
Read the original at www.reddit.com→How little of a pretrained large language model (LLM) has to change for it to acquire new capabilities? We find that zeroing fewer than 0.05% of its weights, with no other modification, matches full fine-tuning (FFT)...
Original headline: "The Surprising Effectiveness of Deleting Weights in LLM Post-Training"
Coverage timeline
- Nov 10, 19:44 UTC r/LocalLLaMA lead source The Surprising Effectiveness of Deleting Weights in LLM Post-Training