ShortOPD: recovering pruned LLMs with short-to-long on-policy distillation

arXiv:2607.13124v1 Announce Type: new Abstract: Structured pruning is a hardware-friendly way to compress LLMs, but it is mostly validated on multiple-choice recognition tasks, while the same compressed checkpoints...