STAPLE: static sparse upcycling of pretrained checkpoints via embedding modules
Read the original at arxiv.org→arXiv:2610.12599v1 Announce Type: new Abstract: Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures remains...
Original headline: "STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules"
Coverage timeline
- Oct 12, 04:00 UTC arXiv cs.LG lead source STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules