LatentLoop scales looped MoE models by asymmetric compute allocation across visits; learning separate residual coefficients shows reduced-scale first visit improves efficiency under matched budgets
Read the original at www.reddit.com→Looped Mixture-of-Experts (MoE) models combine parameter reuse with sparse computation to improve performance under matched budgets. Prior work demonstrates that different loop visits can play different roles even...
Original headline: "[Paper] LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocation"
Coverage timeline
- Oct 23, 06:50 UTC r/LocalLLaMA lead source [Paper] LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocation