Loopie: a pair of Mixture-of-Experts looped transformers claim “most powerful looped Transformer to date” with 20B and 6B parameter variants
Read the original at arxiv.org→arXiv:2607.16051v2 Announce Type: new Abstract: We present the Loopie series, consisting of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6B-parameter model with 0.6B...
Original headline: "Loop the Loopies!"
Coverage timeline
- Jul 20, 04:00 UTC arXiv cs.CL lead source Loop the Loopies!