Prefill disruptions in MoE models: using fewer routed experts increases short-answer F1 loss; a small set of prompt tokens carries most of the damage.
Read the original at arxiv.org→arXiv:2610.22454v1 Announce Type: new Abstract: Running prompt tokens through fewer routed experts during prefill is a simple way to make mixture-of-experts (MoE) serving cheaper, yet it is unclear what this...
Original headline: "Where Prefill Expert Reduction Hurts MoE Models: A Few Prompt Tokens Carry Most of the Short-Answer F1 Loss"
Coverage timeline
- Oct 19, 04:00 UTC arXiv cs.CL lead source Where Prefill Expert Reduction Hurts MoE Models: A Few Prompt Tokens Carry Most of the Short-Answer F1 Loss