Speed of prefill for Qwen 27b and Qwen Flash remains stagnant despite faster decode; user reports improving prefill with QFN but accuracy tradeoffs ongoing
Read the original at www.reddit.com→I see everyone improving decode with bespoke models, like Sushi or Splash with Qwen 27b and Qwen Flash Next, and I'm running QFN now at 40tk/s on my M3 Max which is awesome. But prefill feels stagnant. I was trying...
Original headline: "Is anyone working on improving prefill speed? (Qwen 27b, Flash)"
Coverage timeline
- Oct 15, 09:38 UTC r/LocalLLaMA lead source Is anyone working on improving prefill speed? (Qwen 27b, Flash)