r/LocalLLaMA ·1h ago

Skepticism over near 400 tokens per second claim; author notes speed varies by prompt and demonstrates 376 tps on a realistic coding task

I woke up to see people accusing me of (and in some cases banning me for) lying about the top speed number, but that's just how MTP works. Different prompts are going to get different speeds. This one gets damn close...

Original headline: "Some people are skeptical of my (almost) 400 tokens per second claim from yesterday"

Coverage timeline

Oct 27, 11:59 UTC r/LocalLLaMA lead source Some people are skeptical of my (almost) 400 tokens per second claim from yesterday