Why MTP is slower than standard decode
Read the original at www.reddit.com→I'm trying to get Ornith 1.5 9B running (I can also barely run 35B A3B but I want something that I can run relatively easily) with the latest llama.cpp with this command: llama serve --flash-attn on --cache_type-k...
Original headline: "Why is MTP slower than standard decode?"
Coverage timeline
- Oct 12, 13:59 UTC r/LocalLLaMA lead source Why is MTP slower than standard decode?