Combining MTP with ngram-mod to improve repeated token output in coding tasks; user tests show bursts of output when repeating a code block with spec-type and spec-ngram-mod settings
Read the original at old.reddit.com→As I understood it, ngram helps with repeated token output, instead of re-generating identical output, the speculative decoding notices the same pattern in the past (e.g. think-block) and suggest a repeat,...
Original headline: "Combining MTP with ngram-mod worth it for coding?"
Coverage timeline
- Aug 18, 10:36 UTC r/LocalLLaMA lead source Combining MTP with ngram-mod worth it for coding?