Q38FN with dflash2; lower draft model quant recommended for faster performance
Read the original at www.reddit.com→Back with Qwen 3.8 27B i heard the advice that a Q4 draft model would produce roughly the same acceptance rate as a Q8 or higher. I have had my agent create a Q38FN Draft model in F16, Q8 and Q4, and the Q4 seemed to...
Original headline: "Using Q38FN with dflash2? Try lower draft model quants."
Coverage timeline
- Oct 27, 10:35 UTC r/LocalLLaMA lead source Using Q38FN with dflash2? Try lower draft model quants.