I re-trained the DFlash 2 drafter for Ternary Bonsai 2 27B: 2.2x on an L4, 3.2x on code edits with ngram lookup, 1.5x on a Mac, and 1.2x in Chrome.
Read the original at www.reddit.com→PrismML's Ternary Bonsai 2 27B fits a 24 GB card or Mac, but decodes at ~30 tok/s on an L4 and ~21 on an M4 Pro. z-lab's DFlash 2 drafter was trained on bf16 Qwen3.8-27B, so it guesses worse on the ternary model. I...
Original headline: "I re-trained the DFlash 2 drafter for Ternary Bonsai 2 27B: 2.2x on an L4 (3.2x on code edits with ngram lookup), 1.5x on a Mac, 1.2x in Chrome"
Coverage timeline
- Oct 7, 08:13 UTC r/LocalLLaMA lead source I re-trained the DFlash 2 drafter for Ternary Bonsai 2 27B: 2.2x on an L4 (3.2x on code edits with ngram lookup), 1.5x on a Mac, 1.2x in Chrome