BitNet 2B4T on CPU outputs incorrect results; Bonsai 1.7B loads correctly everywhere, while ik_llama.cpp with -rtr is typically fastest on tested hardware
Read the original at www.reddit.com→I wanted to know which low-bit (1-bit / ternary / 2-bit) GGUF files actually give correct output on CPU, and how fast they run, so I compared each runtime against a reference instead of judging by how the text looks....
Original headline: "I checked BitNet and Bonsai on CPU against references: bitnet.cpp main still gives wrong output, Bonsai 1.7B is correct wherever it loads, and ik_llama.cpp with -rtr is usually fastest"