BitRun enables 1-bit LLM inference on ordinary laptops with 98–127 tokens/sec on Apple M4 Pro and 40–45 tokens/sec on Intel Core i5; author fixes kernel math bugs in Microsoft research codebase bitnet.cpp
Read the original at www.reddit.com→I built BitRun because I wanted to run 1-bit ternary LLMs on ordinary laptops without melting my battery, spinning up a dedicated GPU, or wrestling with complex build environments. Right now, BitRun gives you 98 to...
Original headline: "I built BitRun: A standalone runner for 1-bit LLMs (100–127 t/s on Apple Silicon & 40–45 t/s on PC CPUs with zero compile)"
Coverage timeline
- Oct 14, 11:58 UTC r/LocalLLaMA lead source I built BitRun: A standalone runner for 1-bit LLMs (100–127 t/s on Apple Silicon & 40–45 t/s on PC CPUs with zero compile)