SonicBoom v0.0.3 outperforms llama.cpp in decoding on a desktop RTX 3050 8 GB, with benchmark results shown for Gemma 4 E4B Q3_K_M and Qwen2.5-0.5B.
Read the original at www.reddit.com→Hi folks! We are HardenedLinux, an old-school free software community. SonicBoom is our open-source (GPLv3) inference runtime: a compiler-first alternative to the usual Python stack. No Python at build or run time —...
Original headline: "Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode"
Coverage timeline
- Oct 19, 03:40 UTC r/LocalLLaMA lead source Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode