Llama.cpp adds WebGPU Q2_0 support with testing on RTX 3060 using native Dawn WebGPU
Read the original at www.reddit.com→Tested on an RTX 3060 with native Dawn WebGPU. llama-bench -m MODEL -p 512 -n 128 -r 5 -ngl 99 -dev WebGPU -t 2 -fa off Model Test master (t/s) PR (t/s) Ternary Bonsai 1.7B Q2_0 pp512 35.03 ± 0.46 1915.91 ±...
Original headline: "webgpu : add Q2_0 support by teddygood · Pull Request #30914 · ggml-org/llama.cpp"
Coverage timeline
- Oct 22, 09:17 UTC r/LocalLLaMA lead source webgpu : add Q2_0 support by teddygood · Pull Request #30914 · ggml-org/llama.cpp