Ternary Bonsai 2 (2B, 4B, and 9B) released on Hugging Face, with the 2B able to run entirely locally in the browser at up to 500 tok/s on an M4 Max using custom WebGPU kernels.
Read the original at www.reddit.com→Models on HF: https://huggingface.co/collections/prism-ml/bonsai-2-small WebGPU demo: https://huggingface.co/spaces/webml-community/ternary-bonsai-2-hermes-webgpu submitted by /u/xenovatech [link] [comments]
Original headline: "Ternary Bonsai 2 (2B, 4B, and 9B) just released on Hugging Face. The 2B can run entirely locally in the browser at up to 500 tok/s on an M4 Max with custom WebGPU kernels."
Coverage timeline
- Oct 15, 20:09 UTC r/LocalLLaMA lead source Ternary Bonsai 2 (2B, 4B, and 9B) just released on Hugging Face. The 2B can run entirely locally in the browser at up to 500 tok/s on an M4 Max with custom WebGPU kernels.