Apache-2.0 Tritium releases an open source ternary LLM engine in Rust/CUDA for quantization, serving, and training on consumer GPUs
Read the original at old.reddit.com→This post was not written by a clanker. Hey guys, I'm a comp sci major who wanted to introduce a cool project I built for quantizing models to ternary (1.58 bit) with as minimal of loss as possible, a process that...
Original headline: "Open Source Ternary LLM Engine in Rust/CUDA for Quantization, Serving, and Training of models on consumer GPUs, called Tritium (Apache 2.0)"
Coverage timeline
- Jul 31, 03:18 UTC r/LocalLLaMA lead source Open Source Ternary LLM Engine in Rust/CUDA for Quantization, Serving, and Training of models on consumer GPUs, called Tritium (Apache 2.0)