Kimi K3 quantized to 8, 4, 2, and 1 bit; 1-bit model is 594 GB and preserves 78.9% accuracy for local use, released by Unsloth
Read the original at old.reddit.com→The model was quantized to 8, 4, 2, and 1 bit. Characteristics: Q8: 8-bit 1.56 TB, lossless Q4: 4-bit, 1.51 TB Q2: 2-bit: 861 GB Q1: 1-bit, 594 GB The smallest Q1 model keeps 78.9% accuracy, while being almost 3...
Original headline: "Kimi K3 for local use (1.56TB → 594GB) compressed and released by Unsloth"
Coverage timeline
- Jul 29, 19:39 UTC r/LocalLLaMA lead source Kimi K3 for local use (1.56TB → 594GB) compressed and released by Unsloth