r/LocalLLaMA ·4h ago

I built a tool to test which weights matter before quantizing, quantizing one weight group at a time and measuring divergence with KL across prompts; applies to Qwen3.6-27B across Bedrock/Tightrope/Gambit

Most quantization works like this: pick a bit depth, apply it everywhere, maybe let imatrix take a rough guess at what matters, ship it. Most don't check which specific weight groups can take a hit and which ones...

Original headline: "I built a tool to actually test which weights matter before quantizing, instead of guessing (Qwen3.6-27B, 3 builds: Bedrock/Tightrope/Gambit)"

Coverage timeline

Jul 28, 04:19 UTC r/LocalLLaMA lead source I built a tool to actually test which weights matter before quantizing, instead of guessing (Qwen3.6-27B, 3 builds: Bedrock/Tightrope/Gambit)