Nanbeige 4.3-33B quantised GGUF; 33B model compresses from 35.2GB to 13.7GB with a 20.3GB sweet spot on 32GB hardware and 92.0% top-1 on AD-IQ4_XS vs BF16
Read the original at www.reddit.com→Quant ladder of Nanbeige's new 33B model from 35.2GB down to 13.7GB For 32GB hardware the sweet spot is 20.3GB AD-IQ4_XS with a top-1% of 92.0% compared to BF16 submitted by /u/gladkos [link] [comments]
Original headline: "Nanbeige4.3-33B Quantised GGUF"
Coverage timeline
- Oct 25, 21:33 UTC r/LocalLLaMA lead source Nanbeige4.3-33B Quantised GGUF