Older Nvidia Maxwell GPUs run Qwen3.8-Flash-Next-GSQ-RCO-IQ1 on Strata with 32GB RAM, delivering notable prefill numbers; Qwen3.8-Flash-Next-GSQ-RCO-Q2 can run faster with more RAM
Read the original at www.reddit.com→2 Quadro M6000 24GB (48GB VRAM) running Qwen3.8-Flash-Next-GSQ-RCO-IQ1 on Strata since I only had 32GB of RAM. Pretty impressive prefill numbers on this inference engine. The decode has always been decent for the...
Original headline: "older Nvidia Maxwell running Strata"
Coverage timeline
- Oct 19, 16:02 UTC r/LocalLLaMA lead source older Nvidia Maxwell running Strata