Update on the “Monstrosity”: 6 BC-250 board cluster testing Qwen Next Flash IQ2_XS and 3.6 35b Q4 setups with llama, Vulkan, and RPC over 1gb Ethernet
Read the original at www.reddit.com→This is 6 bc-250 ex mining boards with 5 in the asrock 4u12g case they came in. After a lot of testing my current preferred setup is 4 boards running Qwen Next Flash IQ2_XS at 100k context with around 28 tok/s for...
Original headline: "Update on the “Monstrosity”. 6 BC-250 board cluster"
Coverage timeline
- Oct 2, 02:49 UTC r/LocalLLaMA lead source Update on the “Monstrosity”. 6 BC-250 board cluster