Can a multi-GPU setup with 8x 5060ti/3060 and 256GB RAM run high-speed, high-precision MoE model inference, or are cheaper multi-GPU configurations suboptimal?
Read the original at www.reddit.com→4x 5060ti16gb 4x 3060 12gb 256gb ddr4 3200 ram (4-channel speeds) On a mobo with 128 pcie4 lanes I’ve been slowly collecting gpus whenever seeing deals over time and ended up with this. Ended up comparable in cost to...
Original headline: "Is this overkill or viable? (8x gpus, 256gb ddr4)"
Coverage timeline
- Oct 25, 00:57 UTC r/LocalLLaMA lead source Is this overkill or viable? (8x gpus, 256gb ddr4)