I tested a 1.56TB MoE model on a 6GB RTX 4050 laptop; here are the results.
Read the original at old.reddit.com→The Test Bench Setup I tested running a massive 1.56TB Mixture-of-Experts (MoE) checkpoint (96 shards, 93 layers, 896 experts/layer, ~4.46 bits/param MXFP4) on a budget gaming laptop. Laptop: HP Victus 15 GPU: NVIDIA...
Original headline: "I tried running a 1.56TB MoE model on a 6GB RTX 4050 Laptop, Here’s the result"
Coverage timeline
- Jul 29, 01:12 UTC r/LocalLLaMA lead source I tried running a 1.56TB MoE model on a 6GB RTX 4050 Laptop, Here’s the result