How many layers to place on CPU/RAM vs GPU in MoE models; user asks for specs and speeds across hardware configurations
Read the original at old.reddit.com→I am trying to figure out if it's worth upgrading my RAM, but I've noticed that some MoE models don't seem to do well with many layers shared from VRAM --> CPU/RAM. This may be something on my end; a software config...
Original headline: "Those who use many layers in CPU/RAM and some in GPU - what are your specs and speeds?"
Coverage timeline
- Jul 29, 14:54 UTC r/LocalLLaMA lead source Those who use many layers in CPU/RAM and some in GPU - what are your specs and speeds?