Network inference using llama.cpp rpc-server across four systems with multiple GPUs and over 10000 version; experiments benchmark MoE models locally
Read the original at old.reddit.com→Llama.cpp is now over version 10000 so I wanted see what was new with bigger MoE models running locally using RPC-SERVER My test beds: All running Kubuntu 24.04 or 26.04 System 1: Ryzen 9 3950X, 64GB DDR4 RAM, Radeon...
Original headline: "Network inference using llama.cpp rpc-server 4 system 9 GPUs."
Coverage timeline
- Aug 2, 01:39 UTC r/LocalLLaMA lead source Network inference using llama.cpp rpc-server 4 system 9 GPUs.