Mac inference backend enables larger MoE models on Macs with SSD offloading and Apple Silicon optimizations; author seeks benchmarks from others on M-series Macs.
Read the original at www.reddit.com→I've built an inference backend optimized for M-series Macs that makes it possible to run larger MoE models on machines without eating up all the RAM (for example, it can run Qwen 3.8 Flash Next on a 24GB Mac) using...
Original headline: "Anyone willing to benchmark my MoE runtime on their M-series Mac?"
Coverage timeline
- Nov 10, 20:57 UTC r/LocalLLaMA lead source Anyone willing to benchmark my MoE runtime on their M-series Mac?