Pairing an M5 Ultra 256 GB with an M3 Ultra 512 GB for disaggregated prefill and decode using Thunderbolt 5 RDMA
Read the original at www.reddit.com→The problem I'm solving: I have an M3 Ultra 512 GB and the memory is still the reason to own it, but prefill on long prompts is slow. A 17.5k-token prompt takes 41 s before the first token. The M5 Ultra fixes prefill...
Original headline: "Instead of waiting for an M5 Ultra 512 GB: pairing a new M5 Ultra 256 GB with my M3 Ultra 512 GB for disaggregated prefill/decode (GLM-5.3-Flash, MLX, Thunderbolt 5 RDMA)"
Coverage timeline
- Oct 13, 19:32 UTC r/LocalLLaMA lead source Instead of waiting for an M5 Ultra 512 GB: pairing a new M5 Ultra 256 GB with my M3 Ultra 512 GB for disaggregated prefill/decode (GLM-5.3-Flash, MLX, Thunderbolt 5 RDMA)