vLLM runs on 4 x Intel B70 with QFN 3.8 FP8; LLM tests show 450 t/s prefill and 20 t/s generation
Read the original at www.reddit.com→I think it's all working. I asked various models to make it all work for me, with the bulk being GLM 5.3 flash. Its not as fast as some of the benchmarks I've found but getting them working apparently requires even...
Original headline: "vLLM finally working on 4 x Intel B70, QFN 3.8 FP8"
Coverage timeline
- Oct 17, 02:33 UTC r/LocalLLaMA lead source vLLM finally working on 4 x Intel B70, QFN 3.8 FP8