Delta Inference Engine v0.93 adds 64/96/128 GB RAM support, Unsloth quants, and multi-GPU basics; runs 41 tok/s at 192 GB RAM with improved quality and a new native Engram reader.
Read the original at www.reddit.com→Big update to the Delta Inference Engine, you guys asked for more features, more benchmarks and less clickbait. I'm here to deliver The short version: I've added almost everything requested and more. Delta Inference...
Original headline: "DeepSeek-V4.1-Flash-1017 on one RTX 3090: Big updates to the Delta Inference Engine v0.93. Same 41 tok/s at 192GB/IQ2 at better quality, usable from 64 GB RAM, first token ~40% sooner, new Unsloth quants at full speed, EXL3, basic multi-GPU, public CI and more"
Coverage timeline
- Nov 5, 18:23 UTC r/LocalLLaMA lead source DeepSeek-V4.1-Flash-1017 on one RTX 3090: Big updates to the Delta Inference Engine v0.93. Same 41 tok/s at 192GB/IQ2 at better quality, usable from 64 GB RAM, first token ~40% sooner, new Unsloth quants at full speed, EXL3, basic multi-GPU, public CI and more