ik_llama.cpp fork update seeks feedback; reports speed comparisons on 6GB VRAM RTX 2060 and prefills speeds, with Strata benchmarks discussed
Read the original at www.reddit.com→Continuation of https://www.reddit.com/r/LocalLLaMA/comments/1wynomx/k_llamacpp_moe_optimizations_expert_residency/ This time I finally get speeds compared to Strata for ik_llama.cpp on 6gb vram rtx 2060 ( generates...
Original headline: "Just update of ik_llama.cpp fork and seek of feedback"
Coverage timeline
- Oct 24, 16:34 UTC r/LocalLLaMA lead source Just update of ik_llama.cpp fork and seek of feedback