Qwen4exp: build the QSA selection without the kq_mask; ggml-org/llama.cpp PR 30965
Read the original at www.reddit.com→gigabytes of VRAM saved (Qwen Flash Next) "This is one of the things Strata does to squeeze the most juice from the experts' cache." submitted by /u/jacek2023 [link] [comments]
Original headline: "qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp"
Coverage timeline
- Oct 23, 18:43 UTC r/LocalLLaMA lead source qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp