Gandalf: Flash Shall Not Die! Increasing flash storage lifetime for KV cache offloading in LLM serving
Read the original at www.reddit.com→So the last post on SSD wear was removed by the mods for some reason, even though it contained a lot of useful commentary from the community providing clarity on SSD wear. Highly relevant since many of the new models...
Original headline: "Gandalf: Flash Shall Not Die! Increasing Flash Storage Lifetime for KV Cache Offloading in LLM Serving"
Coverage timeline
- Oct 26, 00:46 UTC r/LocalLLaMA lead source Gandalf: Flash Shall Not Die! Increasing Flash Storage Lifetime for KV Cache Offloading in LLM Serving