Is there a Strata fork to run lfm 24b a2b on 16GB RAM; and how much tps per second and what quantization to use?
Read the original at www.reddit.com→Is there a fork of strata that let me run lfm 24b a2b on only 16GB of ram comfortably? And how much tps can I expect per second and what quantization should I use? and Thanks! submitted by ...
Original headline: "Is there a Strata for lfm 24b a2b"
Coverage timeline
- Oct 18, 18:52 UTC r/LocalLLaMA lead source Is there a Strata for lfm 24b a2b