Could we have a --disk-moe or --n-disk-moe like --cpu-moe or --n-cpu-moe to use disk/cpu/gpu?
Read the original at old.reddit.com→Explicit title, It would be nice to have the ability to have 3 tiers moe offload :( submitted by /u/storm1er [link] [comments]
Original headline: "Could we have a --disk-moe or --n-disk-moe like --cpu-moe or --n-cpu-moe so we can use disk/cpu/gpu ?"
Coverage timeline
- Aug 5, 21:58 UTC r/LocalLLaMA lead source Could we have a --disk-moe or --n-disk-moe like --cpu-moe or --n-cpu-moe so we can use disk/cpu/gpu ?