Increasing tokens per second by splitting workload across multiple slots on the same GPU for Qwen 3.8 27b, noting context reduction per slot and VRAM overlap considerations
Read the original at www.reddit.com→ok so i am trying to increase performance of llms i have running on my 7900 xtx via hipfire, i notice i get significantly higher total tokens per second when i run mutiple slots at once when running qwen 3.8 27b. i...
Original headline: "Splitting up work to be split across mutiple slots on the same gpu for higher total tokens per second"
Coverage timeline
- Nov 8, 14:21 UTC r/LocalLLaMA lead source Splitting up work to be split across mutiple slots on the same gpu for higher total tokens per second