GFX1030/Multiple V620 users: set -ub 384 and -b to a multiple of that to fix llama.cpp tensor split crashes
Read the original at old.reddit.com→Basically what the title says. For me, it would always crash and burn trying to use tensor split. Apparently, there's some bug where GPU memory gets corrupted with the default microbatch (512) or higher. I will be...
Original headline: "PSA for anyone with multiple V620's or other gfx1030 cards having problems making llama.cpp tensor split work -- set "-ub 384" and -b to a multiple of that depending on number of GPUs"