4x 2080Ti tensor parallel feasibility and performance question
Read the original at old.reddit.com→Hi there, I have a setup with 4x 2080Ti 22GB, but am not able to see much benefits in running models larger than 16-20GB in speed. I did some research and saw something like tensor parallels are possible, but also...
Original headline: "4x2080Ti 22GB tensor parallel possible?"
Coverage timeline
- Aug 8, 04:59 UTC r/LocalLLaMA lead source 4x2080Ti 22GB tensor parallel possible?