Enabling PCIe peer-to-peer for consumer Nvidia cards yields more performance than expected
Read the original at old.reddit.com→Disclaimer - no LLM was used to write this post/note As larger post about my setup will come later, want to give heads-up to folks who use VLLM and >= 2 GPUs. So I have pretty meaty server (8 channel AMD EPYC,...
Original headline: "enabling PCI-E p2p for consumer Nvidia cards will yield you more than you think"
Coverage timeline
- Aug 8, 21:42 UTC r/LocalLLaMA lead source enabling PCI-E p2p for consumer Nvidia cards will yield you more than you think