Does MTP head load in VRAM by default?
Read the original at old.reddit.com→I ran into a doubt when using the following command. It seems that the System RAM usage keeps increasing even though there is >10GB of space left in VRAM while using the MTP mode. Does the MTP head load separately...
Original headline: "Does MTP head get loaded in VRAM by default?"
Coverage timeline
- Jul 30, 15:11 UTC r/LocalLLaMA lead source Does MTP head get loaded in VRAM by default?