Researchers note an 18GB unified memory setup equating to 14GB VRAM plus 4GB system RAM, with trends toward very large and very small models and gaps in between
Read the original at old.reddit.com→Equivalent to 14GB of VRAM and 4GB of system RAM given the limits I assigned to the GPU. EDIT: we be getting lots of big (≥20B), small (≤4B) models these days but very little in between submitted by ...
Original headline: "*Cries in 18GB unified memory*"
Coverage timeline
- Aug 14, 15:25 UTC r/LocalLLaMA lead source *Cries in 18GB unified memory*