Running a 120B model across six devices that had no business loading it; a cluster of a 12GB Windows laptop, an RTX 3060 mini PC, Mac mini, M3 MacBook, 2017 Intel MacBook, and an Android device
Read the original at www.reddit.com→https://preview.redd.it/dnoc1qink9uh1.png?width=1161&format=png&auto=webp&s=5b910520480b75e2e3cadccf5e7841f8854e6cf5 ok so I genuinely did not think this was going to work. None of these machines could load a 120B...
Original headline: "Ran a 120B model across 6 computing devices that had no business running it!"
Coverage timeline
- Oct 8, 15:44 UTC r/LocalLLaMA lead source Ran a 120B model across 6 computing devices that had no business running it!