8B Llama model runs at 17,000 tokens per second on a single chip
Read the original at www.reddit.com→I found this project that I never saw talked about enough which is absolutley baffling to me. The title is not clickbait, and I can't blame you if you thought it is, because I was in your shoes once. Meet ChatJimmy....
Original headline: "8B Llama model running at 17,000 tok/s on ONLY a single chip!!!"
Coverage timeline
- Oct 21, 12:30 UTC r/LocalLLaMA lead source 8B Llama model running at 17,000 tok/s on ONLY a single chip!!!