Joined the 4x R9700 Club; initial benchmarks for QFN 3.8 show ~190 tok/s user prompts, prefill at ~6,800 tok/s, and per-user follow-ups at 80–225 tok/s, with a worst-case ~2.5 minutes to first token.
Read the original at www.reddit.com→Posted this on the r/R9700 subreddit as well, but figured I'd share my initial findings here. tl;dr - One user writes at about 190 tok/s. Prefill runs at about 6,800 tok/s total no matter how many users there are....
Original headline: "Joined the 4x R9700 Club | Initial Benchmarks for QFN 3.8 Below"
Coverage timeline
- Nov 8, 23:36 UTC r/LocalLLaMA lead source Joined the 4x R9700 Club | Initial Benchmarks for QFN 3.8 Below