Trust-me benchmarks on on-device agent platform prep; on IGX Thor RTX PRO 6000 GPUs, authors report latency and concurrency limits for large-context, multi-agent on-device inference
Read the original at www.reddit.com→A month ago we picked up a few IGX Thor boxes equipped with RTX PRO 6000 GPUs. I shared some numbers earlier running Qwen 3.8 27B on SGLang. These machines were bought specifically to run our upcoming on-device agent...
Original headline: "Some "trust me bro" benchmarks while we prep the repo to open-source"
Coverage timeline
- Nov 3, 22:27 UTC r/LocalLLaMA lead source Some "trust me bro" benchmarks while we prep the repo to open-source