8 agents run in parallel with 250k context each on two 3090 GPUs, achieving ~600+ tokens per second total
Read the original at www.reddit.com→I am wide open to the idea that I am misguided or misinformed, but my findings feel worth sharing: Short video of my UI with 8 agents each explaining a codebase in parallel long prose: https://imgur.com/JSqlXIi A few...
Original headline: "8 agents × 250K context, concurrently, on two 3090s (~600+ tok/s total)"
Coverage timeline
- Nov 2, 01:19 UTC r/LocalLLaMA lead source 8 agents × 250K context, concurrently, on two 3090s (~600+ tok/s total)