Ran Flint on RTX Pro 6000 in vLLM; benchmarks show LiveCodeBench v6 82.2 and SWE-bench Verified 80 with OKF knowledge base and webscraping, using 110–180 tok/s single-precision tokens
Read the original at www.reddit.com→Manged to get Flint running in vLLM (with the flint PR) on my RTX 6000 Pro. The model was using my harness well, using OKF knowledge base, doing webscraping, spoke in the character I gave it, etc. I ran a couple of...
Original headline: "Ran Flint on RTX PRO 6000 at NVFP4 in vLLM"
Coverage timeline
- Oct 22, 02:24 UTC r/LocalLLaMA lead source Ran Flint on RTX PRO 6000 at NVFP4 in vLLM