Two flags raise Ling-3.0-flash INT4 performance on a DGX Spark from 20.8 to 38.7 tok/s
Read the original at old.reddit.com→The official INT4 does load on a single DGX Spark. The naive config just leaves most of its speed on the floor, 20.8 tok/s. Two changes take it to 38.7. Quick context on where this comes from: I work on Ling at...
Original headline: "Two flags took the official Ling-3.0-flash INT4 from 20.8 to 38.7 tok/s on one DGX Spark"
Coverage timeline
- Aug 9, 16:10 UTC r/LocalLLaMA lead source Two flags took the official Ling-3.0-flash INT4 from 20.8 to 38.7 tok/s on one DGX Spark