Qwen3.8 27B solves 43% of tasks in Elixir, OCaml, Haskell, and 10 other languages, up from 17% with Qwen3.6 27B; benchmark covers 202 real GitHub tasks across 13 languages.
Read the original at www.reddit.com→Hi! My team and I built a benchmark of 202 real GitHub tasks in 13 languages that rarely show up in coding benchmarks: Elixir, OCaml, Haskell, Clojure, Kotlin, Swift, Julia, R and others. I think a lot of people do...
Original headline: "We tested Qwen3.6 27B, Qwen3.8 27B and GLM-5.3 Flash on real tasks in Elixir, OCaml, Haskell and 10 other less common languages"
Coverage timeline
- Nov 10, 14:48 UTC r/LocalLLaMA lead source We tested Qwen3.6 27B, Qwen3.8 27B and GLM-5.3 Flash on real tasks in Elixir, OCaml, Haskell and 10 other less common languages