JetBrains Mellum 2.1 12B A2.5B MoE achieves high throughput and coding capability across GPUs with 8–16 GB memory

It runs at around m4 16 at 60 tps at full 130k context, you can easily run a Q8 if you have a 16 GB GPU, I was able to get 140TPS on 9070xt, It’s an MoE so I think we can run it well on smaller cards too, 8GB, 12GB...