DeepSeek-V4-Flash-0731 performs worse on oneshot evals than Kimi K3 in cost and score, with $1.29 for 34 prompts and 2.7/5 score versus $0.44 and 3.2/5 for Kimi K3.
Read the original at old.reddit.com→I ran the newly released DeepSeek-V4-Flash-0731 in my oneshot eval harness across 34 prompts and here are the results. https://oneshotlm.com/model/deepseek-deepseek-v4-flash-0731/ The providers on openrouter were...
Original headline: "DeepSeek-V4-Flash-0731: Oneshot evals, surprisingly not token efficient??"
Coverage timeline
- Aug 1, 11:41 UTC r/LocalLLaMA lead source DeepSeek-V4-Flash-0731: Oneshot evals, surprisingly not token efficient??