How much code do AI models and harnesses add? An open-source benchmark measures code complexity against human fixes
Read the original at www.reddit.com→Existing benchmarks measure the solve rate and token output/cost, but don't punish excess code or overengineering, which increases technical debt in the long term. Parsimony counts syntax units (statements, calls,...
Original headline: "How much code do AI models and harnesses add? I made an open-source benchmark to measure code complexity."
Coverage timeline
- Nov 8, 22:19 UTC r/LocalLLaMA lead source How much code do AI models and harnesses add? I made an open-source benchmark to measure code complexity.