Model evaluation question: users test newly released models with a prompt about describing main cast of a niche 2010s cartoon to gauge coding quality; many 30b models fail, some succeed.
Read the original at www.reddit.com→I generally just ask it "describe the main cast of (insert somewhat known cartoon show from the 2010s)", could either be Totally Spies, Randy Cunningham, Slugterra etc, most models in the 30b range completely fumble,...
Original headline: "What's your go to question to check if newly released model is just codemaxxxed slop for the leaderboards or not?"
Coverage timeline
- Oct 7, 21:02 UTC r/LocalLLaMA lead source What's your go to question to check if newly released model is just codemaxxxed slop for the leaderboards or not?