Researchers discuss dedicating a thread for benchmarks and sharing, noting concerns about training data and openness of benchmarking practices.
Read the original at old.reddit.com→Of course, the concern is that in the end, this thread will be fed into the models' training data, but I feel benchmarking isn't so open and very fragmented. submitted by /u/jinnyjuice [link] [comments]
Original headline: "'I ran my own benchmarks on it' seems to be pretty common comment around here. How about dedicating a thread for this and sharing?"
Coverage timeline
- Aug 3, 20:06 UTC r/LocalLLaMA lead source 'I ran my own benchmarks on it' seems to be pretty common comment around here. How about dedicating a thread for this and sharing?