OpenAI's analysis reveals issues in SWE-Bench Pro, a coding benchmark, raising concerns about AI model evaluation reliability and accuracy.
Read the original at openai.com→A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.
Original headline: "Separating signal from noise in coding evaluations"