Swapping the judge changes the ranking: a preregistered study of cross-provider replication in LLM-judged system comparisons
Read the original at arxiv.org→arXiv:2611.04090v1 Announce Type: new Abstract: Practitioners who compare systems with an LLM judge and have no human labels often check a conclusion by repeating it with a judge from another provider. We test how...
Original headline: "Swapping the Judge Changes the Ranking: A Preregistered Study of Cross-Provider Replication in LLM-Judged System Comparisons"
Coverage timeline
- Nov 4, 05:00 UTC arXiv cs.CL lead source Swapping the Judge Changes the Ranking: A Preregistered Study of Cross-Provider Replication in LLM-Judged System Comparisons