AGORA uses item-response theory to weight LLM judges for reliability-aware aggregation
Read the original at arxiv.org→arXiv:2611.05556v1 Announce Type: new Abstract: LLM-as-a-judge is a common substitute for costly human evaluation, and aggregating several judges is the standard remedy for any single judge's bias. Judge pools mix...
Original headline: "AGORA: Reliability-Aware Aggregation of LLM Judges via Item-Response Theory"
Coverage timeline
- Nov 5, 05:00 UTC arXiv cs.AI lead source AGORA: Reliability-Aware Aggregation of LLM Judges via Item-Response Theory