Streaming quantile matching for ordinal-aware LLM-as-a-judge calibration; quantiles, not points, improves post-hoc alignment with human ratings
Read the original at arxiv.org→arXiv:2611.00447v1 Announce Type: new Abstract: LLM-as-a-judge scores are often distributionally shifted from human ratings on any given evaluation task, and post-hoc calibration (mapping LLM to human scores in a...
Original headline: "Quantiles, Not Points: Streaming Quantile Matching for Ordinal-Aware LLM-as-a-Judge Calibration"
Coverage timeline
- Nov 3, 05:00 UTC arXiv cs.AI lead source Quantiles, Not Points: Streaming Quantile Matching for Ordinal-Aware LLM-as-a-Judge Calibration