Small language models under 3B parameters can use entropy-based confidence signals to improve accuracy on NLU benchmarks; token-level entropy early stopping and uncertainty-aware routing to larger experts are evaluated across model pairs.
Read the original at arxiv.org→arXiv:2609.20824v1 Announce Type: new Abstract: We explore whether entropy-based confidence signals can be leveraged to improve the accuracy of Small Language Models (SLMs) with fewer than 3 billion parameters,...
Original headline: "Do small language models know what they don't know?"
Coverage timeline
- Sep 21, 04:00 UTC arXiv cs.CL lead source Do small language models know what they don't know?