Ensemble routing for prompt-adaptive cost-aware LLM inference; introduces PISA for prompt-adaptive, cost-aware routing and PISA-E for ensemble routing.
Read the original at arxiv.org→arXiv:2611.10193v1 Announce Type: new Abstract: Large language models (LLMs) increasingly support Web-based services that must balance response quality and inference cost across heterogeneous requests. Conventional...
Original headline: "Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference"
Coverage timeline
- Nov 10, 05:00 UTC arXiv cs.AI lead source Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference