FleetSieve selects measurements for SLO-aware LLM fleet configuration to reduce unnecessary profiling; it targets resource-coupled decisions on tensor-parallel and replica counts
Read the original at arxiv.org→arXiv:2608.19659v1 Announce Type: new Abstract: Choosing tensor-parallel (TP) degrees and replica counts for an LLM serving fleet is difficult because performance is not monotonic in TP and the feasible choice can...
Original headline: "FleetSieve: Decision-Critical Profiling for SLO-Aware LLM Fleet Configuration"
Coverage timeline
- Aug 21, 04:00 UTC arXiv cs.LG lead source FleetSieve: Decision-Critical Profiling for SLO-Aware LLM Fleet Configuration