FlagRot: selective rotation for low-bit KV caches reduces memory while preserving accuracy through per-piece rotation decisions

arXiv:2611.03992v1 Announce Type: new Abstract: Long-context inference is bound by the KV cache, which 2-bit codes could shrink about sixfold if their error stays in check. The usual remedy rotates keys before...