Virtual Head Attention (VHA) reduces KV cache overhead while maintaining quality and lowering training and prefill compute in long-context language models.

The design of attention mechanisms in long-context large language models faces an ``impossible triangle'' among KV cache size, training and prefill FLOPs, and model quality. GQA achieves strong quality but incurs...