PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States

Linear-attention language models keep their context in a recurrent state of fixed size, where an inference system stores one copy of this state for every sequence it decodes at the same time. Storing the state at...