Idea: compress Qwen 3.8 KV cache by using a single bit for the token "wait
Read the original at old.reddit.com→Not even sure if I'm joking, my thinking history is about 50% "wait". submitted by /u/pixelpoet_nz [link] [comments]
Original headline: "Idea: massively compress Qwen 3.8 KV cache by using a single bit for the token "wait""
Coverage timeline
- Aug 18, 18:22 UTC r/LocalLLaMA lead source Idea: massively compress Qwen 3.8 KV cache by using a single bit for the token "wait"