Revision prompting reduces output tokens by trading decoded output for prefilled input tokens
Read the original at old.reddit.com→TL;DR: If you re-run the same prompt whenever the input changes, try sending the old input/output plus a diff of the input, and ask the model for a patch to the output. You generate ~2-10x fewer output tokens, and...
Original headline: "Revision Prompting: Trades slow (decoded) output tokens for cheap (prefilled) input tokens."
Coverage timeline
- Aug 11, 12:22 UTC r/LocalLLaMA lead source Revision Prompting: Trades slow (decoded) output tokens for cheap (prefilled) input tokens.