Train on prefill, monitor at decode: TopicScope trains causal topic classifiers on SAE features to label every token and allow topic changes mid-response
Read the original at arxiv.org→arXiv:2611.12935v1 Announce Type: new Abstract: Topic monitoring is the task of labelling the topic of each token in a model's response during generation. Activation-based monitors that run at generation time have...
Original headline: "Train on prefill, monitor at decode: Using sparse autoencoders for topic monitoring"
Coverage timeline
- Nov 11, 05:00 UTC arXiv cs.AI lead source Train on prefill, monitor at decode: Using sparse autoencoders for topic monitoring