Which question is your attention metric answering? attention rows as compositional data
Read the original at arxiv.org→arXiv:2608.14712v1 Announce Type: new Abstract: Each row of a transformer's attention matrix is a probability distribution over tokens, and in trained models most of that probability lands on a single \emph{sink}...
Original headline: "Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data"
Coverage timeline
- Aug 18, 04:00 UTC arXiv cs.CL lead source Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data