Syntax heads influence model syntax predictions less than subjects; forcing heads to attend to distractors does not fully shift predictions, the study finds
Read the original at arxiv.org→arXiv:2610.19231v1 Announce Type: new Abstract: Transformer language models develop attention heads that specialize in syntactic relations, for example, attending from verbs to their subjects. Do these heads play a...
Original headline: "Do Syntax Heads Influence Model Behavior?"
Coverage timeline
- Oct 15, 04:00 UTC arXiv cs.CL lead source Do Syntax Heads Influence Model Behavior?