Emergence and sparsification of a syntactic circuit in language models during training on subject-verb number agreement
Read the original at arxiv.org→arXiv:2610.33299v1 Announce Type: new Abstract: To understand the emergence of grammatical behavior in language models, we track circuits for subject-verb number agreement in pythia-1b and ten seeds of pythia-410m...
Original headline: "The Emergence and Sparsification of a Syntactic Circuit"
Coverage timeline
- Oct 27, 04:00 UTC arXiv cs.CL lead source The Emergence and Sparsification of a Syntactic Circuit