Mixture of Wide Experts: Sparse routing for interpretable native coordinates
Read the original at arxiv.org→arXiv:2611.03655v1 Announce Type: new Abstract: Language models often spread a concept across many internal dimensions, making their features difficult to interpret. Sparse autoencoders (SAEs) recover clearer...
Original headline: "Mixture of Wide Experts: Sparse Routing for Interpretable Native Coordinates"
Coverage timeline
- Nov 4, 05:00 UTC arXiv cs.LG lead source Mixture of Wide Experts: Sparse Routing for Interpretable Native Coordinates