Diminishing returns of data in sparse autoencoders: functional behavior does not guarantee feature reproducibility
Read the original at arxiv.org→arXiv:2610.20680v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary....
Original headline: "Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility"
Coverage timeline
- Oct 16, 04:00 UTC arXiv cs.LG lead source Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility