SuTRA: structurally-unified tokenization with root awareness
Read the original at arxiv.org→arXiv:2608.18087v1 Announce Type: new Abstract: Existing subword tokenizers optimize statistical compression but ignore morphological structure, particularly the relationship between roots and affixes. This is...
Original headline: "SuTRA : Structurally-Unified Tokenization with Root Awareness"
Coverage timeline
- Aug 20, 04:00 UTC arXiv cs.CL lead source SuTRA : Structurally-Unified Tokenization with Root Awareness