POSPAN: position-constrained span masking for language model pre-training
Read the original at arxiv.org→arXiv:2609.16061v1 Announce Type: new Abstract: Span-level masked language modeling (MLM) has shown to be advantageous to pre-trained language models over the original single-token MLM, as entities/phrases and their...
Original headline: "POSPAN: Position-Constrained Span Masking for Language Model Pre-training"
Coverage timeline
- Sep 16, 04:00 UTC arXiv cs.LG lead source POSPAN: Position-Constrained Span Masking for Language Model Pre-training