Data Scout uses targeted web crawling guided by an LLM to build domain-specific pretraining corpora
Read the original at arxiv.org→arXiv:2609.05766v1 Announce Type: new Abstract: The dominant approach to building domain-specific pretraining corpora is to filter large web archives such as CommonCrawl. This works well for popular domains but...
Original headline: "Data Scout: Targeted Web Crawling for Domain-Specific Pretraining Corpora"
Coverage timeline
- Sep 9, 04:00 UTC arXiv cs.LG lead source Data Scout: Targeted Web Crawling for Domain-Specific Pretraining Corpora