🔥 Ultra-FineWeb-L1 is here —— 1T+ tokens of high-quality web data are now available!

High-quality data is the foundation of powerful LLMs. Ultra-FineWeb-L1 is now open-sourced as part of the UltraData ecosystem, providing a carefully processed English web corpus from Common Crawl.

Built for Better Data Quality
1️⃣ Fresh Web Data: Collected from recent Common Crawl snapshots, covering data up to CC-MAIN-2025-51, with 1T+ tokens and ~1.14B documents.
2️⃣ Refined Processing Pipeline: Combines text extraction, language filtering, heuristic filtering, sensitive-field replacement, deduplication, and customized cleaning to improve data quality.

⚡ Highlights:
- 1T+ tokens of filtered English web data
- Enhanced cleaning for noisy web content, encoding issues, and abnormal documents
- Designed as the L1 filtered layer in the UltraData management framework
- Aligned with the latest Ultra-FineWeb updates https://t.co/pXDfgulD38

🔗 Resources
🤗 Dataset: https://t.co/3aR5qtgw53
📄 Paper: https://t.co/Kg9LLUqZgB
🧩 Classifier: https://t.co/sP5yLlL68n