๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Anton Lozhkov
@anton_lozhkov
Open-sourcing Language Models @huggingface โœจ
๊ฐ€์ž… January 2015
341 ํŒ”๋กœ์ž‰ ์ค‘    2.5K ํŒฌ
For over two years, the largest open code dataset was The Stack v2โ€ฆ until today. ๐Ÿฅž The Stack v3 is out: the largest open code dataset ever released: 114 TB, 770 languages, 224M repositories, ~5T tokens of deduplicated and filtered source code. Fully open, no restrictively licensed code included. The upgrade: - v2 (2024): 68 TB raw -> 2 TB / ~550B tokens, 618 languages filtered - v3 (2026): 114 TB raw -> 15.9 TB / ~5T tokens, 713 languages filtered C++ x15, TypeScript x7.5, Rust x7, Python x4.8. Even the COBOL corner of GitHub got a bigger slice. Part of that is two fresh years of open source. Part of it is a bug we found in v2's deduplication - story below. ๐Ÿ‘€ Things that make v3 different: 1. Contents inline. The #1# complaint about v2 was "cool dataset, where's the actual code?" v2 shipped file IDs from the Software Heritage graph, and fetching contents was a DIY treasure hunt. v3 is self-contained: sources embedded directly, one row = one repository. Download finishes -> you start training. 2. Fresh crawl. v2 was a 2023 snapshot of even older crawls. v3 is a direct re-crawl of GitHub at the latest commit, completed by August 2025: 224M repositories, 44B files. Forks were only included if they had 5+ stars. Two ways in: ๐Ÿฅž stack-v3-train - near-deduplicated, quality-filtered, PII-redacted, contents inline. Point load_dataset at it and go. ๐Ÿ— stack-v3-full - the entire 114 TB corpus as an HF Storage Bucket: every duplicate kept with cluster IDs, stubs for excluded files. Roll your own dedup, filters, and mixes. The record isn't really ours to claim - it's millions of everyone's repositories, neatly stacked. Thanks to every developer who keeps their code public, the BigCode community, and friends who built the infrastructure that made this possible!
๋” ๋ณด๊ธฐ