๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Guangxuan Xiao
@Guangxuan_Xiao
Pretraining @thinkymachines | Ph.D. @MITEECS
๊ฐ€์ž… February 2020
877 ํŒ”๋กœ์ž‰ ์ค‘    5.9K ํŒฌ
8.4T tokens in 68 hours, natively in NVFP4 ๐Ÿ”ฅ that was the main pretraining phase for Inkling 2 Small. no magic, just solid things done right. so proud of this team! this is just one rung of our predictive scaling ladder ๐Ÿชœ more coming soon!
๋” ๋ณด๊ธฐ