Datology is one of the best (if not the best) data research labs. Their tech reports are packed with practical details and tricks for curating data properly across pre / post-training and evals. Some of my favorite resources and write ups from them:
- DatBench:
- DatSmith:
- Pretrain vs. CPT analysis:
- UberWeb:
- BeyondWeb:
Really cool to see them packaging these learnings / best practices into a unified platform!
Today, we’re excited to launch the Datology Curation Studio, customizable frontier data curation for every team training its own models.
Data quality is the ultimate compute multiplier, and Curation Studio makes it easy for everyone.