Przemek Chojecki | PC
@prz_chojecki
Research-level Math Data + hard STEM & coding RL tasks for LLMs @ PhD in mathematics.
参加 April 2016
1.1K フォロー中 15.2K ファン
data is the moat
This is notable. DeepSeek, a lab usually first to pioneer novel algorithms and architectures, is saying that at this point, the ROI of improving data quality far exceeds that of working on novel post-training algorithms.
I think this has already been true for some time for non-lab practitioners. If you're doing llm post-training, 80% of your effort should go into looking at your data.
This means:
- Hiring experts to dig through your RL tasks
- Sifting through rollouts and sft data by hand to remove suspicious samples. Make sure all tasks are actually passable.
- Making sure your data is diverse in both difficulty and category.
もっと見る
