Rylan Schaeffer
@RylanSchaeffer
Creating something new. Ex-Meta TBD. On-Leave from Stanford w/ @sanmikoyejo. Prev @ Gemini, MIT, Harvard, Uber, UCL, UC Davis
参加 October 2011
2K フォロー中 7.1K ファン
Really cool work on data-constrained pretraining!
Key idea is to assign repeated data an effectiveness: how many fresh tokens would have produced the same validation loss? That lets them put repeated and fresh data on a common scale.
1/7
もっと見る
