Rylan Schaeffer
@RylanSchaeffer
Creating something new. Ex-Meta TBD. On-Leave from Stanford w/ @sanmikoyejo. Prev @ Gemini, MIT, Harvard, Uber, UCL, UC Davis
加入 October 2011
2K 正在關注 6.9K 粉絲
Really cool work on data-constrained pretraining!
Key idea is to assign repeated data an effectiveness: how many fresh tokens would have produced the same validation loss? That lets them put repeated and fresh data on a common scale.
1/7
顯示更多
