Samip(@industriaalist):2/ As you scale the population, Dust beats backprop at 100k and 1M tokens and comes close at 10M and 20M. At 20M, the current trend even puts Dust's asymptote below backprop! EGGROLL is way worse at every budget. We describe in the post how Dust also integrates well with Adam.

Samip

@industriaalist

cofounder Q Labs ( solving sample efficiency and generalization

参加 June 2016

116 フォロー中 6.8K ファン

Samip@industriaalist

2026.10.05 19:41

2/ As you scale the population, Dust beats backprop at 100k and 1M tokens and comes close at 10M and 20M. At 20M, the current trend even puts Dust's asymptote below backprop! EGGROLL is way worse at every budget. We describe in the post how Dust also integrates well with Adam.