Samip(@industriaalist):2/ As you scale the population, Dust beats backprop at 100k and 1M tokens and comes close at 10M and 20M. At 20M, the current trend even puts Dust's asymptote below backprop! EGGROLL is way worse at every budget. We describe in the post how Dust also integrates well with Adam.

Samip

@industriaalist

cofounder Q Labs ( solving sample efficiency and generalization

加入 June 2016

116 正在关注 6.8K 粉丝

Samip@industriaalist

2026.10.05 19:41

2/ As you scale the population, Dust beats backprop at 100k and 1M tokens and comes close at 10M and 20M. At 20M, the current trend even puts Dust's asymptote below backprop! EGGROLL is way worse at every budget. We describe in the post how Dust also integrates well with Adam.

显示更多