注册并分享邀请链接,可获得视频播放与邀请奖励。

Samip
@industriaalist
cofounder Q Labs ( solving data efficiency and generalization
加入 June 2016
122 正在关注    8.3K 粉丝
Replacing backprop with zeroth-order optimization is inevitable. Introducing Noise: a speedrun for pretraining without backprop. Last week we presented Dust, the first method to approach backprop with SGD for pretraining transformers. We think another 1,000x improvement on top of it is possible, and it will require *co-designing* credit assignment, optimizers, architectures, and eventually, hardware. We built Noise to iterate on this faster.
显示更多
0
25
786
67
转发到社区