Dave Blundin(@DaveBlundin):The mates are in agreement: we are looking at mainstream, beyond transformers architectures taking significant market share this year. @alexwg highlights MoEs, diffusion transformers, linearized attention, and recurrence as a disassembling of "Attention Is All You Need" until nothing original is left. I would add that @liquidai already proved there are many new, discoverable architectures. There are simply too many ways in which vanilla transformers are grossly inefficient.

Dave Blundin

@DaveBlundin

DB2. Co-Founder, DataSage, EverQuote & Vestmark. Godfather of Quantization. GP at Link Ventures.

加入 June 2012

143 正在关注 11.6K 粉丝

Dave Blundin@DaveBlundin

2026.08.21 20:54

The mates are in agreement: we are looking at mainstream, beyond transformers architectures taking significant market share this year. @alexwg highlights MoEs, diffusion transformers, linearized attention, and recurrence as a disassembling of "Attention Is All You Need" until nothing original is left. I would add that @liquidai already proved there are many new, discoverable architectures. There are simply too many ways in which vanilla transformers are grossly inefficient.

显示更多