FULL INTERVIEW: @beffjezos says Z1T hits up to 140x the energy efficiency of a GPU, and 100x at the data center level. The end game, he says, is chips in space powered by the sun. @beffjezos is founder and CEO of @extropic. He joined @schisofrenia to launch Z1T, the first family of transformer-like models built for sparse probabilistic hardware: 01:12 what Z1T is, and how long they've been cooking it 02:32 why models on GPUs are dense and his chip isn't 03:15 a new scaling law where you scale connectivity, not just flops 04:08 how you get 140x, and why a pbit uses 10,000 fewer transistors 07:12 where Z1 sits next to GPUs, and why it's decode not prefill 08:37 100x energy efficiency at the data center level 09:14 you might see Extropic chips in space sooner rather than later 09:45 open weights vs an open training recipe 10:24 Z1T is barely GPT-2, and they started a run last night for GPT-3 11:05 the chips are cooking right now at a major fab overseas 12:58 gated convolutional attention, and the layer they haven't cracked 14:03 the Hardware Lottery, and why he wants people gambling again 15:20 what happens if inference gets 1,000x more efficient 16:59 why you can do this research without a basement full of GPUs