Every so often you meet a founder so sharp that every conversation lights up new pathways in your thinking and quickly deepens your grasp of their business and technology. Working with people like that is a privilege.

Training a 100B parameter model normally means renting a block of matched, high-spec hardware and holding it unbroken for the length of the run. Orion-100B ran across five data centres instead, on single A100s at around $1.25 an hour each. That puts a full replica at roughly $20 an hour, and the entry cost about 2.5x below approaches that need high-spec nodes throughout. It ran as a viability proof rather than to a finished model, and it held. We're in Montreal on Monday talking about what that changes. @ExploitSummit

We’re excited to sponsor @a16z speedrun’s AI Faire at @SFTechWeek on October 9. Runware founder Ioana will be presenting, alongside live demos from AI startups across the speedrun community. Request a spot!

Exploit Summit opens Monday in Montreal. We'll be talking about training on compute that was never set aside for the job, and what it takes to keep a run alive when the hardware underneath it keeps changing. Find us there.

We trained Orion-16B without a reserved cluster anywhere in it. The numbers underneath that: It sustained around 80k tokens per second across 4090s, 5090s, A100s, L40s, and A6000s, at roughly 20% MFU. The accessible pool came to about 18 B200s equivalent at FP16. Nodes dropped, throughput swung, and machines ran inconsistently throughout. Training carried on without direct intervention. Imperfect compute, one finished model.

Grok 4.7 is the current Grok version, which means we're only 3 steps away from Grok 5. Since a new Grok model is scheduled to be released every month, December is the most likely release date for Grok 5.