Two years ago Bernardo Kastrup was doing nights in an attic. Yesterday he announced a chip that consumes 100x energy
maybe the next 🇪🇺 success story?
dude is kind of a legend, ex-ASML, philosophy books, designed a chip aimed at neural nets that, on paper, burns a fraction of what a GPU burns because it stops dragging data back and forth across the package.
This week his company raised €200m+, with Samsung as a co lead Peter Wennink (ex ASML CEO) is chair.
they ship the physical system in 2028 (and license the silicon if someone else wants to build on it)
Everyone agrees inference is becoming the meter.
The technical problem is not mysterious, a lot of the watts never do the actual work. They pay for the trip between memory and compute. If you collapse that trip, you need fewer racks for the same token work, and basically that's the whole company.
the Samsung link is quite interesting. Earlier on they were already in the manufacturing conversation, and now they are in the round. They already sell the memory, if compute and memory get designed together, that is their fight.
Whether any of this works is still to see
rn: no public rack, no measured watt/dollar, and until those exist, the performance claims are just claims
but if that works, two things change. Cost per token stops being a GPU list price problem, and memory vendors stop being a side quest