I interviewed DHH, the creator of Omarchy - The Agent Operating system + Ruby on Rails, one of the most important frameworks of our time. He's a father, builder, race-car driver, and such an inspiring person. One of the most positive people I've had the honor of meeting.

@Austen @johnternus has the clear path to owning sovereign AI on the desktop and phone thanks to Jobs making uber own silicon AND ramping up support for @exolabs AND putting 128gigs+ in machines Predict they buy @perplexity_ai

Apple's AI strategy is Apple Silicon. The iPhone, iPad, MacBook, Mac Mini and Mac Studio all use the same hardware architecture. Apple Silicon is energy efficient, quiet, and the memory unit economics are incredible. Apple has leaned hard into Local AI, now serving every segment of Local AI, from SLMs running on an iPhone/iPad to 2T models running on a cluster of 4 x 512GB M5 Ultra with RDMA using @exolabs (the cluster has 2TB memory @ 4.8TB/s). Most inference will run locally, and Apple wants it to run on their silicon. The main issue I see right now is the software. We need a stable, canonical inference engine rather than 10 unstable, incomplete ones. Who is solving this?

Interesting article in @theinformation. Macs are Apple's fastest growing business right now, driven by massive AI demand. Apple's AI strategy IS the Mac. Trillion dollar opportunity imho.

why do I have four mac studios? because my inference cost is zero dollars. > 291B parameters > tensor-sharded across four ultra chips > wired in a thunderbolt 5 ring > at 80Gbps with RDMA sorry anthropic

exo is currently top of r/LocalLLM (in response to this tweet). answering questions there in detail on my alt: Longjumping_Crow_597. feel free to ask any questions, exo or local AI related. link:

2 years ago, we achieved the first big milestone with @exolabs. We clustered 2 MacBooks to run Llama 405B. It felt like magic. The consensus was running this model was only possible in a data center. We ran it on consumer hardware, on 2 M3 Max MacBook Pros. Most people thought it was a gimmick. It only ran at 2 tok/sec! But, we believed that improvements to the software, hardware, and models would all compound. So that maybe in a few years, we thought, this would improve 10x in software, 10x in hardware, 10x models = 1000x. That was the vision. We imagined a world where you would have frontier intelligence running quietly on your desk. Today is the day that vision became reality. The M5 Ultra is a 10x step-change improvement vs the M3 Max we originally clustered. That, compounded with software improvements like RDMA over Thunderbolt, MTP and better kernels, and high intelligence density models like Qwen 3.8 27B, means we now have 1,000x better Local AI than when we started. I am so grateful to the small group of people at Apple (including @doogie69 @awnihannun @angeloskath @DiganiJagrit @doogie69) who believed in this vision and had the foresight as well as the courage to take a swing at this early on. I'm confident they're just getting warmed up (looking forward to 4-bit / 8-bit compute units in M7 Ultra🤞). With the M5 Ultra Mac Studio, we are going to have unmetered tokens running at API speeds at effectively zero marginal cost, running on your desk, so quietly and consuming so little power you won't even notice it. Local AI is good now.

exo featured on Apple's new M5 Ultra Mac Studio and M6 / M5 Pro Mac Mini pages. Over the past year, we have worked closely with Apple on low-latency RDMA networking over Thunderbolt 5, enabling clusters of Macs to run massive models like Kimi K3 and GLM-5.3 at API speeds. With RDMA, aggregate memory bandwidth across Macs scales ~linearly. A cluster of 4 x M5 Ultra Mac Studios scale to an aggregate memory bandwidth of ~4.8TB/s. Previously, these were speeds only achievable with data center GPUs. Our vision is a data center on every desk. Apple Silicon's superior memory unit economics, power efficiency, and out of the box experience for Local AI make that possible. Thank you to @angeloskath, @awnihannun, @twid and countless others at Apple who tirelessly to bring this technology to the world.

If you're getting serious about local hardware for open-weight models, you need to checkout Local AI by @exolabs. Pick the hardware, model, and dimension you're evaluating. It's a think of beauty. Invite link