This is serious engineering work born in blood and sweat.
Introducing the Together Compiler: continual compilation for AI inference! I’m super proud to finally share this work from our research team! ✨
Today’s inference is “Polymorphic” or shape-shifting, it doesn’t sit still. On OpenRouter, 80% of traffic goes to models released in the last three months and the top ten most popular open models each use a different architecture. Our customers constantly change their usage behaviors. Together Compiler leverages agents to continually optimize our deployments immediately as these changes occur.
Together Compiler research highlights include:
1. 252x better throughput: TKCC lets agents try 252x more optimization experiments per hour compared to prior agent inference optimization systems.
2. State-of-the-art TPS across InferenceX workloads we tried: TKCC outperforms the best InferenceX engine across all tested settings in just 24 hours & 2 nodes per compile. Including 1111 TPS on MiniMax M3.1 B200 and 1205 TPS on DeepSeek v4.1 Flash GB300 at batch size 1. NVIDIA, AMD, Trainium2.
3. Continuous recompilation: TKCC currently serves many trillions of production tokens per day and adapts to traffic shifts within just a few hours of recompilation.
4. Bitter lessons: TKCC’s design builds on 2+ years of building frameworks to accelerate AI Inference, starting from ThunderKittens and Megakernels. Compilers have always used abstractions and IRs, but we show that agentic compilers now reach higher performance ceilings without abstractions.
From an incredible team
@reyna_abhyankar @dylan__lim @realDanFu @togethercompute!!
Show more
One potential positive is maybe we can finally start going back to building out incredible AI desktops, stacked with all the super fast RAM that's about to get blocked from datacenters in the US, and we can finally get self-sovereign, personal AI.
Show more
NBC News Exit Poll asked them about artificial intelligence. 57% said AI will do more to hurt society than help it. 76% said they oppose building a data center in their own community. Ouch.
Some thoughts on the election and what it means for the future of AI.
Expect gridlock. And this is a great thing. It will stop the most aggressively idiotic ideas (kill switches/total bans) from getting through as Trump goes Mr. Veto. Would have been even better if the house split, but same difference mostly.
What the world needs now is time. What do I mean?
People's first reaction to any tech is usually fear, but as they get their hands on it, figure out how it helps them in their lives, opinions can and do change gradually and then all at once.
As technology diffuses into the world, it's almost never anything like what the hypsters or fear mongers warn about and always something in between. Early fears prove nonsensical (cell phones cause brain cancer!) and those fears get replaced by more mundane problems/challenges (my kid spends too much time online and isn't socializing enough.)
Build outs of AI datacenters will not stop. There is a scarcity of compute and prices are just too expensive for tokens.
As SemiAnalysis wrote, the permits and everything else are already locked in through 2027. The problems will likely be in 2028 if some of the most idiotic populist nonsense gets traction. (
But those problems will be just another footgun for American because it means we will be moving our industrial capacity overseas AGAIN. A horrible and avoidable outcome. AI datacenters will simply move those high paying blue collar jobs overseas and countries that welcome power and tech build outs and tax incentives will get the datacenters.
Long the Middle East and Asia.
And it will move DCs into space too, but I am not long space yet because only one guy can do it, Elon.
If the "bubble" pops it will be because of the usual reasons (too much credit, other price pressures, etc) but it will not change the fact that this is one of the most, if not the most, important tech developments in the history of the world, big enough to put the printing press, the steam engine, the invention of money itself, the tractor and the Internet in the distant rearview mirror.
How you will use it will change as builders figure out the right form factor/interface. You can't just graft the old model onto the new and it takes time and energy to understand what the right balance is.
But anyone hoping that AI will not be useful and will just go away need to stop hitting the crack pipe so hard.
You already use AI every single day without even thinking about it. Your anti-lock breaks. When you search Google (long before the AI at the top.) When you use Google maps. When you use your cell phone to find directions.
Many drugs that you take were found through various AI discovery methods and drug screening methods. Protein folding already won the Noble prize and is accelerating medicine. Multiple anti-cancer drugs, designed by AI to target a specific person's cancer are moving forward and that is incredible stuff.
Self driving cars are saving actual lives right now for something humans do very, very badly. Humans kill 1.2 million people on the road every year and injure 50 million more and AI is already an order of magnitude better at driving.
There is no timeline, of any future in the multiverse, where AI is not useful and people just live without it.
All tech in history goes through a rapid build out and then breakdown phase as people figure out the best places for it and new business models to make it work in an endless push/pull of competing forces and fears and desires.
Even if this is a bubble (which it might be), it doesn't make the tech any less useful and it will only become more so in the years to come. died in the early heat death of the Dot Com bubble Internet but we got Uber/Lyft/Airbnb/Spotify/TikTok/QR codes/digital payments/cell phones and a thousand other useful services and inventions years later.
Things take time.
Every early tech causes fear. But the faster people get their hands on something, the faster that fear disappears like smoke in the wind and it just becomes another thing in our lives, like rocks or grass or a phone.
Also, this election wave was inevitable. This is how America does it. Without fail. We vote in one group. They claim a majority and the mandate of Heaven, only to get tossed out on their ass 4 or 8 years later.
It's by design. The Founding Father's knew absolute power corrupts absolutely, so best to design a beautiful, blunt instrument that just rotates power in and out every few years.
Works beautifully so far, if imperfectly.
Show more
We’re deep in the chaotic phase of enterprise adoption of agents and vibe coding.
Hardware shortages may *FINALLY* force us to go back to the basics and search for better and more efficient ML algorithms.
I had a really amazing idea for a product but then I went on Product Hunt and saw that it has been built 117 times in the last 10 seconds.
Brutal.
AI is fake. Also, AI is destroying all the jobs.
This is what happens when they don't teach critical thinking in school, boys and girls.
The AI bubble bursting is going to be one of the biggest reality checks in tech history. These companies have spent years convincing us that AI is the future while burning billions, destroying jobs, draining natural resources and forcing products nobody asked for into our daily lives. Eventually, the hype has to meet reality.
Show more
The Verification problem in action.
When you don’t read ChatGPT’s output
The great gift of agents is how they allow you to route around every insufferable gatekeeper in software.
Congrats to Kimi K3 and other open source models for democratizing AI. 😂
1820: 70% of people are farmers.
1920: 20% of people are farmers.
2020: 2% of people are farmers.
Tractors and pesticides will eliminate all jobs and create a singularity of total abundance.
No money will be necessary!
Show more
Nothing changed in any substantial way. Previously, the human did the work and then used the machine to validate it. Today, the machine is doing the work, and the human is validating it.
Humans were needed then and are needed now. It's no different from using a tractor, a crane, an electric screwdriver, or a dishwasher.
Show more
Moravec's paradox: AI can solve the Navier-Stokes problem but not do the job of Mary from accounting.
this terribly dumb feature was probably the only reason I was still using google search at all since it was faster than asking a normal AI chat bot for a coherent answer to a question
We're going to clean up a lot of Human Slop in the coming days. 😂
The Indian security company that used 4 as a private key also lacks basic account security.
To sign into your account, you enter your email and receive a 4 digit code.
There are no limits on login attempts. You can try all 10,000 combinations.
The Indian government uses this.
Show more
The real AI psychosis is thinking "this is just another tool" or "it's just a stochastic parrot" and "humans still need to review every line". You gotta snap out of it.
Last night an AI agent performed brain surgery on a mouse.
We gave it 19 gene therapy vectors in a spreadsheet, a mouse model of epilepsy, access to a motorize surgery rig with cameras and one goal: reduce seizures.
It picked the drug. It picked the brain region. It derived the coordinates from three papers. Then it drove the stereotaxic arms into the brain and injected 250 nanolitres while watching the entry site through the rig camera for bleeding.
A licensed vet gated every irreversible step and held a physical kill switch to stop the AI agent.
Thread on what it decided, what it saw, and the moment it refused to let me override it.
[Writing this on the spot out of excitement, but I've been 18hrs in front of my laptop coding and coordinating this, so keeping it brief and descriptive, more cohesive thoughts on this soon...]
Show more
It’s almost incomprehensible how wide the distance is between someone who’s working at the frontier with AI and basically everyone else right now.
The crazy thing is these lines are not converging, but only growing farther apart. If the curve of AI progress had leveled off, it would certainly have given most people and companies time to catch up or implement their workflows in a standardized way. The technology would have neutralized competitive advantage because everyone would be using it the same way.
But what’s happened is the opposite. The curve of AI progress is basically going vertical right now, which means if you had codified your processes 3-6 months ago you’re likely already behind again. Now imagine most companies that kicked into gear a year ago and thought that they were done. Even further behind now.
I’ve never seen a technology in my life and career that improves at this rate, which consequently means you need to upgrade your expectations at a similar rate. Our brains weren’t wired for this, and our organizational change management processes certainly weren’t either.
With any form of recursive self improvement, things just get wilder. It’s a time when you will need to be constantly reevaluating the things that didn’t work just a couple months ago to see if they now can be solved. AI native companies certainly have a strong advantage due to this right now, so now it comes down to can the bigger companies keep up. Wild times.
Show more
"Claudes generally established democracies, GPTs formed hives."
Awesome read.
In collaboration with
@olam_labs, we ran a series of experiments letting agent swarms compete in Minecraft and Mindustry.
We wanted to see what kinds of agent dynamics would emerge when they shared a world over long periods of time.
We found civil bureaucracy, thousands of telegraphic A2A messages, and OpenAI models retrieving the game's source code to beat Claude.
More on our findings below:
Show more
Once again your reminder that as frontier labs try to ban or restrict your access to open weights models, it's the open weights models defending against cyber assault.
Fight their laws by any means necessary.
Or we're going to be like French shops in the riots. Defenseless. Begging for thugs and thieves and scum to stop wrecking our life's work and smashing our stores and running off with everything we worked so hard to build.
Fight back. Fight back hard.
Show more
OpenCode is currently under a sustained cyberattack, and mid-incident they ran into the same problem Hugging Face did: the frontier models they were using for defense began refusing to help fight attacking agents because analyzing the attacks started triggering safety guardrails.
Show more