templar
@tplr_ai
incentivised internet-wide training - an order of @covenant_ai
4 Following 13.7K Followers
Tomorrow on TGIF, @DistStateAndMe recaps the busiest month since @covenant_ai launched.
Covenant-72B went viral. @FireworksAI_HQ cited PULSE in production. Sam sat down with @Jason on @TWiStartups. Autoresearch-RL extended Karpathy's framework to RL fine-tuning on @basilic_ai. And @tplr_ai Crusades multi-GPU tourney set new MFU highs.
First TGIF since March 6. A lot to cover.
Show more
Templar builds the foundational knowledge. Grail sharpens the reasoning through RL post-training.
@erfan_mhi's PULSE paper showed how to make distributed RL practical by cutting weight sync bandwidth by 100x. This week @FireworksAI_HQ confirmed the same approach works at 1T scale for @cursor_ai Composer 2.
When Templar's next models are ready, Grail's infrastructure is proven at scale.
Paper:
Show more
"Challenges the political economy of AI."
That's how Jack Clark ( @jackclarkSF), co-founded @AnthropicAI, ran policy at @OpenAI, described what Templar is doing. He's now featured it in Import AI twice.
1/n
Show more
On the @theallinpod this week, @chamath asked @nvidia CEO Jensen Huang about decentralized AI training, calling our Covenant-72B run "a pretty crazy technical accomplishment."
One correction: it's 72 billion parameters, not four. Trained permissionlessly across 70+ contributors on commodity internet. The largest model ever pre-trained on fully decentralized infrastructure.
Jensen's answer is worth hearing too.
Show more
We just completed the largest decentralised LLM pre-training run in history: Covenant-72B. Permissionless, on Bittensor subnet 3.
72B parameters. ~1.1T tokens. Commodity internet. No centralized cluster. No whitelist. Anyone with GPUs could join or leave freely.
1/n
Show more
This week on TGIF, @DistStateAndMe previews the Covenant72B report ahead of the full release on Monday. He'll walk through what the training run produced, what we learned, and what comes next.
Also on the agenda: multi-GPU Crusades went live this week and other updates across the ecosystem.
Show more
