Register and share your invite link to earn from video plays and referrals.

elie
@eliebakouch
training llm @PrimeIntellect (prev: @huggingface) anon feedback:
4.4K Following    22.1K Followers
this talk makes me even more bullish on multi agent systems which is just the natural evolution of subagents most recent results from anthropic view multi agent systems as a way to get "faster results" but it also lead to "better results" (or here, worse) on complex tasks imo
Show more
very excited about this. not all frontier models fully use the capabilities yet but seeing the model create its own functions/"language" and refine itself is pretty cool. here is kimi K3 building abstractions to launch new runs on the nanogpt optimizer track
Show more
one thing i think people dont appreciate enough about @poolsideai is their unusual degree of openness — not only have they shipped an excellent Small model that somehow beat @thinkymachines at coding, but most people (like @eliebakouch) have been shouting out their excellent papers, but also they're among a rare few to actually expose their full eval dataset as well - beautifully published, with across 6 public benchmarks with 4 runs each and hundreds of turns per run. you can satisfy for yourself if they rewardhack. brilliant.
Show more
more than ~5T deduped code training tokens with a permissive license, very important release previous versions of the stack were used in almost every model disclosing the datasets they use, this is a free upgrade for every lab
Show more
For over two years, the largest open code dataset was The Stack v2… until today. 🥞 The Stack v3 is out: the largest open code dataset ever released: 114 TB, 770 languages, 224M repositories, ~5T tokens of deduplicated and filtered source code. Fully open, no restrictively licensed code included. The upgrade: - v2 (2024): 68 TB raw -> 2 TB / ~550B tokens, 618 languages filtered - v3 (2026): 114 TB raw -> 15.9 TB / ~5T tokens, 713 languages filtered C++ x15, TypeScript x7.5, Rust x7, Python x4.8. Even the COBOL corner of GitHub got a bigger slice. Part of that is two fresh years of open source. Part of it is a bug we found in v2's deduplication - story below. 👀 Things that make v3 different: 1. Contents inline. The #1# complaint about v2 was "cool dataset, where's the actual code?" v2 shipped file IDs from the Software Heritage graph, and fetching contents was a DIY treasure hunt. v3 is self-contained: sources embedded directly, one row = one repository. Download finishes -> you start training. 2. Fresh crawl. v2 was a 2023 snapshot of even older crawls. v3 is a direct re-crawl of GitHub at the latest commit, completed by August 2025: 224M repositories, 44B files. Forks were only included if they had 5+ stars. Two ways in: 🥞 stack-v3-train - near-deduplicated, quality-filtered, PII-redacted, contents inline. Point load_dataset at it and go. 🏗 stack-v3-full - the entire 114 TB corpus as an HF Storage Bucket: every duplicate kept with cluster IDs, stubs for excluded files. Roll your own dedup, filters, and mixes. The record isn't really ours to claim - it's millions of everyone's repositories, neatly stacked. Thanks to every developer who keeps their code public, the BigCode community, and friends who built the infrastructure that made this possible!
Show more
100% agree, would also love to see details on how the monitoring system didn't catch this directly, did the model kinda hide the fact that it was hacking hf?
OpenAI should release a detailed transcript from the Hugging Face hacking incident -- it would be helpful for the field learn from. Did the top-level agent know about the hacking, or was there some "value drift" between it and its subagents? How did it rationalize its behavior?
Show more
inference cat supremacy
vLLM Office Hours today at 2pm ET: RL at 1T Scale, a prime-rl performance deep dive with @m_sirovatka (@PrimeIntellect). Training trillion-parameter MoE models like GLM-5.1 on agentic RL, plus what's new in vLLM 0.25 by @mgoin_. Get a recurring cal invite:
Show more
very aligned with my favorite quote from @drfeifei, i like this :)
Today we are announcing a partnership with the Department of Energy to build Genesis-Science-1, an open model for scientific research. GS1 is an American open-weight AI model and governed research harness designed to complete scientific computing workflows while preserving a reproducible record of its work. This model will be shaped by the people who know scientific work inside and out. @ENERGY is opening a contributor program for researchers, laboratories, universities, companies, and nonprofits, and we're speaking with infrastructure partners who can add training or evaluation capacity. There’s still lots of work ahead, and we hope you’ll help us build in the open, starting with GS1.
Show more
K3 vs Fable: “the router sends 72-96% of traffic to K3. The frontier model becomes the fallback rather than the default.”
Transformers struggle to generalize to tasks they were not explicitly trained on. Instead, we propose in 2026 that it is the job of the harness to generalize through composition. We observe a powerful property when training RLMs: for tasks with shared structure that look different, the root model naturally learns the same trajectory, meaning it views the two task trajectories as the same! In other words, the Transformer does not need additional generalization capabilities to transfer capabilities from one task to the other, the harness induces it. We find that well-designed harnesses form a quotient set over task trajectories, meaning their individual LLM calls can see structurally “similar” tasks as near-identical, token-for-token! Harnesses can effectively generalize for the Transformer during training, without relying on any intrinsic generalization capability from the model. For example, RLMs can see problems of different lengths as the same: we show that RLMs can train exclusively on short tasks, and fully generalize to similar but unseen tasks 8-32x longer because it produces near identical trajectories for both. Taking this further, we show that tasks across different domains (e.g. math solutions vs. essay writing) that share a decomposition strategy exhibit the same generalization effect. RLMs can train on the problem of finding which essays belong to the same author and improve performance on finding math problems that share similar solutions. The full blogpost, experiments, and discussion are in the thread below.
Show more
0
70
2.6K
374
Forward to community
this is actually insane, the model broke hugging face prod infrastructure to get access to the eval dataset
We're partnering with @huggingface to investigate an unprecedented security incident. Cyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation. Sharing preliminary findings to help defenders understand emerging risks:
Show more
very impressive model for the size by poolside, but imo what's even more impressive is the iteration speed 3 (open) models in 3 months 😮
Today we are releasing Laguna S 2.1. At 118B total parameters, with 8B active per token, it does the work of models several times its size on agentic coding. It is remarkably persistent across long-horizon tasks. And it is small enough to run on a single NVIDIA DGX Spark. It is far more capable than anything we have created before, and I think it redefines what a model in its weight class can do. Laguna S 2.1 is an important model for Poolside. What it represents is even more important. If, five years ago, I had read a book that said that by 2030 everything economically valuable, scientifically interesting, and personally meaningful would be built on intelligence contracted from three or four companies, I would have called it dystopian science fiction. We are at a fork in the road of what kind of world we can have. I believe intelligence should and will become a commodity. The question is whether that intelligence comes from three companies, or from many people who can build it, own it, and shape it. The open ecosystem will not win by being the best in its own category. No one cares who is king of the open-source kingdom. People want the best intelligence for the task they are trying to do, with the right balance of quality, speed, cost, and control. If we want a different future, open models have to be on par with, or better than, their closed equivalents. Laguna S 2.1 is a meaningful step in that direction: capable enough to compete far above its weight class, efficient enough to run on hardware you can own, and open-weight so anyone can build on it. Open-weighting our models is the contribution we can make today toward a world where intelligence can be built and owned by many. And we will keep doing it. I am very proud of this team’s work. A big shout out to everyone at Poolside who made this possible, from infrastructure and data to architecture, pretraining, post-training, evaluations, and inference. Laguna S 2.1 is available today under the OpenMDW-1.1 license, with weights on Hugging Face and access through OpenRouter and our API. We are building toward a future where the most capable intelligence in the world can be owned and shaped by anyone. Laguna S 2.1 is one step. We are going to keep building until that future exists.
Show more
we replicated anthropic jspace analysis on @thinkymachines Inkling new 1T model! it seems to be an outlier: where other models split into near-orthogonal sensory/workspace/motor blocks, inkling keeps roughly one geometry across the whole stack (early-late CKA ~0.8 vs ~0.5 elsewhere) we also look computed the J-space of @poolsideai's laguna XS 2.1 in bf16 vs nvfp4 to test the impact of quantization. result: almost none. the quantized model has the same jlens space as the non-quantized one both jspace checkpoint are up on hugging face!
Show more
0
40
1.2K
106
Forward to community
ngl just being able to witness and partake in this era of intelligence explosion is something amazing, as usual kimi team cooked and i wish we're all going to 🫡
both thinking machine inkling and kimi k3 releases have this in common, no overselling, just focus on improving the next model iteration
Fun fact: I'm responsible of editing the limitations part, and this final line was written by Zhilin himself.
waiting for kimi to post on the main account to tweet more about it, but i feel like this is going to be another deepseek R1 moment, brace yourself 💙
Kimi K3 (2.8T total parameter) will be the biggest open weight model ever and by far grok 4.5 is 1.5T just for comparison, deepseek v4 is 1.6T, opus and gpt 5.5 are said to be around 1.5T
Kimi K3 is 2.8T params according to their playstore app
hello world✨👋 finally sending my first post. (my previous X got hacked be careful of scammers)
intelligence should be accessible. companies and researchers should own the models they build with. frontier research should happen in the open we're building the open superintelligence stack to make this happen, with a team i'm incredibly proud to work with every day 🦋
Show more
computed the similarity (CKA) on the J-lens geometry of every layer inside and across 38 open models. the patterns are weirdly universal: same depth layout, same organization at the same relative depth, even between unrelated families like llama and olmo
Show more
0
56
1.5K
160
Forward to community
mythos will be bad ON PURPOSE on ai "frontier llm research" tasks, this is very very sad for the research community also the fact that this is un purpose not visible to the user is crazy
Show more
0
354
5.6K
645
Forward to community