Register and share your invite link to earn from video plays and referrals.

catid
@MrCatid
Training the models. Prior: Saronic, Anduril, Oculus VR, Game Closure, MSEE@GATech
925 Following    4.4K Followers
We think we’ve found a better path to continual post-training: merging models in input space. 🔹 A 256-token soft prompt (~0.01% of params) matches full-parameter RL 🔹 Plug-and-play composition: prompts trained separately just stack, with no joint training, and the combo beats the experts. No more MOPD! 🔹 Better OOD generalization, almost no forgetting 🔹 ~0 inference overhead, even with a different prompt per request. Built for personalization.
Show more
Language models process each token through a single residual stream (blue arrows): reading / writing memory and applying MLPs at each layer, and finally decoding the next token. Can we scale up this single stream computation? Introducing Latent Lattice Language Models (LLLMs): augmenting this vertical stream with a horizontal axis of computation—the Lattice of Latents (orange arrows). RNN 🤝 Transformers 🧵
Show more
“I want this country to realize that we stand on the edge of oblivion. I want every man, woman and child to understand how close we are to chaos. I want everyone to remember why they need us!" -V for Vendetta
Show more
Labeling video data with Sol-6.2 costs about 2x more than Sol-6.1 and Carbon right now. Combining Sol + Astra gives us about 90% score vs 89% for Sol 6.2 alone on synthetic benchmarks, while costing 5x more to do both. So Sol 6.2 is the best option right now in practice. Sol 6.1 scored 85% and merging it with Carbon scored 86%, for about the same cost as Sol-6.2, but also 2x slower and more complex.
Show more
GPT-6.2 is getting roasted on X meanwhile, Astra and Sol just took #1# and #2# on my object detection benchmark so I have some awkwardly positive results to share ↓ benchmarks and examples
Show more
If you ask an OpenAI model to label everything it sees in a video frame.. it will be extremely detailed. Compared to Carbon (lower right). Google tends to give just the obvious items. This is much more useful behavior for producing datasets for video models.
Show more
Upgraded teamcodex to support Astra-6.2 and it now has a 3 account limit to match policy:
Wow! Astra-6.2 is cheaper than running GLM-5.3. This is huge for people actually using agent swarms on real problems.
Pre-orders are open now for Framework Wireless Touchpad Keyboard. Finally, a living room keyboard you want to use.
0
127
1.9K
74
Forward to community
Anthropic investors seem to be funding a coordinated and obvious influencer campaign to discredit and downplay the latest OpenAI release. Tons of people on day 1 saying it sucks before I even have access to it in codex.
Show more
🚀 Stop using Transformers for weight-space programming If you are not training with gradients on a GPU, why default to a Transformer? When an LLM writes network weights directly, letting it also design the architecture achieves: 📉 ~1,800× fewer parameters at matched accuracy on ARC (AI2 Reasoning Challenge), compared to the best Transformer the same LLM wrote. ⚡ Up to 17× faster GPU inference with a memory architecture built for these tables, compared to a Transformer of the same size. 🧵 Paper & code at the end 👇
Show more
My theory is that the US is entering a war posture right now and “they” are intentionally breaking things one after the other to find where we are weak in the event of a real attack.
I'm adding value by periodically saying things to the swarm like "We're juggling a lot of agents - consider ways to improve the effectiveness of the swarm by adding new agents or passing some targetted suggestions" or "Add an agent working on checking for bugs"
Show more
Scaling matrix optimizers to wide networks is expensive. Our new research post introduces Echelon, a rank-anchored formulation of Muon that orthogonalizes only the top subspace in O(mnr) time while rescaling the tail: up to 10.5× faster optimizer steps, with Muon's loss at 1.3B. Blog: We pair the algorithm with the first Lean-checked convergence proof for spectral steepest descent. Lean proofs:
Show more
1/ From 7B to 30B parameters, zeroing fewer than 0.05% of frozen weights with GRPO matches full fine-tuning on math, and on Qwen2.5-7B it beats the base model even at pass@128 (80.08% vs 74.01%) without writing a single new weight. 🧵
Show more
Fruit-fly connectome optimized inference for DeepSeek-V4.1-Flash on 2× NVIDIA GB300 DGX Stations! It impressively hits 56k tok/s prefill and 201.7 tok/s c1 decode. Well done little dude.
Show more
Getting 261 tokens/second single session and 1479 tokens/second for batch 16 for DS4F with DFlash on GB300 DGX Station. Second one arrives tomorrow