"Bigger is better" may no longer be a given. This is an attempt to lift small models to frontier-level quality through training automation alone 🔬 Title: Tiny AutoScientist: Supersized Intelligence for Small Models URL: 🔬 Overview Tiny AutoScientist is an automated research system that automates the entire training and alignment process for the small models (roughly 0.8B-8B) commonly used in production, aiming to make them perform at frontier-level quality. ❓ Challenges Solved In production, latency, cost, and device constraints often push you toward small models. ・But small-model training is hyperparameter-sensitive, prone to overfitting, and tricky to handle ・So you're often forced into a painful choice between a small model that fits constraints or a large one with enough capability 💡 Methodology & How It Works ・It automatically co-optimizes your data and model-training recipes ・It self-improves both until quality converges on your objective ・It automates the full R&D loop once reserved for frontier labs, absorbing the hyperparameter sensitivity and overfitting that plague small-model training 📊 Experimental Results ・35% relative improvement over human-configured training ・Consistent gains across dataset sizes from 5K to 100K samples ・Works across multiple model architectures ・Delivers frontier-level performance in days instead of months 🌍 Use Cases It unlocks previously impractical use cases: edge deployment, on-device inference, latency-sensitive apps, and regulated industries with strict data boundaries. Since small-model tuning tends to be artisanal, automating it to beat human-configured runs carries real practical weight. #SmallModels# #AutoML#

Are Small Models Really a Threat? The Anthropic Fear Factory just released a new report warning that small models may be only months away from self-replicating across cheap IoT devices like off-brand Android TV boxes and smart ovens. They say such an AI worm would be hard to shut down since there’s no central kill switch. We’ve gone from “The Terminator is coming” to “your toaster might kill you.” But the experiments don’t support the dramatic headlines. They ran a 26B model across 15 $55 TV boxes at 0.39 tokens per second. The end-to-end test used emulated devices with deliberately open ADB ports, so the “hacking” step was trivial. The capable model often ran on a separate server. Agents were harnessed and told to propagate. This is not proof of autonomous rogue AI worms in the wild. Nor is the threat even novel. Worms that do this without AI already exist. Mirai in 2016 scanned for default credentials and built massive DDoS botnets. Kimwolf has infected over two million devices, mostly cheap Android TV boxes, by hitting open ADB ports. Small models don’t currently add much that matters on these targets. The bottleneck has never been figuring out the device. It’s the huge number of devices that ship with open ports, default passwords, and no patching. Traditional worms already exploit that at scale. A practical attacker would use the best models available offline to design a better traditional worm, then ship the lean binary. You get the intelligence benefit at design time without dragging a slow, heavy model along with every infection. Embedding a weak model in the payload is the worse option for anyone who actually wants the largest, fastest botnet. The timing of Anthropic’s latest scare campaign is hard to ignore. The day before, Brian Armstrong posted that Coinbase fine-tuned an open-weight small model (Qwen-3.5-9B) on proprietary fraud data. It beat frontier models, including Anthropic’s own Opus 4.5, while being faster and much cheaper. In testing it cut fraudulent transactions by about 30%. This is the emerging pattern. For narrow enterprise workloads you don’t need a general superintelligence. You need a small specialist model trained on your own data. Cheaper to run, especially as compute gets tighter, and often better on the metric that matters. That is a direct threat to the frontier-company model of selling access to ever-larger general models. Safety advocates should be welcoming this development. If the real concern is existential risk from ever-larger frontier systems — the ones that could theoretically do sustained autonomous R&D or become hard to control — then a shift toward smaller specialized models is the safer outcome. Small models specialized on fraud detection or support tickets are not pushing the frontier. They are not in the category of systems that could become a new species. The theoretical loss-of-control risk is created by models getting bigger and more generally capable. Safety advocates focused on those risks should welcome companies having a cheaper alternative to constantly scaling general models. Instead we get dramatic warnings about small models at the exact moment they are starting to take real enterprise share from closed-model companies like Anthropic.

The model and harness are designed together because small models fail in harnesses built for frontier models. Portable is a minimal system prompt, skills that load on demand, connectors as compact CLI tools instead of MCP servers, self-verification, and an always-on sandbox.

Super happy to release SmolDataEnvs: 5,000 verifiable RL environment tasks for hill-climbing small models in code and data science by @adithya_s_k 100% open source: environments, evals, training!

In 6 months, 95% of tasks will be done by small 250B open source models The large models will perform very complex tasks including train the smaller models The majority of token usage will be small models

🚀 MERA: Model Evolution and Routing with Skill Adaptation for Agentic Systems at Scale How can we make small models stronger for on-device agent deployment? MERA uses stronger models to guide an iterative loop of RL/GRPO, skill learning, and router optimization. Student failures become verified demonstrations, reusable SkillBook procedures, and LoRA updates, helping the small model take on more work over time. 🔥 Results: Qwen2.5-Coder-1.5B: 28.7% → 49.7% coding pass Qwen3.5-2B on TAU-2: 14/35 → 18/35 Fine-tuned 2B matches an unadapted 4B model Don’t just route around small models. Evolve them. 📄 💻

🚀 MERA: Model Evolution and Routing with Skill Adaptation for Agentic Systems at Scale What if small models could improve themselves, not just be routed around? MERA evolves both small-model skills and weights, guided by stronger models. Student failures become verified teacher demonstrations, reusable SkillBook procedures, and LoRA updates. 🔥 Results: Qwen2.5-Coder-1.5B: 28.7% → 49.7% direct pass on coding tasks Qwen3.5-2B on TAU-2: 14/35 (40.0%) → 18/35 (51.4%) The adapted 2B model matches an unadapted 4B model Small models don’t just get routed around. They get better. 📄 Paper: 💻 Code:

Excellent LFM3 GGUFs from AtomicChat, as usual. These results also show how tricky it is to quantize small models while preserving BF16 fidelity. I usually aim for 98 or 99 EAR, but that looks tough for this model. 96 EAR might be a more realistic target. I’ll double-check the accuracy on a few task benchmarks. (Also interesting: the difference in next-token probability distributions between FP16 and BF16.)