"Bigger is better" may no longer be a given. This is an attempt to lift small models to frontier-level quality through training automation alone 🔬 Title: Tiny AutoScientist: Supersized Intelligence for Small Models URL: 🔬 Overview Tiny AutoScientist is an automated research system that automates the entire training and alignment process for the small models (roughly 0.8B-8B) commonly used in production, aiming to make them perform at frontier-level quality. ❓ Challenges Solved In production, latency, cost, and device constraints often push you toward small models. ・But small-model training is hyperparameter-sensitive, prone to overfitting, and tricky to handle ・So you're often forced into a painful choice between a small model that fits constraints or a large one with enough capability 💡 Methodology & How It Works ・It automatically co-optimizes your data and model-training recipes ・It self-improves both until quality converges on your objective ・It automates the full R&D loop once reserved for frontier labs, absorbing the hyperparameter sensitivity and overfitting that plague small-model training 📊 Experimental Results ・35% relative improvement over human-configured training ・Consistent gains across dataset sizes from 5K to 100K samples ・Works across multiple model architectures ・Delivers frontier-level performance in days instead of months 🌍 Use Cases It unlocks previously impractical use cases: edge deployment, on-device inference, latency-sensitive apps, and regulated industries with strict data boundaries. Since small-model tuning tends to be artisanal, automating it to beat human-configured runs carries real practical weight. #SmallModels# #AutoML#

If you love fine-tuning open-source models (like me), then listen. > Start with 1B, 2B, 4B, and 8B models. (Don't start with a 27B model or bigger at first.) > Use WebGPU providers. I use Google Colab Pro for any model smaller than 9B. A single A100 80GB costs around $0.60/hr, which is cheap. Enough for small models. > Don’t buy GPUs unless you fine-tune 7 to 10 models. You'll understand the nitty-gritty in the process. > Use Codex 5.5 × DeepSeek v4 Pro to create datasets. Codex to plan, DeepSeek v4 Pro to generate rows. > Use Unsloth's instruct models as a base from Hugging Face. Yes, there are others too, but Unsloth also provides fast fine-tuning notebooks. > Use Unsloth's fine-tuning notebooks as a reference. Paste them into Codex, and Codex will write a custom notebook with the configs you need. > Spend 1 day learning about: - SFT (supervised fine-tuning) - RL training (GRPO, DPO, PPO, etc.) - LoRA / QLoRA training - Quantization and types - Local inference engines (llama.cpp) - KV cache and prompt cache > Just get started. Claude, Codex, and ChatGPT can design a step-by-step plan for how you can fine-tune your first AI model. Future tech is moving toward small 5B to 15B ELMs (Expert Language Models) rather than general 1T LLMs. So fine-tuning is an important skill that anyone can acquire today. Tune models, test them, use them. Then fine-tune for companies and make a career out of it. (Companies pay $50k+ to fine-tune models on their data so they can get personalized AI models.) Shoot your questions below. I'll be sharing in-depth raw findings about this topic in the coming days.

🚀 Introducing FlashQLA: high-performance linear attention kernels built on TileLang. ⚡ 2–3× forward speedup. 2× backward speedup. 💻 Purpose-built for agentic AI on your personal devices. 💡Key insights: 1. Gate-driven automatic intra-card CP. 2. Hardware-friendly algebraic reformulation. 3. TileLang fused warp-specialized kernels. FlashQLA boosts SM utilization via automatic intra-device CP. The gains are especially pronounced for TP setups, small models, and long-context workloads. Instead of fusing the entire GDN flow into a single kernel, we split it into two kernels optimized for CP and backward efficiency. At large batch sizes this incurs extra memory I/O overhead vs. a fully fused approach, but it delivers better real-world performance on edge devices and long-context workloads. The backward pass was the hardest part: we built a 16-stage warp-specialized pipeline under extremely tight on-chip memory constraints, ultimately achieving 2×+ kernel-level speedups. We hope this is useful to the community!🫶🫶 Learn more: 📖 Blog: 💻 Code:

Practices for embedding AI agents into enterprise systems [Supervisor / Router] 💡 The key to "ask anything" enterprise AI is smart traffic control behind the scenes. A lightweight classifier instantly routes requests to the right domain agent. 🔥 Problems Solved - Cramming all capabilities into one giant prompt degrades quality - Cannot use optimized prompts, tools, and models per domain - Processing every request with the most powerful model breaks the budget - Unclassifiable requests get lost with no fallback 🏗️ Proposed Pattern A cheap, fast classifier determines user intent and delegates to specialized agents for sales, IT, HR, engineering, and more. Ambiguous inputs trigger clarification questions before routing. Each delegation carries permission caps, cost limits, and timeouts to prevent runaway behavior. A fallback route to a default agent or human escalation is always required for unclassifiable requests. ✅ Selection Criteria - Fit: company-wide deployments covering diverse operations, environments with multiple domain agents - Not Fit: single-domain agents where routing is unnecessary ⚠️ Pitfalls - Underestimating misrouting cost (sending complex tasks to small models tanks quality) - Relying on static rules without measuring classification accuracy fails as business evolves - Missing fallback routes cause unknown requests to loop endlessly 🛠️ Implementation Approach 1. Use a fine-tuned lightweight model (e.g., distilBERT) or rule-based classifier for intent classification to minimize latency and cost 2. Manage the routing table in a config file (YAML/JSON) so domain agents can be added or changed without code modifications 3. Always implement a fallback route -- either a default agent or human escalation via Slack -- for unclassifiable requests 4. Pass permission caps, cost limits, and timeouts as parameters to delegate agents, with policies centrally managed via OPA/Cedar 5. Continuously measure classification accuracy through A/B testing and weekly reports, retraining the classifier based on misrouting rates #AIAgents# #EnterpriseArchitecture#

We're introducing GLM-5.2, our latest flagship model for long-horizon tasks. It marks a substantial leap in long-horizon task capability over its predecessor GLM-5.1 and, for the first time, delivers that capability on a solid 1M-token context. GLM-5.2's new capabilities include: Solid 1M Context: A solid 1M-token context that stably sustains long-horizon work Advanced Coding with Flexible Effort: Stronger coding capabilities with multiple thinking effort levels to balance performance and latency Improved Architecture: We propose IndexShare, which reuses the same indexer across every four sparse attention layers, reducing per-token FLOPs by 2.9× at a 1M context length. We also improve GLM-5.2’s MTP layer for speculative decoding, increasing the acceptance length by up to 20% Pure Open: An MIT open-source license — no regional limits, technical access without borders Supporting long-horizon tasks starts with making long context engineering-usable: the model must maintain quality across long, messy coding-agent trajectories, not just accept more tokens. A 1M context is easy to claim, but much harder to keep reliable under real engineering pressure. To this end, we substantially expanded 1M-context training for coding-agent scenarios, covering large-scale implementation, automated research, performance optimization, and complex debugging. The result is a long-context system that is not only wide in scope, but solid in execution: a practical substrate for sustained engineering work. This capability is reflected in GLM-5.2's performance on three long-horizon coding benchmarks. FrontierSWE measures whether an agent can complete open-ended technical projects at the scale of hours to tens of hours, spanning systems optimization, large-scale code construction, and applied ML research. On this benchmark, GLM-5.2 trails Opus 4.8 by only 1%, while edging out GPT-5.5 by 1% and Opus 4.7 by 11%. On PostTrainBench, where each agent is given an H100 GPU and evaluated by how much it can improve small models through post-training, GLM-5.2 outperforms both Opus 4.7 and GPT-5.5, ranking second only to Opus 4.8. On SWE-Marathon, an ultra-long-horizon software engineering benchmark covering tasks such as building compilers, optimizing kernels, and developing production-grade services, GLM-5.2 still has room to grow, trailing Opus 4.8 by 13% while remaining second only to the Opus series. Across all three benchmarks, GLM-5.2 is the highest-ranked open-source model, showing that its 1M context has translated into practical long-horizon delivery capability.