Introducing Contrastive Language Model (CLM): an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions. CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks. With lightweight fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks. We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed. Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size. 📄 Blog: 💻 Code: 🗣️ Discord: 🤗 Data & Models: More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵

Really impressive results — self-verification scaling this well with DeepSeek V4 Flash is a big deal for open models. I recently open-sourced a DeepSeek Harness plugin that implements Best-of-3/5 LLM-as-a-Verifier for coding tasks: isolated candidates (worktrees), auto-test filtering, then the verifier ranks the best patch (with strict double-confirmation before applying anything). Would love feedback:

Took the LLM-as-a-Verifier paper and turned it into a simple /ultra command for coding agents 🚀 Run your task N times in parallel, each in its own git worktree. The same model then verifies the diffs (a Probabilistic Pivot Tournament) and applies the winner. Terminal-Bench, @deepseek_ai V4 Flash 0731 as both agent and verifier: 24% → 33% overall 40% → 75% on recoverable tasks Model-agnostic: it pushes whatever model you run to its best. An @opencode plugin + a CLI for any agent (Claude Code, @grok, @cline, @pidotdev). Spread attempts across one or several models in a single pass. MIT. npm: agent-ultramode Repo + how it works below.

For anyone wanting to read the research behind this, the paper is available on Papers with Code :) "LLM-as-a-Verifier: A General-Purpose Verification Framework" Find it here:

If LLM-as-a-verifier works as well as my last post suggests, this is going to be massive for local AI. Like the dev pointed out, pairing a large model with a small model lets you run verification on the cheap. This means if you combine the GLM-5.3 API with Deepseek-V4-Flash running on a DGX Spark, you get a huge jump in performance without driving up costs. If you have a 4xDGX Spark, you could multi-batch both GLM and Deepseek on a single node and beat Fable entirely offline without touching an API. Welcome to the era of local AI.

Our latest finding: scaling self-verification can make open-weight models significantly more capable at a fraction of the cost. With DeepSeek V4 Flash, sampling just 5 candidate solutions and using the same model to rank them with LLM-as-a-Verifier improves Terminal-Bench 2.1 accuracy from 79% → 88%—outperforming Claude Fable 5 while costing 11× less. 💰 As open-weight models become more capable, we can generate many high-quality candidate solutions and verify them at very low cost. Try it out: More on verification scaling in @jackyk02's previous post: @StanfordAILab @StanfordEng

Stanford is definitely cooking something crazy. By enabling self-verification during the intermediate process, Deepseek-V4-Flash easily outperforms Fable. 11x cheaper and a much higher Terminal bench score. This is exactly why I love X. People just casually drop massive breakthroughs in a single post. Github ⬇️

LLM-as-a-Verifier keeps pushing the frontier of cost vs. capability! On Terminal-Bench 2.1, it made DeepSeek V4 Flash accuracy go from 79% → 88%, while being 4-11x cheaper than competitors! Try it here: @jackyk02

Scaling self-verification with DeepSeek V4 Flash beats Claude Fable 5 on Terminal-Bench 2.1, while being 11x cheaper 💰 As open-source models become more capable, they can now generate large numbers of high-quality candidate solutions and verify their own outputs at very low cost. For example, we find that sampling just 5 solutions with DeepSeek V4 Flash and ranking them using the same model with LLM-as-a-Verifier can lead to a significant boost in accuracy (79% → 88%), outperforming closed frontier models on Terminal-Bench. Try it out today: More on verification scaling in my previous post.

Agents are becoming increasingly difficult to verify as they take on open-ended, real-world problems. That is why we’re organizing the 🔍 Who Verifies Agents? 🦘 Workshop at #NeurIPS2026#. We’re excited to bring together researchers working on robust verification, environment-grounded evaluation, process-level signals, simulators, agent safety, and real-world agent systems. Check out our website for more details, and follow our official account for future updates!

Thanks to @whoisnnamdi and @Lightspeedvp for the shout-out to LLM-as-a-Verifier on the Lightwork podcast! Check out the episode for a great explanation of "loop engineering" and agent verification:

Muse Spark 1.1 debuts on the TB2.1 leaderboard, punching above it's weight in terms of both cost and speed (and notably no detected reward hacks). Cheaper than all GPT 5.6 variants, it lands right between luna and terra. Full leaderboard in 🧵

Introducing Exo 🦋 – an open-source agent that can rewrite every part of itself. It learns new environments, adapts policies, even optimize costs (and play games!!) We've been hacking away at this project with @martin_casado from @a16z and @ankrgyl from @braintrust and friends.

Verification is emerging as a new scaling axis for AI! Scaling pre-training, post-training, and test-time compute have driven much of the recent progress in large language models. Our new work explores a fourth scaling axis: #verification# —the ability to determine whether a solution is actually correct. In LLM-as-a-Verifier, we introduce a general-purpose framework that provides fine-grained feedback across diverse modalities without additional training. We show that three simple ingredients—higher score granularity, repeated evaluation, and criteria decomposition—consistently improve verification performance. The approach achieves state-of-the-art results across robotics, coding, and medical AI, including RoboRewardBench, Terminal-Bench V2, SWE-Bench Verified, and MedAgentBench. I'm particularly optimistic about the implications for #Robotics# and #PhysicalAI#, where verification can serve as a dense reward signal for reinforcement learning, significantly improving the sample efficiency of SAC and GRPO and, in turn, enabling more capable and reliable autonomous systems. As AI continues to scale, I believe verification will become a foundational capability for building more capable and trustworthy autonomous AI agents. 🌐 Website: 📄 Paper: 💻 Code: Outstanding work led by @jackyk02, in collaboration with @shululi256, @pranav_atreya, @liu_yuejiang, @jyx_su, @chelseabfinn, @istoica05, and @Azaliamirh. #AI# #LLMs# #Verification# #Reasoning# #AgenticAI# #Robotics# #PhysicalAI# #ReinforcementLearning# @StanfordAILab @StanfordEng