You can now train your own decision model in our platform AC2! Jev-like models, custom fit for your use case :) We implement decision models through a language-model backbone with a small learned output layer called a decision head. The backbone processes the input, question, and answer choices, and the head produces scores which are softmaxed to produce a probability distribution over answer choices. Our software interface makes it easy to add a variable-sized decision head to frontier open-weight models. Choose any open source model, and train it as a decision model using Brier loss or cross-entropy loss. Training updates both the decision head and the language backbone. We tried it on 50k sampled training datapoints from the Civil Comments dataset, starting from Perplexity’s pplx-decider-v1-27b, and F1 for detecting toxic comments increased from 0.482 to 0.677 on 20k sampled eval datapoints. The training job took less than an hour on four B300 GPUs. Reach out to get started with your own classification task:

We're sharing a key step towards enabling Continual Learning at Applied Compute. In On-Policy Self-Distillation, the teacher model supervises training with privileged information: a hint. The training signal depends heavily on hint quality, but manually refining hints across runs is impractical, and isolating their individual impact is difficult. We instead optimize hints automatically using efficient proxies for their eventual training value.

When is harness optimization enough? When does post-training make sense? What will determine who becomes the new AI hyperscaler? @ypatil125, co-founder of @appliedcompute, was the perfect person to discuss this and more. Yash worked on Codex at OpenAI before leaving to co-found Applied Compute with fellow OpenAI researchers Rhythm Garg and Linden Li. Sixteen months in, the company is a unicorn, and is serving enterprises including DoorDash and Cognition. We cover: 0:00 Intro 1:42 The Case for Owning Your Intelligence 4:35 OpenAI x Baseten and the Multi-Model Future 7:19 Can Post-Training Beat Frontier Models? 9:03 What Data Stays Out of Distribution 11:51 Data Efficiency and Continual Learning 14:31 When Updating Model Weights Makes Sense 16:01 Online Training in Practice 18:16 Lowering the Cost of Post-Training 19:48 Does Every Company Need Its Own Model? 24:06 RL in Non-Verifiable Domains 26:59 How Applied Compute Picks Customers 28:46 Building an Inference Business 34:43 The Case for a New AI Hyperscaler 38:21 Building RL Environments 41:33 Quickfire YouTube: Spotify: Apple:

At Applied Compute most of our RL and eval pipelines grade with an LLM judge. These produce one result per rubric criterion, typically a paragraph of text accompanying a score. For some complicated tasks, grading can be a slow, pricey part of the loop. We swapped a grader on a code QA benchmark in AC2 for a decision grader: one call, a yes/no probability for every criterion at once. On 100 sample tasks (407 criteria), same answers: → ~32× cheaper
→ ~8× faster
→ 94% agreement with the LLM judge And it’s even better than the original judge: When the decision grader was confident, it matched the LLM judge 99% of the time. When it wasn't, the decision grader indicated low confidence (30-70%), pointing to ambiguous rubric items. This allowed us to identify noisy tasks and make the dataset cleaner. A cheap, fast, calibrated grader means you can grade more rollouts, ask more questions per call, and downweight or review the shaky calls instead of training on them.

We're adding reward-hacking monitors into AC2. Every RL run gets checked for cheating, and our research agent, Ari, automatically investigates the flags. After a model discovers an exploit, it can become its default strategy within a few training steps. We used to hunt for reward hacking manually, reading traces and running one-off agent queries. This was error prone and didn't scale. We decided to automate detection instead.

I implemented a system in @appliedcompute’s platform for automated failure mode clustering with Jev to surface errors at an even larger scale than before. RL training produces billions of tokens in traces. I always manually read many traces to understand model behavior, but finding agent failures (like reward hacking / hallucinations) at scale is easy to miss without automation. Here’s how it works: