Agree. It's much harder to measure agents' economic outcomes today. I have been exploring this with a few of my automations (basic RSI) @dair_ai and starting to see some interesting results. The challenge has been finding the right market feedback/signals, but with enough insights and iterations, you can really get something interesting going. I find agents terrible at negotiating, and that's the next thing I'm focusing on: what context and signals the agent needs to make better decisions. So many interesting things to test there.

Own your intelligence stack, folks! If you train or fine-tune your own models, you also need a way to prove they are yours once the weights are out in the world. Model fingerprinting does this. The Fingerprinting Leaderboard from @Sentient_found is an open benchmark for these methods. It scores each one on how much model quality it keeps, how well it survives removal attempts, and how reliably the owner can detect it.

Agents are quickly becoming your API and docs' main users. The @readme team says agents now make up 72% of traffic to API docs. And when an agent gets stuck, it rarely goes looking for the docs. Restless puts the fix steps and a doc link right in the error response. If the agent is still stuck, it can report what is missing, and that turns into doc updates or issues. I build a lot of agents that call APIs, and I want every API to have this.

New paper from Microsoft and colleagues on generating enterprise data for agents. If you train or evaluate agents on long-running business work, it's worth your time. (bookmark it) MiniCorp is an office simulator. A firm of agents runs an e-commerce company, discusses decisions in messages and meetings, and sets prices, inventory, and ad spend. A simulated market with customers and adaptive competitors responds, and the firm reacts to the results. The market is checked against published studies of real ones. A four-week 20% discount lifts units by 71%, and sales stay 31% above control after prices return to normal. Checkpoints let the same market state replay with a different decision, giving paired outcomes that real company records cannot provide. Paper: Chat with Paper:

Pay attention to decision models. From my recent tests, decision models can be extremely transformative for businesses. Many calls in an agent workflow are structured decisions, so paying frontier prices doesn't make sense. SubQ Snap from @subquadratic is trained with RL for those calls. It holds 384K tokens of context and leads Jev on 13 of 16 enterprise benchmarks, so it can decide with a whole repo or a stack of contracts in view.

Your agent traces are your best eval data. The most important failures only show up in production, and they rarely make it into your test set. DigitalOcean's Signals flags sessions where users got frustrated or tools failed and turns them into evals with one click. Simulations then replay them against your fix on your agent's real harness before it ships. If you're running agents in production, you need this loop.

Must-read paper on the effectiveness of multi-agent systems and persistent memory. We have recently seen many viral examples of multi-agent systems scaled via persistent memory. This work takes a closer look. They test one three-tier chain-of-agents system. Splitting long-context work across agents cuts peak KV cache to 14.3 MiB per query, against about 35 MiB for single-pass and RAG baselines. The persistent trace-recall tier adds 0.368 MiB of peak cache and changes accuracy by +0.015, with a confidence interval that crosses zero. The authors explain why. Single-question benchmarks give each item its own evidence and score it independently, so stored traces have nothing useful to recall. Paper: Chat with Paper:

This will be a fun one. @MilksandMatcha runs DevX at Cerebras and @vivianmshen is a General Partner at YC. Two people who work deep in AI every day are now talking about what life inside Silicon Valley is really like. SOTA Wives just launched. Watch the intro below.

Nice paper on agent memory. It explores an effective routing mechanism for memory in agents. The overall finding is that you want a separate retrieval strategy for each type of memory instead of one embedding search over everything. The authors prove that any single retrieval strategy has an upper bound on expected precision when the memory store holds several types. MemoType classifies each memory and each query by type, retrieves memories that match the query type, and applies the strategy for that type. A new dataset, TriMEM, provides the type labels needed to train the classifier. Recall@1 improves by up to 16.18% across three datasets. Paper:

Pay close attention to custom harnesses. This is a super interesting paper showing the impact of training inside the harness. Qwen3-14B goes from 22.2% to 54.8% on Spider 2.0-SQLite when it is trained in the same execution harness it uses at deployment. Text-to-SQL models are usually trained to write one static query. Deployed database agents inspect schemas, run probe queries and revise, and the harness for that only appears at inference time. HarnessSQL builds isolated executable databases with hidden answer checks. Teachers run inside the target harness; we keep only verified trajectories for SFT, then apply execution-reward RL. In addition, Qwen3-8B rises from 15.5% to 45.2%, and both models transfer to BIRD-Interact and LiveSQLBench. Paper: Chat with Paper:

Recommended paper from Sakana AI on recursive self-improvement. They propose an interesting way to scale recursive self-improvement through multi-agent self-supervision. In this line of research, self-improvement loops usually need an external verifier, so open-ended tasks without a checker are left out. MASS removes that requirement. One base model proposes multi-agent workflows, runs them and grades them, and an evolutionary search keeps the workflows that score best. The model is then fine-tuned on its own traces, and the improved model starts the next cycle as a better optimizer and grader. Two cycles on Qwen3.6-27B raise performance per output token from 1.2 to 1.6x on four open-ended benchmarks. A student trained on multi-agent traces also beats a single-agent student trained on 1.4x more tokens. Paper: Chat with Paper:

Super interesting NVIDIA paper on choosing base models for coding agents. It's actually a clever way to rank base checkpoints by how well each one is likely to do as a coding agent after post-training. They document that base models are really hard to evaluate on agentic coding tasks. They ran six base models on SWE-bench Verified, and five of them solved zero tasks. So instead, they look at the one step in a coding run that actually fixes the task. In other words, they take tasks that a strong post-trained agent already solved, replay its code edits one by one, and run the tests after each edit. The first edit that makes the tests pass is the decisive edit. Then they give the base model everything that happened before that edit and check whether it can come up with that fix. They score this in three ways. They check how likely the base model is to write the fix, whether it can pick the fix out of a set of rejected patches, and whether any fix it writes on its own passes the tests. All three rankings closely match post-trained SWE-bench Verified scores across ten base and post-trained model pairs. Why is this useful? If you pick checkpoints for agentic post-training, this method can give you a signal before you spend the training budget. Paper:

Banger paper from Meta Superintelligence Labs on user simulators for agent training. Agent RL setups usually let an assistant LLM play the user, so the simulated user is too cooperative and too explicit. A fixed GPT-5.5 agent finds tau-bench tasks easier with these users than with real people. This work trains MIMESIS, a 9B user simulator, on human conversations and 13 behavior patterns observed in real users. It beats Claude Opus 5 on behavioral fidelity by 13.4 points. Agents trained against it outperform agents trained against GPT-5.5 under all nine user simulators they never saw. They find that adding a coaching step that turns the simulator's private reasoning into feedback adds further gains. Paper:

Step 5 Preview from @StepFun_ai hit #1# on OpenRouter Trending, a day after launch. It's available in Kilo Code, Cline, Hermes Agent, and OpenCode. Keep your workflow and just switch models. I've tested the model. It’s a great model and has some nice properties. I've been using it as a coding agent since early access. It checks its own work and stops when the task is done. I would use it for long agent runs that I don't watch closely. StepFun built it for real engineering work. That covers fixing bugs in unfamiliar codebases, implementing features across files, and refactoring, with changes that run and pass checks. Frontend is the other big focus. You can give it a screenshot, a mockup, or a PRD, and it can return a working web page. It can also build a playable browser game from a set of assets. Beyond code, it reads annual reports and operating data and turns them into financial reports, editable spreadsheets, and slide decks. The context window is 1M tokens, with up to 1M output tokens. That is enough for large codebases and long multi-step tasks. Under the hood, Step 5 Preview is a sparse MoE with 600B total parameters and 27B active per token. Open weights are coming on Oct 15. Very much looking forward to its release.

Bullish on this trend of making post-training more accessible. A new post-training era is upon us. If you work on agentic RL, long-context tasks (a big focus today) are expensive, inefficient, and don't scale well. I've been diving into RL envs and evals for long-context tasks, and I can see this being useful. In agent RL, rollouts use most of the tokens. Every turn re-reads the whole growing context, including tool outputs, files, and earlier turns. Tinker just cut the price of those tokens. Long-context prefill and sampling now cost the same as short context. This means that evaluating your trained models on long inputs also gets cheaper. Huge win here. I believe RL will keep unlocking specialized models that slash the cost of critical agent operations. Cheaper long rollouts make them more practical to build. Own your intelligence stack!

NEW: Microsoft also releases its System One model, Microsoft-Decision-1. What a crazy effect Jev has had in the space. Also, a very cool application of decision-making models to power LLM judges and improve scientific discovery pipelines (e.g., screening candidate hypotheses). I'll add this model to my ever-growing list of eval runs. In my ongoing evals, consistency and more complex decision-making are two areas where I see these decision models struggle. I'm also exploring a bunch of science applications at @dair_ai and will share when ready.

Great feature in Codex. I have had my own composer prediction tool in my agent orchestrator for months. It's tunable and adapts to my preferences as I use it more. In fact, I use smaller models for this, like Haiku and Luna. It's a nice quality-of-life little feature that makes agents a bit more proactive and boosts productivity.

Step 5 Preview from @StepFun_ai hit #1# on OpenRouter Trending, a day after launch. It's available in Kilo Code, Cline, Hermes Agent, and OpenCode. Keep your workflow and just switch models. I've tested the model. It’s a great model and has some nice properties. I've been using it as a coding agent since early access. It checks its own work and stops when the task is done. I would use it for long agent runs that I don't watch closely. StepFun built it for real engineering work. That covers fixing bugs in unfamiliar codebases, implementing features across files, and refactoring, with changes that run and pass checks. Frontend is the other big focus. You can give it a screenshot, a mockup, or a PRD, and it can return a working web page. It can also build a playable browser game from a set of assets. Beyond code, it reads annual reports and operating data and turns them into financial reports, editable spreadsheets, and slide decks. The context window is 1M tokens, with up to 1M output tokens. That is enough for large codebases and long multi-step tasks. Under the hood, Step 5 Preview is a sparse MoE with 600B total parameters and 27B active per token. Open weights are coming on Oct 15. Very much looking forward to its release.

This is one of the strongest use cases I've seen for System One models like Jev. Pay close attention if you build custom harnesses. Jev is now available as a packaged router. What is it? You send the conversation to typesafe/jev-router, and OpenRouter picks the model and reasoning effort for each request. Choosing the right model and reasoning effort is an art, and doing it manually is inefficient, to say the least. It makes sense to offload reasoning effort, but you want to do it efficiently, smartly, and cheaply. System One Models like Jev are a great fit for this. Does it work? I tested Jev Router on a small support agent built with a custom harness I built with the Pi SDK. I ran the same 8 cases against a fixed GPT-6 Sol baseline, 32 real calls in total. Both got every case right. The router cost less than half as much ($0.008 vs $0.018) and had a lower median response time (1.5s vs 1.9s). The sample is small, but it's clear that, at scale, this could make a huge difference in cost and efficiency. One of my biggest concerns about Jev-as-a-Router is caching. While it's not fully solved, Jev Router gives me hope for smarter, more efficient routing patterns in custom harnesses that leads to better tradeoffs. I have shared an implementation of a routing pattern I'm excited about here: The takeaway here is that Jev is unlocking interesting new patterns (routing, verification, guardrails, dynamic workflows, judges, and more) that you can tap into in your custom harness. I expect this trend to continue growing. And harnesses that are easier to customize, like Pi, will benefit even more.