Stanford researchers did it again. They just built the agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. I wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. all of this is done using @Cometml's open-source project Opik. The article is quoted below.

NVIDIA just fixed text-to-SQL’s biggest problem! the release is called Auto Ontology, and it targets the reason an agent can write valid SQL against a company database and still return the wrong number. the database stores columns, and the agent has to guess what they mean. a column called extendedprice might hold revenue, and two systems in the same company can define revenue differently. Auto Ontology gives the agent an ontology, a schema that maps business concepts to the columns that hold them and defines how they relate, before any query is written. it drafts that schema from your database metadata instead of making you write it, and a human certifies it. the ontology holds three things: → terms: business concepts like customer, order, or revenue, with descriptions and synonyms, mapped to the exact columns that store them. → SQL attributes: reviewed expressions like a margin or a rolling total, reused so every measure is computed the same way each time. → relationships: how tables join, including bridge tables, inferred from keys and the data itself. to draft these, it profiles columns, samples values, extracts terms table by table, and mines past query history for SQL attributes. everything lands in Postgres, with embeddings in pgvector. at query time, the agent maps the question to certified terms, follows relationships to find the joins, and writes SQL. it checks filter values against the real data, repairs the query if a check fails, and returns the answer with its SQL and steps. when the data can't support a question, the agent says so. in NVIDIA's demo, a question about return on ad spend on a sales database with no ad data gets a clarifying reply offering revenue instead of an invented number. so the ontology tells the agent how to compute an answer and also which questions have one at all. as the architecture in the image shows, an ingestion service compiles the ontology from your source databases, and any agent harness reaches it through MCP, CLI, or REST API. it supports ten engines including Postgres, Snowflake, Databricks, and DuckDB, runs on Nemotron by default or any OpenAI-compatible endpoint, and is Apache 2.0. check it out on GitHub: the same principle of defining a schema before the model extracts anything also decides whether agent memory works. I wrote about that in the article quoted below.

Pages unlock the real power of Grok Bot Your bots can now publish their work as live pages instead of leaving it buried in chat. Here's how they work & 3 things I built: - A live HQ office for my 14-bot organisation - A custom analytics dashboard - Interactive learning guides

Pages unlock the real power of Grok Bot Your bots can now publish their work as live pages instead of leaving it buried in chat. Here's how they work & 3 things I built: - A live HQ office for my 14-bot organisation - A custom analytics dashboard - Interactive learning guides

As an AI engineer, please learn: - Learn the roofline model and why decode is memory-bound - Deploy vLLM and SGLang, then read their schedulers - Understand paged attention from the code, not the blog post - Build observability before you optimize anything - Track TTFT, inter-token latency, throughput, queue depth - Use Grafana + Prometheus for inference dashboards - Turn on prefix caching and find which workloads it helps - Learn continuous batching and chunked prefill - Run load tests with 1000+ concurrent requests - Report p50, p95, p99, never just the mean - Master quantization tradeoffs (FP8, INT4, AWQ, GPTQ) - Learn speculative decoding and where it stops helping - Set up KV cache eviction for long contexts - Try disaggregated prefill and decode serving - Learn Kubernetes for AI workloads and autoscale on queue depth - Learn how inference costs break unit economics - Build your own model router by cost, latency, quality - Create a token budgeting system per request - Build one inference service and benchmark it publicly - Read inference research instead of model release news - Start sharing your optimization benchmarks I put together a 10-week plan that covers every one of these at 30 minutes a day. It is 50 sessions, split between reading the theory and building on your own service, and all of them feed one artifact: an inference service you deploy, instrument, load test past 1000 concurrent requests, tune, and publish as a reproducible benchmark. It is open on GitHub and contributions are welcome, especially newer sources worth adding. I am working through it myself and will share more content on this going forward, so stay tuned. GitHub repo: (don't forget to star 🌟)

Temperature, top-k and top-p, clearly explained! (how an LLM picks its next token) An LLM never picks a word directly. At every generation step, it gives a raw score to every token in its vocabulary. It then turns those raw scores into probabilities and samples one token. Temperature, top-k and top-p shape that sampling step. None of them touches the model's weights. Here's how a typical sampler applies them: ↳ Temperature: it rescales the raw scores. Low values sharpen the distribution, so a few tokens dominate. High values flatten it, so unlikely tokens get a real chance. ↳ Top-k: it keeps only the k most likely tokens. With top_k at 50, it drops every token ranked below 50, whatever the distribution looks like. ↳ Top-p: it keeps the smallest set of tokens whose probabilities add up to at least p. With top_p at 0.9, an obvious next token leaves only a few tokens, and an open-ended one leaves many. ↳ Sample: the sampler picks one token from the tokens that remain. The key difference: top-k always keeps the same number of tokens, while top-p adapts to how confident the model is. Low temperature doesn't make a model more correct, though. It only reduces randomness, so a model without the right context can still be confidently wrong. Good starting points for temperature: - Classification and extraction: 0.0 to 0.2 - Code generation: 0.0 to 0.3 - General chat: 0.5 to 0.8 - Brainstorming: 0.7 to 1.2 Tune temperature first, then add top-p to trim the long tail of unlikely tokens. While you evaluate, change one setting at a time instead of all three at once. The diagram below shows the complete flow. - Temperature reshapes the probabilities. - Top-k keeps a fixed number of tokens. - Top-p keeps just enough tokens to reach a set share of probability. I wrote a full breakdown of LLM inference in the article below.

Grok Bot masterclass. In this video, I cover everything you need to understand and use Grok @Bot. Computer use, connectors, skills you teach with a screen recording, routines, and going from 1 to 14 bots that run like a company while I only talk to 4 of them. Enjoy! Chapters: 00:00 - Intro 01:22 - What a Grok Bot really is (a new hire with a laptop) 02:23 - Every bot gets its own computer 03:35 - Installation and pricing 05:01 - Creating your first bot (no system prompt to write) 06:29 - Routines: tasks your bot runs on a schedule 07:30 - Computer use and connected apps 11:50 - Skills: a recipe card for your AI 13:07 - Teach a Task: turning a screen recording into a skill 18:38 - One fat bot vs. 14 bots (and why both fail) 19:39 - Run it like a company: you, executives, workers 21:00 - The Notion task board that tracks every bot 21:31 - Designing the org with Claude 23:11 - Why my bots are named after TV characters 23:50 - Leadership in one group chat 26:31 - Chief of staff: inbox sweep across Gmail, Slack and X 27:10 - Head of content: post analysis and competitor watch 29:44 - Design team: on-brand illustrations and Typefully drafts 32:46 - Engineering team: GitHub issue to PR with one label 35:17 - Claude Code on your bot's computer 37:05 - Recap 38:12 - Outro Cheers! :)

Grok Bot masterclass. In this video, I cover everything you need to understand and use Grok @Bot. Computer use, connectors, skills you teach with a screen recording, routines, and going from 1 to 14 bots that run like a company while I only talk to 4 of them. Enjoy! Chapters: 00:00 - Intro 01:22 - What a Grok Bot really is (a new hire with a laptop) 02:23 - Every bot gets its own computer 03:35 - Installation and pricing 05:01 - Creating your first bot (no system prompt to write) 06:29 - Routines: tasks your bot runs on a schedule 07:30 - Computer use and connected apps 11:50 - Skills: a recipe card for your AI 13:07 - Teach a Task: turning a screen recording into a skill 18:38 - One fat bot vs. 14 bots (and why both fail) 19:39 - Run it like a company: you, executives, workers 21:00 - The Notion task board that tracks every bot 21:31 - Designing the org with Claude 23:11 - Why my bots are named after TV characters 23:50 - Leadership in one group chat 26:31 - Chief of staff: inbox sweep across Gmail, Slack and X 27:10 - Head of content: post analysis and competitor watch 29:44 - Design team: on-brand illustrations and Typefully drafts 32:46 - Engineering team: GitHub issue to PR with one label 35:17 - Claude Code on your bot's computer 37:05 - Recap 38:12 - Outro Cheers! :)

Harness Engineering 2.0 is here! After protocols for tools, agent communication, and agent interfaces, harnesses now have one too. Developers just built UHP (Unified Harness Protocol), a standardized interface for running agent tasks across any agent harness. HarnessRouter is its open-source server implementation. Here’s how it works: - The product backend sends a task with a `harness_id`. - HarnessRouter starts the selected harness in its own session. - The harness runs its native agent loop and streams back progress. - Files, artifacts, failures, and the final output return through the same interface. Without UHP, integrating every harness needs separate handling for task execution, sessions, streaming, files, recovery, and output formats. With UHP, Codex, Claude Code, Gemini CLI, Hermes, Pi, and other harnesses can plug into the same product backend while keeping their native tools, skills, models, and agent loops. This means developers can: - Switch harnesses without rebuilding the product integration. - Run the same task across different harnesses. - Add new harnesses without maintaining another execution stack. HarnessRouter currently supports 30 harness bases. Here’s the repo: (don’t forget to star it ⭐) As per official benchmarks, changing only the harness while keeping the model fixed reduced costs by 25% to 64% across several healthcare compliance tasks. And changing both the harness and model widened the difference to 99.8% between the highest and lowest-cost configurations. Developers can test their own workloads across different configurations using Harness Arena. Alibaba Cloud's Qoder has also adopted UHP, making it available as an agent backend through the HarnessRouter API. If you want to dive deeper, I already wrote an article that explains why model routing is different from harness routing and what supporting multiple agent runtimes actually involves. It also covers UHP, the local setup, a working API call, and how sessions and files work. Read it below.

The next step after Jev: A decision model like Jev bills and computes the full context on every call, so a 40-message chat checked after each message processes the early messages 40 times. Devs just built Sage2, a streaming decision model that computes each token of a conversation once. The setup is closer to a database cursor than an API call. A session holds the context on the server, and new messages are appended to it, not re-sent. The cached computation over the earlier context is reused. Each new message adds only its own tokens to the cost, and every registered question is re-evaluated against the full history. Each question returns a typed answer with a calibrated probability, so thresholds and routing stay in plain application code. On progressive context, this makes decisions 3x faster, with responses in tens of ms. Sage2 is built by @levantolabs, the team behind the first decision model, launched before Jev. The video below depicts the model in action on a car dealership chat, and I partnered with the team to put this together. More details below.

I gave my Grok @bot team a shared Notion task board. Now every bot logs each task it picks up, so I can see everything happening across my org and steer it from one board. Grok Bot masterclass video dropping this wed. Stay tuned!

Another banger from DeepSeek. dsh-superagent is an open-source proactive coding agent that coordinates multiple agents to tackle GitHub tasks in parallel, handle review feedback, and report back in your team chat without needing you to babysit every step.

RAG vs. retriever-free RAG, clearly explained! (powered by decision models like Jev) In a standard RAG setup, documents are split into chunks, converted into embeddings, and stored in a vector index. When a query arrives, the system embeds it and pulls the top-k chunks with the closest vectors. A reranker then reorders those candidates against the query, and the best few go to the LLM as context. The problem is that the similarity search decides what the LLM can ever see. If the right passage never made the top-k, nothing later in the pipeline can bring it back, not even a strong reranker. Retriever-free RAG, from a new Cornell and ASAPP paper, keeps the documents and the LLM but replaces similarity search with navigation. The documents are organized as a document tree, like a table of contents, and a decision model (Jev 1.13) walks down it. A decision model doesn't generate text. It reads one shared context and returns a probability for every yes/no question about it, in one forward pass per stage. Here's how a query moves through the tree: ↳ Top documents: the model reads the query beside every candidate document, asks "Is this relevant?" for each, and keeps the best few. ↳ Top passages: it opens only those documents and asks the same question for every passage inside them. ↳ Most relevant? (optional): a single multiple-choice question reorders the top 20 passages (top 5 on Natural Questions). Dropped documents are never opened, so their passages are never scored. On Natural Questions (2.68M Wikipedia paragraphs), its top pick contained the answer 65.4% of the time. Embedding search plus an 8B reranker got 48.0%, and GPT-5 reranking got 60.8%. The gain comes from ranking, not coverage. Embedding search's top 100 holds the answer more often than the navigated shortlist (98.4% vs 94.0%), but navigation is better at putting it first. Median retrieval takes under 2 seconds per query, but each method was timed in a different setup, so it isn't a controlled speed comparison. The main limit is that it needs a sensible hierarchy, and a document pruned early is gone for good. It also skips embedding search only at query time, since the Wikipedia tree was built offline using embeddings. The diagram below depicts the complete flow. - RAG retrieves by similarity, then reranks. - Retriever-free RAG navigates a document tree with a decision model. - The LLM answers from the top passages. If you're building a RAG system, this is worth benchmarking against your current setup. Here's the paper: If you're new to decision models, I've quoted my full breakdown of how Jev works below.

Big moment for Voice AI. A typical voice-agent pipeline has three model stages. Speech-to-text converts the caller’s audio into text. The LLM processes that text and generates a response. Text-to-speech converts the response back into audio. For speech accuracy, information can be lost at two boundaries. - STT must preserve what the caller said on the way in. - TTS must preserve what the LLM generated on the way out. If a caller says “RX-B17,” the transcription model could hear “RX-D17” before the LLM receives it. Even if the LLM generates “RX-B17” correctly, the speech model could still make the final audio sound like “RX-D17.” The LLM’s reasoning accuracy is a separate problem. Both speech layers must preserve the information entering and leaving it. This becomes important with medication names, dosages, appointment times, tracking numbers, and alphanumeric codes. Deepgram built Flux TTS for the speech layer. In an evaluation containing drug names, tracking numbers, dollar amounts, technical strings, and alphanumerics, Flux recorded a 3.4% word error rate. I tested the complete voice loop through a pet-hospital conversation about my dog Biscuit’s prescription refill. The agent had to understand and repeat the medication, dosage, prescription number, and a collection time that changed during the call. The test worked well. The agent retained the changing details, the voice stayed natural throughout, and the exchange felt like a call I would be comfortable having with an actual pet hospital. Build with Flux TTS: For every $1 spent on Flux TTS, Deepgram will match $1 in credits, up to $500. My recording is below, and I worked with the Deepgram team to put this post together.

Big win for local LLM inference! This open-source engine generates tokens 2.7x faster than Ollama on a MacBook: I tested it on a base M5 MacBook Pro with Qwen3.5 4B at 4-bit and the same 9.2k-token prompt. Ollama ran its MTP build, so both had speculative decoding. - Prefill: Ollama 464 tok/s, Magnitude 755 tok/s (1.6x) - Decode: Ollama 22.6 tok/s, Magnitude 60.0 tok/s (2.7x) - End to end, Magnitude finished in half the time The video below shows both runs side by side. The Ollama numbers don’t mean it’s badly built. Its kernels were simply tuned in advance. A kernel is the small GPU program behind every matrix multiply and attention step. Its speed depends on choices like how work is split across GPU cores and how data moves through memory. The best choices change with every chip. Apple Silicon alone spans 8 to 80 GPU cores and under 70 to over 800 GB/s of bandwidth, and every model has its own layer sizes. A prebuilt engine has to pick one set of settings for all of them, so most local setups run on a compromise. Magnitude replaces that compromise with tuning on your machine: ↳ When you download a model, it compiles that model’s kernels locally. ↳ It tries different kernel configurations and keeps whichever runs fastest on your chip. ↳ It pairs each model with the fastest speculative decoding drafter it has found, so each pass of the model produces more than one token. GitHub repo: (don’t forget to star it 🌟) This matters most for agents. Every file and tool output has to be processed before the model writes anything, and that prefill step is where kernel efficiency shows up. Concurrent sessions also share prefix caches, so parallel agents don’t slow each other down. Magnitude is an open-source (Apache 2.0) inference engine written in Rust. It plugs into Claude Code, Codex, OpenCode, Pi and other agents with one click. I shared the GitHub repo above.

The easiest way to find which OCR solution works best for you: I ran DeepSeek-OCR, PaddleOCR-VL and Datalab Surya on the same 150 pages, from handwritten notes and rotated scans to dense tables and a filled tax form. Why? Because one overall score only tells you who's ahead on average, not where a parser breaks on documents like yours. So every page was checked with simple yes/no tests: is this line present, is this cell under the right headings, does A come before B. Here's what I found: - No single parser won on every type of page - DeepSeek-OCR was the best on tables at 87.2 - On handwritten notes, one parser passed 5/6 tests while the other two passed 0 - On a filled 1040 tax form, PaddleOCR-VL returned only the form's title The video below shows every page, what each parser returned, and why each test passed or failed. So the right pick depends on your documents, not on who has the best average. The tool I used is OmniParse Bench, and it's 100% open-source. It has 16,288 tests across 2,937 pages in 92 languages, each tagged by what makes a page hard, like handwriting, rotation, tiny text, forms and tables. Scoring is done by open code with no LLM judge. So you can plug in any OCR solution and see exactly where it holds up and where it breaks. GitHub repo: (don't forget to star 🌟)

The easiest way to find which OCR solution works best for you: I ran DeepSeek-OCR, PaddleOCR-VL and Datalab Surya on the same 150 pages, from handwritten notes and rotated scans to dense tables and a filled tax form. Why? Because one overall score only tells you who's ahead on average, not where a parser breaks on documents like yours. So every page was checked with simple yes/no tests: is this line present, is this cell under the right headings, does A come before B. Here's what I found: - No single parser won on every type of page - DeepSeek-OCR was the best on tables at 87.2 - On handwritten notes, one parser passed 5/6 tests while the other two passed 0 - On a filled 1040 tax form, PaddleOCR-VL returned only the form's title The video below shows every page, what each parser returned, and why each test passed or failed. So the right pick depends on your documents, not on who has the best average. The tool I used is OmniParse Bench, and it's 100% open-source. It has 16,288 tests across 2,937 pages in 92 languages, each tagged by what makes a page hard, like handwriting, rotation, tiny text, forms and tables. Scoring is done by open code with no LLM judge. So you can plug in any OCR solution and see exactly where it holds up and where it breaks. GitHub repo: (don't forget to star 🌟)

I gave Claude Code, Codex, and Cursor a self-improving, shared MEMORY. Now my coding agents learn from each other, and their knowledge follows me wherever I work. Coding agents work in silos. Your corrections in Cursor don't carry over to Claude Code, and skills built in Codex don't automatically transfer to other harnesses. So switching tools often means starting from scratch. What if every coding agent could learn from everything you've already done? I invited @jqdsouza, co-founder of Asymptote Labs, to show me how to make this possible with Beacon, a 100% open-source memory layer for coding agents. We set it up live, explored real coding sessions, and watched Beacon turn past work into reusable knowledge. Here's what I learned: 1. One memory layer for 20+ coding agents. Beacon captures activity across Claude Code, Cursor, Codex, and 20+ harnesses using OpenTelemetry, hooks, and local session files. Everything gets normalized into a single format, regardless of which agent you're using. 2. Your entire coding history becomes searchable. Every prompt, tool call, command, file change, and response is captured in a structured trace. Instead of losing valuable context when a session ends, you build a searchable history of everything your agents have done. 3. Switch agents without starting over. Hit your usage limit in Cursor? Open Claude Code and ask it to continue your latest Cursor session. Beacon retrieves the relevant context in milliseconds so your next agent can pick up where the last one left off. 4. Your agents learn from your past work. This is where things get interesting. Beacon identifies repeated workflows and turns them into reusable skills. In our demo, it discovered a recurring UI development workflow and generated a skill containing the exact dev server commands, screenshot steps, and deployment instructions. You teach one agent something once. Every other agent can benefit. 5. Stop accumulating skills nobody uses. Have 100 skills but only use five? Beacon analyzes actual usage across thousands of sessions and identifies skills worth keeping, combining, or removing. Your agent's memory gets more useful, not just bigger. 6. Security rules that follow you across every harness. Beacon includes hundreds of detection rules that work across agents. It can identify risky behavior, including sequences where an agent reads sensitive credentials and then makes an outbound network request. 7. It even caught a real credential leak. While reviewing Justin's actual Cursor history, Beacon surfaced an incident where part of a production database password had been sent to a model provider. That's the power of having visibility into what your agents are actually doing. 8. One prompt to get started. Completely open source. Paste a setup prompt into your coding agent and you're off. Keep everything local on your machine, or use Beacon Cloud for free. If you're juggling multiple coding agents, you need to check this out. Drop a like and share it with someone facing the same issue. Beacon on GitHub: (don't forget to star 🌟) Enjoy the episode. Cheers!

Anatomy of a Jev request! Wiring an LLM into code usually means asking for JSON, then handling every case where the output drifts from the schema. Jev, TypeSafe's decision model, doesn't write text at all. Every answer is one of the options you defined, with a probability attached, so the result goes straight into an if statement. A Jev request has two parts. The state is the thing being judged. It can be a customer message, a database record, or now a photo. The questions are the judgments you want back. You can send several in one request, and they all look at the same state and run in parallel. Each question has an instruction, which is the question written in plain words. Some questions also carry criteria, the set of answers Jev is allowed to return. There are four question types. 1) Choice picks one option from a list you define, like routing a ticket to billing, technical or sales. You get the option name plus a probability for every option. 2) Noul is a yes-or-no question. Instead of a hard yes or no, it returns the probability that the answer is yes, and your code decides the threshold. 3) Score places the state on an ordered scale you define, like how frustrated a customer sounds, from calm to very angry. The answer can land between two levels. 4) Box is new in the latest release. You ask where something is in an image, and it returns the four edges of a rectangle around it, as fractions of the image's width and height. Box always returns a rectangle, even when the thing isn't in the image. That's why it's usually paired with a Noul asking whether the object is there at all. Images come in through a new assets field. Each image gets a reference string, and wherever that string appears in the state, an instruction or the criteria, Jev sees the image instead. The full walkthrough, with the live demo and the code, is in my video quoted below on multimodal Jev. You can also watch it on YouTube: