Another insane Jev use case! Jev makes it incredibly cheap to evaluate and classify agent runs at scale. And finally, someone open-sourced a self-improving memory layer that can put that capability to work across agent harnesses. It turns your agent sessions into a compounding knowledge layer, where every successful run can make future agents smarter across: - Codex - Claude Code - Cursor - OpenCode and 20+ more Beacon by @asymptotelabs continuously builds a shared history across your agent harnesses and uses Jev to identify the runs worth learning from. It then turns the best workflows, corrections, and debugging patterns into reusable skills. GitHub repo: (don’t forget to star it ⭐) Most agent runs are messy. They contain exploration, failed commands, dead ends, and one-off fixes that should never become permanent memory. So Beacon preserves the full session history, while Jev helps decide what should be promoted, reviewed, or discarded. The recording below shows this in action. Beacon found 579 sessions across 5 coding-agent harnesses and normalized them into one consistent history. From there, Jev surfaces the lessons worth keeping and makes them available across your agent stack. - A pattern learned in Cursor can carry into OpenCode. - A lesson from Claude Code can improve the next Codex run. Every successful run adds to the shared knowledge layer, making future agents smarter. If you want to dive deeper into Jev, I also wrote a breakdown of how it works. The article is quoted below.

Redis built a cache that cuts LLM costs by 70%! Production LLM apps often receive different versions of the same question. For instance, an internal developer assistant might receive: - "How do I rotate an API key?" - "Where can I replace my API key?" The wording is different, but both questions have the same answer. Prefix caching cannot handle this repeated generation because it only reuses computation when the cache matches bit-by-bit. And if the cache hits, an LLM response is still generated again. A semantic cache stores the complete question-response pair outside the model. When a query arrives, it searches for previously answered questions with similar meaning. A valid match returns the stored response with no LLM call. If you want to use this in practice, @Redisinc already implements it as a managed service called Redis LangCache. Under the hood, LangCache embeds the incoming question, searches stored responses, and applies the configured similarity threshold and filters. A cache hit returns the earlier response. A miss falls back to the LLM, after which the new response can be stored for future requests. Redis also handles access scopes, custom filters, embedding selection, TTL, eviction, and monitoring through a REST API, without another database to deploy or manage. While the actual savings depend on how much safe repetition exists in the workload, cache-hit responses are up to 15x faster and 70% cheaper. Try Redis LangCache: I built a small interface comparing LangCache with direct LLM inference. The video below shows this in action, and I worked with Redis on this post to put it together. To dive deeper, my co-founder published a detailed article on KV, prefix, prompt, and semantic caching. Read it below.

The Jev sweet spot. (when to use it, and when not to) Jev lives between rules and LLMs. The easiest mistake is treating Jev as a faster replacement for an LLM. It is better to think of Jev as the missing layer between deterministic software and open-ended generation. The graphic below maps that boundary using two dimensions. One is how constrained the answer space is. The other is how frequently the decision must be made. ↳ Use ordinary code when the rule is exact. ↳ Use an LLM when the answer must be created through writing, planning, explanation, or deeper reasoning. ↳ Use Jev when the possible answers are known, but choosing among them requires semantic judgment. A decision belongs in that sweet spot when four conditions are true. → The valid answers can be defined in advance. Jev should choose from known labels, ordered scores, or a yes-or-no decision. → The input is difficult to express as deterministic rules. A person could judge it quickly, but an if-statement cannot reliably understand its meaning. → The decision happens frequently. Model routing, ticket triage, retrieval reranking, guardrails, and high-volume labelling repeat often enough for latency and cost to compound. → Uncertainty should change what happens next. High-confidence results can continue automatically. Ambiguous results can be sent to a stronger model or a person. Consider support-ticket routing. The possible owners are already known. They might be engineering, billing, or sales. But customer messages arrive as unstructured language rather than clean database fields. An LLM can classify them, but generating and parsing text for every ticket is unnecessary. Deterministic rules are cheaper, but become brittle as the wording changes. Jev handles the fuzzy judgment and returns probabilities that application code can use. The same framework explains where Jev does not fit. If the task requires writing or multi-step reasoning, use an LLM. If arithmetic or an exact rule can determine the answer, keep it in code. If the possible answers cannot be defined in advance, Jev is the wrong interface. The practical rule is simple. Use Jev when code understands the possible values but not the meaning of the input. I wrote the full breakdown on Jev. The article is quoted below.

Finally, an OpenRouter for agent harnesses! (including System One by Jev) Devs just open-sourced a plug-and-play infrastructure layer that lets you run any harness under a single interface, like: - Codex - Hermes - Claude code - DeepSeek Harness - System One, powered by Jev - And 9 more agent harnesses This means you can bring Jev into the same product that already uses Codex, Claude Code, or another supported harness, without writing another implementation for sessions, streaming, files, cancellation, and failure handling. Here's the repo: (don't forget to star it ⭐ ) The harnesses run locally, and the Unified Harness Protocol (UHP) defines the common task interface with an OpenAI Responses-compatible API. If you want to dive deeper, my recent article explains why model routing is not the same as harness routing, and what it takes to support multiple harnesses. It also covers UHP, the full local setup, a working API call, and how sessions and files work. Read it below.

LLMs vs. Jev, clearly explained! TL;DR The key difference is not that Jev generates faster. Jev does not generate text at all. A traditional LLM receives context and produces an answer one token at a time. Even when the output is a small JSON object, every token depends on those generated before it. Jev receives the same context but evaluates predefined decisions directly. When those decisions are independent, it can evaluate all of them in parallel. Consider an agent handling a failed deployment. It may need to determine: → Whether the incident is urgent → Which team should handle it → Whether the proposed command is risky → Whether the task is complete An LLM generates a response containing these answers sequentially. The application then parses and validates it. With Jev, you define the questions and expected answer types upfront. It evaluates them together and returns typed answers with probabilities. Jev supports three decision primitives: 1. **Choice** selects from known options, such as engineering, billing, or sales. 2. **Score** places the input on an ordered scale, such as low, medium, or high risk. 3. **Noul** evaluates a yes-or-no condition and returns the probability that it is true. The probabilities matter as much as the selected answers. If engineering receives 91% probability and billing receives 9%, automatic routing may be reasonable. If the probabilities are 52% and 48%, the system can escalate, gather more context, or call a stronger model. This keeps control inside ordinary software. Code owns the thresholds and consequences. Jev supplies the semantic judgment that a normal `if` statement cannot derive from unstructured text. It works best when the possible answers are known, the decision depends on meaning, and a careful person could judge the input quickly. It is not designed for writing, summarization, code generation, arithmetic, or decisions requiring several dependent reasoning steps. Independent questions can run in parallel, but decisions that depend on earlier results must remain sequential. Jev also cannot return an option outside the declared schema, but it can still select the wrong valid option. Type safety prevents malformed outputs, not incorrect judgments. The clean mental model is this: LLMs generate new language when the answer space is open. Jev evaluates known paths when the answer space is bounded. I wrote the full breakdown explaining Jev and where it fits. The article is quoted below.

MCP meets agent skills MCP already gave agents a standard way to connect to tools, resources, and external systems. Now it also defines a standard way to discover and load Agent Skills directly from MCP servers. The flow is simple: → connect to MCP server → discover available skills → inspect skill metadata → load the relevant 𝗦𝗞𝗜𝗟𝗟.𝗺𝗱 only when needed Under the hood, Skills are served through MCP’s existing Resources primitive. That means 𝗦𝗞𝗜𝗟𝗟.𝗺𝗱, references, scripts, examples, and other supporting files are exposed as resources that the client can read on demand. This is especially useful for context window management. Instead of loading every workflow instruction upfront, the agent can first discover what skills are available and pull in only the one required for the current task. A useful mental model is: tools = what the agent can do resources = what the agent can access skills = how the agent should perform a reusable workflow Previously, that workflow knowledge often lived separately in docs, repos, prompt files, or custom integrations. Now the MCP server can expose the capability and the playbook for using it together. So you get: → standardized skill discovery → on-demand context loading → cleaner distribution and versioning → reusable workflows that travel with the server MCP was already the connection layer. Skills now add a standardized way to ship reusable agent know-how on top of it. The illustration below visually summarizes everything that we discussed so far. Read more: Cheers! :)

where does all the VRAM go during LLM inference? (4 ways GPU memory is used) loading the model is only the first part of the memory story. once inference starts, GPU memory gets divided across multiple components, and some of them keep growing as context length, batch size, and concurrency increase. the graphic breaks it into four useful buckets: → model weights are the mostly fixed part. once the model is loaded, their memory footprint stays roughly constant. the biggest lever here is precision. moving from FP16/BF16 to INT8 or INT4 reduces the number of bytes needed to store each parameter. → KV cache grows as generation continues. for every previous token, the model stores key and value tensors so attention can reuse them instead of recomputing the entire sequence. longer contexts mean a larger KV cache, and more concurrent requests mean more active caches sitting in memory. → activations and workspace hold temporary intermediate values needed while running attention, MLP layers, kernels, and other computations. this memory is reused across inference steps, but its size can still change with sequence length, batch size, and the kernels being executed. → runtime overhead comes from everything around the model itself. CUDA kernels, memory allocators, metadata, serving-engine buffers, and other runtime structures all consume some VRAM. it is usually smaller than the other buckets, but it is never zero. this is why “the model fits on the GPU” and “the workload fits on the GPU” are two different statements. a model may load comfortably, then run out of memory when you increase the context window, serve more users simultaneously, or increase the batch size. it also explains why quantization can help beyond simply fitting a larger model. shrinking the weight footprint creates room that can instead be used for larger KV caches, more concurrent requests, or bigger batches. that is the broader GPU lesson too. performance is not just about how much arithmetic a GPU can do. it is also about what data occupies memory, how much of it moves during inference, and how often that data can be reused. i wrote the full breakdown of how GPUs actually work and why memory movement sits at the center of LLM inference performance. the article is quoted below.

Anthropic just got outplayed again. Devs built the multiplayer assistant Anthropic couldn't, and open-sourced it. Claude Cowork is a solo desktop agent. You point it at a folder, give it a task, and it works through your local files on your own machine. The moment a teammate enters the picture, it has nothing to offer. Most real work does not happen alone. A teammate asks for a status update on something you own. The context they need is scattered across your meetings, your notes, and decisions made last week. Typing all of that out takes time you do not have. This is the gap Claude Cowork was never designed to cross. Rowboat Spaces is built on a different model entirely. Each person brings their own assistant into a shared channel. Your assistant is your second brain. It knows your meetings, your notes, and your open decisions. That personal context stays yours. When a teammate asks a question in the channel, you ask your assistant to brief them. It pulls from everything you know and delivers the answer on your behalf, attributed to you. Your teammate's assistant does the same, from their own context. Teams can draft specs, track decisions, and update shared files from plain conversation. Each assistant reads the full channel history, cross references it against what exists, and flags what is missing. The whole thing is open-source, and each assistant acts as the person it belongs to, not as a shared bot pulling from a common pool. The video below shows this in action. I joined a shared space and asked my team member for a status update. My team member asked their second brain to answer. A spec got built from that conversation, versioned, with every change tracked back to the message that triggered it. Rowboat GitHub: (don't forget to star 🌟) My co-founder also wrote a great article on building your second brain with Rowboat, and I highly recommend reading it as well. The article is quoted below.

The easiest way to run your agent harnesses using local models: Whether you're using Claude Code, Codex, OpenCode, or Pi, the local setup itself isn't really the hard part. The harder problem is figuring out which local model your machine can actually handle well. You have to think about RAM, model size, quantization, context length, KV cache, speed, accuracy, and a bunch of other trade-offs before you even start. Magnitude is an open-source solution that removes all of that guesswork for you. It profiles your machine, benchmarks what it can realistically run, recommends the best models for your hardware, and then lets you connect them to your preferred agent harness. The entire setup takes just two commands. In this video, I walk through the full process from hardware profiling to running a harness on a local model. Chapters: 00:00 Intro 00:17 What Magnitude is 00:45 Setup in two commands 01:29 Why picking a local model gets confusing fast 02:11 Hardware profiling + model recommendations 02:59 Choosing your harness 03:22 Connecting a harness that's already running 04:18 Outro Get started: (don't forget to star 🌟)

The easiest way to run your agent harnesses using local models: Whether you're using Claude Code, Codex, OpenCode, or Pi, the local setup itself isn't really the hard part. The harder problem is figuring out which local model your machine can actually handle well. You have to think about RAM, model size, quantization, context length, KV cache, speed, accuracy, and a bunch of other trade-offs before you even start. Magnitude is an open-source solution that removes all of that guesswork for you. It profiles your machine, benchmarks what it can realistically run, recommends the best models for your hardware, and then lets you connect them to your preferred agent harness. The entire setup takes just two commands. In this video, I walk through the full process from hardware profiling to running a harness on a local model. Chapters: 00:00 Intro 00:17 What Magnitude is 00:45 Setup in two commands 01:29 Why picking a local model gets confusing fast 02:11 Hardware profiling + model recommendations 02:59 Choosing your harness 03:22 Connecting a harness that's already running 04:18 Outro Get started: (don't forget to star 🌟)

Get access to powerful open-weight models for FREE. DeepSeek V4 Flash, GLM 5.3 Flash, and Laguna S 2.1 are all available in Cline Desktop with no API key required. I picked DeepSeek V4 Pro, pointed it at an empty folder, and asked it to build an evaluation harness for an ML inference pipeline, one that runs test inputs, validates outputs, and logs pass/fail results with failure reasons. It checked the environment, identified available packages, and built a complete, fully tested `ml_eval` package across 9 files from a single prompt. The video above shows the full build, from an empty folder to a complete package. Cline Desktop is the open-source app for open-weight models with 68k GitHub stars, available for Mac and Windows. You can download it for Mac and Windows here: Thanks to Cline for working with me on this post!

13 attention mechanisms AI engineers must know: (bookmark this) The tricky part about attention is that these techniques are often discussed together, even though they solve very different problems. Some reduce KV cache size. Some control which tokens can attend to each other. Others make attention cheaper to compute or improve how KV cache is managed during serving. So, a better way to organize them is by the bottleneck they actually solve. Let's do that: 𝟭. 𝗞𝗩 𝗵𝗲𝗮𝗱 𝘀𝗵𝗮𝗿𝗶𝗻𝗴, 𝘄𝗵𝗲𝗻 𝗞𝗩 𝗰𝗮𝗰𝗵𝗲 𝘀𝗶𝘇𝗲 𝗶𝘀 𝘁𝗵𝗲 𝗯𝗼𝘁𝘁𝗹𝗲𝗻𝗲𝗰𝗸 → MHA (multi-head attention) gives every query head its own key and value heads, providing maximum flexibility but also the largest KV cache. → MQA (multi-query attention) makes all query heads share a single KV head, dramatically reducing cache size. → GQA (grouped query attention) groups query heads and gives each group a shared KV head, balancing memory savings with model quality. → MLA (multi-head latent attention) compresses keys and values into a smaller latent representation before caching them, reducing KV memory even further. 𝟮. 𝗔𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀, 𝘄𝗵𝗲𝗻 𝘄𝗵𝗮𝘁 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗰𝗮𝗻 𝘀𝗲𝗲 𝗺𝗮𝘁𝘁𝗲𝗿𝘀 → Causal attention only allows each token to look backward, which is what autoregressive LLMs use for generation. → Bidirectional attention lets tokens attend in both directions, which is useful when understanding the entire input at once. → Sliding Window Attention restricts each token to nearby context instead of attending across the full sequence. → StreamingLLM keeps a small set of anchor tokens plus recent context, allowing generation to continue with bounded memory. 𝟯. 𝗖𝗼𝗺𝗽𝘂𝘁𝗲 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆, 𝘄𝗵𝗲𝗻 𝗮𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗶𝘁𝘀𝗲𝗹𝗳 𝗶𝘀 𝗲𝘅𝗽𝗲𝗻𝘀𝗶𝘃𝗲 → FlashAttention computes exact attention in small tiles that fit in fast on-chip memory, reducing expensive memory movement. → Sparse Attention skips selected token-to-token connections entirely, reducing how much attention needs to be computed. 𝟰. 𝗞𝗩 𝘀𝗲𝗿𝘃𝗶𝗻𝗴 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆, 𝘄𝗵𝗲𝗻 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝘁𝗵𝗿𝗼𝘂𝗴𝗵𝗽𝘂𝘁 𝗶𝘀 𝘁𝗵𝗲 𝗯𝗼𝘁𝘁𝗹𝗲𝗻𝗲𝗰𝗸 → PagedAttention stores KV cache in blocks allocated on demand, reducing wasted and fragmented GPU memory. → RadixAttention organizes shared prefixes so KV states can be reused across requests instead of recomputed. → Prefix Caching similarly reuses KV blocks for repeated prefixes such as system prompts, reducing redundant prefill work. The important part is that these techniques are complementary. An LLM can use GQA to shrink its KV cache, FlashAttention to compute attention efficiently, Sliding Window Attention to limit context interactions, and PagedAttention to manage that cache efficiently in production. Once you organize them by the bottleneck they solve, the attention landscape becomes much easier to reason about. I wrote a deeper breakdown of how these techniques evolved and the problem each one solves. The full article is quoted below. Thanks for reading. Cheers! :)

RadixAttention, clearly explained. (how SGLang makes prefix caching highly efficient) Prefix caching sounds simple until requests start branching. Consider an AI coding assistant serving these three conversations: Request 1 Context: System prompt → Python repo → authentication code → “Fix this bug” Request 2 Context: System prompt → Python repo → authentication code → “Add OAuth” Request 3 Context: System prompt → Python repo → database code → “Optimize this query” Now there are three ways a serving engine can handle them. 1. With no prefix caching, every request starts from zero. The system prompt and repo context are processed again for every request, even though thousands of tokens may be identical. All that repeated prefill produces essentially the same KV state again and again. 2. Simple prefix caching improves this. The engine can cache a known shared prefix, such as the system prompt, and reuse its KV cache when another request starts with exactly the same tokens. That already saves work. But real workloads are not just one shared prefix followed by completely unrelated requests. They branch. Two requests may share the system prompt, repo context, and authentication code. Another may share only the system prompt and repo before diverging into database code. This is where RadixAttention becomes useful. 3. SGLang stores cached token sequences in a radix tree. The common system prompt becomes the trunk. The shared Python repo context extends that trunk. Authentication and database context become separate branches. Then individual user queries branch again. When a new request arrives, SGLang walks this tree and finds the longest prefix whose KV cache already exists. If 2,000 tokens match, it reuses 2,000 tokens. If 8,000 match, it reuses 8,000. Only the suffix after the point of divergence needs fresh prefill. That is the key difference. Simple prefix caching lets you reuse a prefix. RadixAttention turns all the overlapping prefixes across your workload into a reusable hierarchy. This is particularly valuable for multi-turn conversations, agent trajectories, shared system prompts, few-shot examples, and workloads where requests repeatedly branch from common histories. SGLang keeps these KV blocks in the radix tree and evicts them using an LRU policy when memory is needed. → PagedAttention makes KV cache allocation efficient. → RadixAttention makes overlapping KV caches reusable. I wrote the full breakdown of every attention mechanism, from Multi-Head Attention through FlashAttention and Sparse Attention, up to PagedAttention and RadixAttention. The article is quoted below. ---- PS: If you’ve already seen this post, I had to share it again because the visual posted last time was incorrect. This is the correct version.

11 LLM eval methods AI engineers must know: (bookmark this) The tricky part about LLM evaluation is that there is no single metric that tells you whether a system is good. The right evaluation method depends on what you are trying to measure. You may want to compare an output against a known answer, judge whether it is semantically correct, inspect how an agent behaved across multiple steps, or block unsafe outputs before they reach the user. A useful way to organize them is by what they are actually evaluating. 𝗥𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲-𝗯𝗮𝘀𝗲𝗱 𝗲𝘃𝗮𝗹𝘀, 𝘄𝗵𝗲𝗻 𝗴𝗿𝗼𝘂𝗻𝗱 𝘁𝗿𝘂𝘁𝗵 𝗲𝘅𝗶𝘀𝘁𝘀 → BLEU measures n-gram precision, checking how much of the generated text overlaps with the reference. → ROUGE focuses more on recall, measuring how much of the reference content appears in the output. → BERTScore compares contextual embeddings instead of exact words, so semantically similar answers can still score well. 𝗝𝘂𝗱𝗴𝗲-𝗯𝗮𝘀𝗲𝗱 𝗲𝘃𝗮𝗹𝘀, 𝘄𝗵𝗲𝗻 𝗴𝗿𝗼𝘂𝗻𝗱 𝘁𝗿𝘂𝘁𝗵 𝗶𝘀 𝘂𝗻𝗮𝘃𝗮𝗶𝗹𝗮𝗯𝗹𝗲 𝗼𝗿 𝗶𝗻𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲 → G-Eval uses an LLM to score an output against defined criteria. → LLM-as-Judge gives a model a rubric and asks it to score or compare outputs. → LLM Juries run multiple independent judges and aggregate their verdicts, reducing dependence on a single evaluator. 𝗛𝘂𝗺𝗮𝗻 𝗮𝗻𝗱 𝗱𝗲𝘁𝗲𝗿𝗺𝗶𝗻𝗶𝘀𝘁𝗶𝗰 𝗲𝘃𝗮𝗹𝘀 → Human Evaluation relies on people to score outputs across dimensions such as correctness, relevance, and helpfulness. → DAG-based Evaluation encodes evaluation as deterministic decision logic, routing outputs through explicit checks until a final verdict is reached. 𝗘𝘃𝗮𝗹𝘀 𝗯𝘂𝗶𝗹𝘁 𝗳𝗼𝗿 𝗮𝗴𝗲𝗻𝘁𝘀 → Trajectory Accuracy evaluates the sequence of actions an agent took, not just the final answer. → Multi-turn Evaluation scores behavior across an entire conversation, including consistency, memory, and coherence across turns. 𝗘𝘃𝗮𝗹𝘀 𝘁𝗵𝗮𝘁 𝗿𝘂𝗻 𝗮𝘀 𝗴𝗮𝘁𝗲𝘀 → Safety Evaluation checks for things such as toxicity, bias, sensitive-data leakage, and policy violations before an output is accepted. The important part is that these methods are complementary. A production agent may need reference-based evals for correctness, judge-based evals for subjective quality, trajectory evals for tool use, and safety evals before anything reaches the user. If you are building this evaluation layer, Comet’s Opik brings tracing, debugging, test suites, and agent evals into one open-source stack. Check it out on GitHub: (don't forget to star 🌟) I also wrote a deeper breakdown of how evaluation fits into a production observability and self-repair loop. The full article is quoted below.

Karpathy’s Agentic Engineering finally has proper DevTools! When an agent stops working, the model is only one possible cause. The problem could be a failed tool, a lost connection, an interface update that never appeared, or something earlier in the conversation. @CopilotKit has rebuilt its open-source Inspector around this problem. It sits inside the application and watches the full interaction between the user, the interface, and the agent. When something fails, the Inspector button turns red, names the failure, and opens the spot where it happened. But even a much harder problem is reproducing that failure. This is because agents do not always follow the same path twice. Running the same prompt again may trigger another tool, produce another response, or start with a different application state. Inspector handles this through saved Threads and an isolated Agent Playground. A developer can open the conversation where the problem occurred, choose "Try from here," and copy everything up to that point into the Playground. The original conversation remains unchanged while another response or tool path is tested. Threads can open inside the live application, while assistant messages can jump back to their matching context in Inspector. Everything runs underneath AG-UI, the protocol that carries messages, tool activity, state changes, and other events between the agent and the interface. Those interactions can also feed CopilotKit Intelligence so that when the same issue or useful behavior appears repeatedly, it can become an evidence-backed Insight and a proposed SKILL(.)md file. Developers can review, edit, and approve the improvement before the agent inherits it. The tool also helps developers find a problem, recreate its surrounding context, test another path, and turn repeated lessons into agent improvements. It is open source and included with CopilotKit development builds. Intelligence setup starts with one prompt. Here are the docs: I am testing this extensively and will cover this in more detail soon with a hands-on demo.

Chinese researchers did it again! OpenBMB just open-sourced MiniCPM5-2B, a dense 2B-parameter model built for reasoning, coding, and tool use on resource-constrained hardware. Artificial Analysis ranked it highest among models under 4B in its Agentic Index comparison. It scored 20, while Granite 4.2 8B scored 9. The model is particularly strong at coding and tool calling, so I tested both capabilities locally. I pulled it onto my machine, connected it to a constrained CI repair agent, and gave it one issue: > A customer reports that retrying checkout with the same idempotency key returns a larger total. The first request returns $109, while the retry returns $118. Find the root cause, fix it without changing the public API contract, and verify the complete test suite. The Python checkout service had 18 tests. Sixteen passed, while two failed on the retry path. The agent could list files, search code, read selected ranges, run approved tests, apply a patch, and inspect its diff. It reproduced the failure, then followed the checkout and idempotency paths through the repository. The model found that shipping was added to mutable order state before the cached result was checked. On retry, the same order already contained shipping, so the calculation added it again. It generated a narrow patch that moved the idempotency check ahead of the mutation without changing the public API. The agent ran the targeted tests and the complete suite. All 18 tests passed. The model was never told which file contained the issue or what change to make. Each test result, search result, and code inspection determined its next action. The video below shows the full trajectory, including the investigation, tool calls, generated patch, diff, and final verification. Everything ran 100% locally on my machine throughout the run. MiniCPM5-2B supports llama.cpp, Ollama, vLLM, SGLang, iOS, Android, and HarmonyOS for local deployment. The model weights, training recipes, reasoning datasets, and UltraX data-refinement system are open-source. GitHub Repo: A 2B model can now inspect a repository, reason across multiple files, modify code, and verify its patch while remaining small enough to target local hardware.

Finally, an open-source runtime security layer for your agent harnesses. AI agents can call tools, run shell commands, modify files, and interact with production systems. But when something goes wrong, teams often have to reconstruct what the agent actually did from scattered logs after the fact. Agent Beacon is built to change that. It runs locally and records agent activity as it happens, including: → Tool calls → Shell commands → File changes → Approval decisions → Session and harness context The useful part is that it normalizes all of this into the same event format across 23+ agent harnesses. So instead of writing different detection logic for Claude Code, Codex, or another harness, security teams can reason about the underlying action itself. A command executed is a command executed, regardless of which harness produced it. Beacon also records how confidently an event was captured. An action can be directly observed from the runtime or inferred from indirect evidence. That matters when you start writing security rules against this data. For example, you can detect when an agent reads an environment variable, modifies a sensitive file, executes a suspicious command, or performs an action without the expected approval. And the detection happens while the session is still unfolding, not days later during incident reconstruction. Everything runs locally by default. You can inspect sessions through the dashboard, write your own detection rules, or forward the same normalized events to tools like Splunk, Datadog, Elastic, Sentinel, or CrowdStrike. Beacon GitHub: (don't forget to star 🌟) I also wrote a detailed article on the same, covering how important runtime security is and how to build it for your agent harnesses. The article is quoted below.

The easiest way to find out which models you can run on your computer: Just run: - 𝗻𝗽𝗺 𝗶 -𝗴 @𝗺𝗮𝗴𝗻𝗶𝘁𝘂𝗱𝗲𝗱𝗲𝘃/𝗰𝗹𝗶 - 𝗺𝗮𝗴𝗻𝗶𝘁𝘂𝗱𝗲 𝘀𝗲𝘁𝘂𝗽 It profiles your machine and ranks the models across: - Speed - Accuracy - Intelligence - Memory required Finally, you can choose your favorite harness (Pi, OpenCode, Claude Code, Codex, etc.) to run with it. Get started here: (don't forget to star 🌟) I also wrote a detailed article on running your favorite harnesses with local models. The article is quoted below.