🔬 Can AI actually "discover" science? Tested on 90 real Nature papers, even the strongest agent beat SOTA on under 20% of tasks. Title: NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers? URL: 📋 Overview Using 90 tasks drawn from Nature-family journals across six domains, this work tests whether AI coding agents can reproduce and surpass published SOTA. An "information firewall" hides the original method, and NatureGym builds per-task containerized environments for strict, unified evaluation with web search disabled. 🎯 The problem it solves Prior agent-on-research benchmarks were fragmented and low in credibility. This work forces genuine discovery instead of reproduction and creates a common ground for cross-domain comparison. 📐 Methodology Tasks are scored with a SOTA-normalized relative gap (g > 0.1 to surpass, g >= 0 to match), comparing across 81 metrics. Ten agents are evaluated across three harnesses under a 4-hour budget with task-based GPU allocation. 📊 Results ・Even the top agent, Claude Opus 4.7, surpassed SOTA on only 17.8% of tasks and matched on 47.8% ・45.5% of successes were just translating a science task into supervised prediction; genuine domain reasoning was only 8.3% ・Failures were driven by wrong method choice (45.1%) and insufficient compute (24.4%), with task misunderstanding at only 3.1% ・Interdisciplinary tasks consistently scored worse A sober snapshot: today's agents are good at "translation" but weak at real "discovery." #AI4Science# #CodingAgents#

Coding agents fire dozens of API calls per task, so a single developer can quietly burn thousands of dollars a week before anyone notices — here's how LangChain killed that "spend unpredictability" internally 💸 The key was folding budget control into the same place as observability. Title: How LangChain Made Coding Agent Spend Predictable URL: 💸 Overview An LLM Gateway built into LangSmith gives a company-wide, minute-level view of model spend and manages budgets centrally. Rather than a bolt-on proxy, it sits on the same foundation as existing tracing, evaluation, and user management. ❓ Challenges Solved Model usage spread from a few teams to the whole company, and premium model prices rose, so costs spiked. ・Coding agents trigger dozens of API calls per task ・Individual developers ran up thousands of dollars a week, unnoticed until month-end 💡 Methodology & Proposed Approach Budgets can be set across multiple layers. ・Caps at the organization, workspace, user, and API-key level ・Default monthly, weekly, daily, and hourly windows for all employees, with exceptions for heavy projects ・Covers agents accessed via Claude Code, Codex, and LangChain Deep Agents ・Deployed via MDM so no one has to set it up manually ・Runs are traced and tied to a user and API key; overspend can be diagnosed by inspecting the trace with evaluation data 🌍 Use Cases Engineering leaders can set team-level limits while still letting people use agents without fear of a surprise bill. The practical value is replacing the month-end billing shock with real-time monitoring. 📊 Lessons & Outcomes ・Static price tables go stale fast, so pricing must be handled dynamically, including caching and tier differences ・Cursor and Claude Desktop didn't route cleanly, so they measured the delta between Gateway-captured traffic and provider settings to correct for it ・Hard limits alone block real work, so they evolved into early-warning alerts and auditable budget-increase requests ・Since internal rollout, LLM costs have stayed within budget #CodingAgents# #LLMOps#

Coding agents don't just autocomplete anymore. They open PRs, run tests, ship features - on their own. Most of what they ship still lands on infra built for humans typing slowly. Canopy was built for the agent that doesn't sleep 🌿

Coding agents spend most of their time executing, editing files, running tests, retrying failures, and iterating. That’s where speed matters 🚀 Our new Responses API support makes it easier to connect coding agents to fast, production-ready models on SambaCloud. Read more:

Coding agents are accelerating different types of software work to different degrees. When we architect teams, understanding these distinctions helps us to have realistic expectations. Listing functions from most accelerated to least, my order is: frontend development, backend, infrastructure, and research. Frontend development — say, building a web page to serve descriptions of products for an ecommerce site — is dramatically sped up because coding agents are fluent in popular frontend languages like TypeScript and JavaScript and frameworks like React and Angular. Additionally, by examining what they have built by operating a web browser, coding agents are now very good at closing the loop and iterating on their own implementations. Granted, LLMs today are still weak at visual design, but given a design (or if a polished design isn’t important), the implementation is fast! Backend development — say, building APIs to respond to queries requesting product data — is harder. It takes more work by human developers to steer modern models to think through corner cases that might lead to subtle bugs or security flaws. Further, a backend bug can lead to non-intuitive downstream effects like a corrupted database that occasionally returns incorrect results, which can be harder to debug than a typical frontend bug. Finally, although database migrations can be easier with coding agents, they’re still hard and need to be handled carefully to prevent data loss. While backend development is much faster with coding agents, they accelerate it less, and skilled developers still design and implement far better backends than inexperienced ones who use coding agents. Infrastructure. Agents are even less effective in tasks like scaling an ecommerce site to 10K active uses while maintaining 99.99% reliability. LLMs' knowledge is still relatively limited with respect to infrastructure and the complex tradeoffs good engineers must make, so I rarely trust them for critical infra decisions. Building good infrastructure often requires a period of testing and experimentation, and coding agents can help with that, but ultimately that’s a significant bottleneck where fast AI coding does not help much. Lastly, finding infrastructure bugs — say, a subtle network misconfiguration — can be incredibly difficult and requires deep engineering expertise. Thus, I’ve found that coding agents accelerate critical infrastructure even less than backend development. Research. Coding agents accelerate research work even less. Research involves thinking through new ideas, formulating hypotheses, running experiments, interpreting them to potentially modify the hypotheses, and iterating until we reach conclusions. Coding agents can speed up the pace at which we can write research code. (I also use coding agents to help me orchestrate and keep track of experiments, which makes it easier for a single researcher to manage more experiments.) But there is a lot of work in research other than coding, and today’s agents help with research only marginally. Categorizing software work into frontend, backend, infra, and research is an extreme simplification, but having a simple mental model for how much different tasks have sped up has been useful for how I organize software teams. For example, I now ask front-end teams to implement products dramatically faster than a year ago, but my expectations for research teams have not shifted nearly as much. I am fascinated by how to organize software teams to use coding agents to achieve speed, and will keep sharing my findings in future posts. [Original text: ]

AI coding agents might create an unexpected problem: More code. Less shared knowledge. If an engineer solves a problem through an AI agent, a lot of the reasoning can stay inside a private human-agent loop. No Stack Overflow answer. No GitHub discussion. No detailed issue thread. Sometimes not even a meaningful commit message. We may become dramatically better at producing software while becoming worse at producing the public knowledge that future engineers learn from. That tradeoff deserves much more attention.

AI coding agents gain trust as developers increasingly deploy AI-generated code without human review, according to Cursor. This shift marks growing confidence in AI reliability.

AI coding agents can read code. ABP AI Agent can also work with runtime context from ABP Studio, including exceptions, logs, requests, containers, tasks, and build validation. Learn how integrated tools help the agent move from guessing to debugging with real evidence 👇 #dotnet# #AI# #AIAgent# #abpframework#

AI coding agents are powerful… but chaotic. Archon + Agent Skills turn them into deterministic PR machines. Parallel agents. Zero merge conflicts. Running locally on my M4 Pro.