TL;DR Reliable agentic AI comes not from "a better model or prompt" but from explicitly engineering context and the orchestration harness—lessons from PRINCE, a Bayer × Thoughtworks pharma system. 🧪 Title: Building Reliable Agentic AI Systems URL: Highlights 🧭 Sequential agents with pause points: clarify intent → Think & Plan → Researcher → Reflection → Writer, verifying step by step 🔁 Three reflection loops: process (trajectory), data (evidence sufficiency), draft (output completeness) catch distinct failures 🔎 Hybrid retrieval: query expansion n=5, weighting 0.7 semantic + 0.3 keyword, bge-reranker cutting ~20 → 7 chunks 🗃️ Structured data via Text-to-SQL: SELECT-only, up to 3 self-corrections, ≤50 rows per query 🛟 Harness engineering: state persisted in PostgreSQL/DynamoDB, resume from failure point, automatic provider fallback 📌 Sentence-level citations plus RAGAS and Langfuse evals; daily batch checks on live traffic catch hallucinations 🏷️ NER extracts entities from study PDFs; high-confidence fields auto-update, low-confidence quarantined for human review The "even with big context windows, selectivity still matters" stance rings true to anyone shipping this stuff. #AIAgents# #LLMOps#

Coding agents fire dozens of API calls per task, so a single developer can quietly burn thousands of dollars a week before anyone notices — here's how LangChain killed that "spend unpredictability" internally 💸 The key was folding budget control into the same place as observability. Title: How LangChain Made Coding Agent Spend Predictable URL: 💸 Overview An LLM Gateway built into LangSmith gives a company-wide, minute-level view of model spend and manages budgets centrally. Rather than a bolt-on proxy, it sits on the same foundation as existing tracing, evaluation, and user management. ❓ Challenges Solved Model usage spread from a few teams to the whole company, and premium model prices rose, so costs spiked. ・Coding agents trigger dozens of API calls per task ・Individual developers ran up thousands of dollars a week, unnoticed until month-end 💡 Methodology & Proposed Approach Budgets can be set across multiple layers. ・Caps at the organization, workspace, user, and API-key level ・Default monthly, weekly, daily, and hourly windows for all employees, with exceptions for heavy projects ・Covers agents accessed via Claude Code, Codex, and LangChain Deep Agents ・Deployed via MDM so no one has to set it up manually ・Runs are traced and tied to a user and API key; overspend can be diagnosed by inspecting the trace with evaluation data 🌍 Use Cases Engineering leaders can set team-level limits while still letting people use agents without fear of a surprise bill. The practical value is replacing the month-end billing shock with real-time monitoring. 📊 Lessons & Outcomes ・Static price tables go stale fast, so pricing must be handled dynamically, including caching and tier differences ・Cursor and Claude Desktop didn't route cleanly, so they measured the delta between Gateway-captured traffic and provider settings to correct for it ・Hard limits alone block real work, so they evolved into early-warning alerts and auditable budget-increase requests ・Since internal rollout, LLM costs have stayed within budget #CodingAgents# #LLMOps#

Logging alone won't prevent incidents. Meet runtime governance that blocks problematic requests before they reach the LLM 🛡️ Title: LangSmith LLM Gateway: runtime governance built into the agent lifecycle URL: 🛡️ Overview A runtime governance layer that sits between agents and their model providers. As an enforcement point inside the LangSmith platform, it aims to stop problems at the source rather than just logging them after the fact. ❓ Challenges Solved Observability (logging) alone can't prevent problems. Logging an incident after it happens is too late — problematic requests should be blocked before they reach external LLM providers. 💡 How It Works ・Spend controls: hard caps at org, workspace, user, or API-key level; returns 402 when exceeded ・Cost visibility: real-time spend tracking across org units ・Data protection: auto-redaction of PII and secrets before they reach the model ・Trace integration: gateway-proxied calls appear in the same workspace ・Audit logging and layered policies Setup is minimal: point base_url at the Gateway, store provider keys in workspace secrets, and define policies in the LangSmith UI. 🌍 Use Cases ・Preventing runaway agent spend from retry loops ・Stopping sensitive data (SSNs, PII) from leaking into provider logs ・Establishing org-wide cost governance and compliance auditing #LLMOps# #AIGovernance#

One of the new, buzzy jobs in Silicon Valley is the AI Forward Deployed Engineer (FDE), an engineer who is embedded within a client organization to help customize solutions, such as building and tuning agentic workflows that suit the client’s particular needs. I’ve heard from people who are wondering anew about the FDE career path since OpenAI and Anthropic started building new teams to place FDEs within client organizations. The rise of FDEs for AI workloads is one way AI is creating new jobs (and why the jobpolcalypse narrative of upcoming job market collapse is false -- there will be many AI and non-AI jobs). However, I believe there will be far more AI Engineer jobs than FDEs, as I explain below. The FDE role was pioneered about two decades ago by Palantir, which sent engineers to government locations to work on secure, air-gapped networks. In addition to having good technical skills, FDEs need communication skills and sometimes business skills. For example, they may need to speak with clients to understand their needs, formulate a strategy to prioritize projects, explain complex technology, and respectfully push back if a client asks for something unrealistic. They’re enjoying a resurgence because of the amount of work involved in taking an off-the-shelf LLM and building it into a custom agentic workflow that fits particular business needs. However, I believe the number of AI Engineer jobs will be far larger. A company might accept a few FDEs to be embedded within its organization. But most companies will want far more of their own employees working on their projects. While my organizations do hire FDEs, we hire far more AI Engineers! Also, a common client concern is that it is hard to find vendor-neutral FDEs — they are, after all, there to deeply integrate a particular vendor’s product into a company. In this moment when it’s hard to predict which AI service will be the best one in a year’s time, optionality (the ability to pick whatever vendor turns out to fit best in the future) is very valuable. In contrast, letting FDEs tightly bind a company’s processes significantly reduces optionality. Right now, I see surging demand for AI Engineers who can build software applications using AI software components (like LLM prompting, agentic frameworks, evals, etc.) and effectively use AI coding agents (like Claude Code, Codex, Antigravity CLI, and OpenCode). As the AI Engineer role matures, I expect it to fragment into more specialized roles, like the generic Software Engineer role from decades ago fragmented into frontend, backend, mobile, data engineering, devops, and so on. What will be the future, specialized AI engineering roles? I don’t know. Perhaps there will be AI FDEs, LLMOps Engineers, Evals Engineers, AI Data Engineers, Harness Engineers, and other roles we don’t have names for yet. But for now, I see a lot of AI engineers who are generalists create a lot of value. Skilled AI Engineers are in very high demand! As our field continues to mature over the coming decade, I look forward to new specializations within AI Engineering that create even more job opportunities. [Original text: The Batch newsletter]