⚙️ TL;DR: without touching the model, automatically improving the software that runs the agent cuts token cost by roughly half. Title: SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness URL: 📌 Highlights 🔍 Searched ~150 directions across ~500 environments, running 3,000+ trials to discover efficiency mechanisms 🧩 4 mechanisms found, including Action Fusion, which merges a file edit and its test run into one request 📉 On EdgeBench: 49.0% less token traffic, 33.2% lower cost, at 93.7% of baseline performance 🔄 Applied to Opus 5 with zero extra tuning, still keeps 44.7% token and 33.5% cost reduction 💰 On Terminal-Bench 4, cost per solved task drops 11.6% ⏱ Estimated $8.75-13.50/hour savings versus native Codex The interesting part: optimizing the harness instead of the model turns out to be a genuinely production-relevant efficiency lever. #AIAgents# #LLMCostOptimization#

💰 Same model, same task, but swap the software wrapper (harness) around it and the cost can jump 5x while success rate barely moves. Title: HarnessTax: How Much Does the Harness Matter for Coding Agents? URL: UC Berkeley and Arena compared three coding agent harnesses, Claude Code, Codex CLI, and Pi, across 7 models and 21 model-harness pairs. Here are 3 highlights. 📊 Statistically, harness barely moves success rate Out of 42 within-model harness comparisons, only 1 was statistically significant (chance alone predicts about 2), and none survived correction for multiple comparisons. 💸 Cost can differ 5x at the same success rate On GPT-5.6 Luna, Claude Code cost $0.15/task at 55.6% success, while Pi cost $0.03/task at 53.3% success, nearly identical accuracy, 5x the price. 🪶 The minimal open-source harness holds its own In 9 of 12 model comparisons, a non-vendor harness (Pi or Codex CLI) achieved the top success rate, showing a stripped-down harness can genuinely compete. The practical takeaway: prioritize cost and reliability over flashy add-on features first. #CodingAgents# #LLMCostOptimization#

Cut your AI API bill by up to 60% — with just one line of code changed. Title: Cheaper Inference - Save up to 60% on AI models URL: Cheaper Inference is an API gateway that aggregates discounted pricing from OpenAI, Anthropic, Google, xAI, AWS Bedrock, and more through a single OpenAI-compatible endpoint. No code changes beyond swapping the base URL and API key. Key Points 💰 Up to 60% savings with price-cap guarantees Live market rates are aggregated across multiple providers, with a hard cap ensuring you never pay more than direct provider pricing. No monthly commitments — start with just $5. Essentially zero-risk to try. 🔌 Zero code changes — just swap the base URL Full OpenAI SDK compatibility means only the base URL changes to ` Supports text and image generation, vision input (up to 10 images, 5MB each), reasoning models with configurable effort, prompt caching, and streaming. 🔒 Enterprise-grade security controls per API key Model allowlists, IP address filtering, rate limits, daily quotas, and monthly budget caps — all configurable per API key. Zero data retention option and audit logging via the History feature make it viable even for compliance-sensitive environments. The practical appeal is clear: cost savings without touching your existing architecture. As AI API costs increasingly factor into product economics, having a procurement layer like this in your stack is worth considering. #LLMCost# #APIGateway#

Do all your agent's LLM calls really need a frontier model? NVIDIA's Switchyard ran the numbers — and the results are surprising. Title: Switchyard Agent Routing Benchmark URL: TL;DR Evaluated 145 multi-step agentic tasks (averaging 6.3 LLM calls each). 93% of calls were handled by a smaller model, achieving 74% cost reduction with only a 6-point accuracy drop. Key Points 🎯 Frontier model needed for just 7% of calls Out of all LLM calls, Claude Opus 4.8 was required for only 7%. The remaining 93% were handled by Nemotron 3.5 Lightning. 💰 74% cost reduction Per-task cost dropped from $0.092 (Opus alone) → $0.026 (routed) → $0.006 (Lightning alone), while maintaining 80% accuracy. 📊 The surprising cost breakdown Despite handling only 7% of calls, the frontier model consumed 68.4% of total spend. The judge model itself added another 21.2% of routed costs. 🔢 The formula for routing ROI "Minimum offload rate = judge cost / (expensive model cost - cheap model cost)" — if the price gap is small, routing may not pay off. ⚡ Two deployment options Run NVIDIA Switchyard as a standalone proxy server, or embed it as middleware inside LangChain's Deep Agents framework. Note: the eval suite had relatively easy tasks, so the benefit of routing could be even larger with harder workloads. Still, highly practical for agents that issue many calls per task. #AIAgents# #LLMCost#

The internet is now cooked soon? With $RDDT to other social media like X more at risk due to AI. You can have new decision models like Jev making autonomous computer interaction at near $0 cost. (With actions becoming more human-like/adapative) Then LLMs outputs influencing behavior or leading conversations. So at scale, a malicious actor can spawn 50,000,000 bots just roaming the internet, with LLM prompts convincing people + manufacturing public opinion. The main bottleneck would be phone number, IP, account age, maybe hardware for compute + device identifiers (which is extremely cheap still)… rather than LLM cost. I’ve already anecdotally seen short sellers use AI bots to spread negative misinformation about companies like $SIVE. Or bots market ____ account as the best one to follow in the comments. Or as seen with X uncovering 200+ bots manipulating debate about datacenter public opinion. But the scale might be exponentially worse soon. Scary to think about.

# Practices for Embedding AI Agents in Software # Semantic Cache with No-Cache Zones 🎯 The Hook Paying for the same answer over and over? Semantic caching slashes costs, but cache the wrong thing and you'll serve stale stock prices or leak one user's data to another. 🔥 The Problem LLM calls are expensive, and when similar queries keep arriving, token costs pile up fast. Exact-match caching barely helps because of phrasing variations. But applying semantic caching indiscriminately creates serious risks: PII-dependent responses served to the wrong user, outdated real-time data, and safety-critical misjudgments replicated at scale. 💡 The Pattern Define "No-Cache Zones" by policy before anything else. Carve out PII-dependent, real-time, and safety-critical categories as forbidden zones. Only within the remaining safe zones does vector-embedding similarity matching apply. Similarity thresholds are tiered by risk level -- 0.92 for low-risk FAQs, 0.95 for medium, 0.97 for high-risk. Cache TTLs include 10-20% jitter to prevent thundering herd effects from mass expiration. ✅ When to Use Use when: - 20%+ of queries are semantically similar repeats - Per-request LLM cost is non-trivial - No-cache zones can be clearly defined by policy Don't use when: - Nearly all queries depend on user-specific context with no reuse potential - Failure cost is uniformly high, making any cached response unacceptable ⚠️ Pitfalls - Changing the embedding model invalidates the entire cache. Record model versions in metadata and plan migration strategy upfront - Weak pattern matching in No-Cache Zone classification lets forbidden queries slip through. Combine rules with an intent classifier in production - Cache poisoning risk: attackers can inject bad responses. Set quality score thresholds on cache writes 🔧 Implementation Approach - Start with rule-based No-Cache Zone detection (pattern matching + metadata flags), then add an intent classifier in production to handle phrasing variations - Set similarity thresholds per risk level, raising them as failure cost increases (starting points: 0.92 low-risk, 0.95 medium, 0.97 high-risk) - For medium-risk zones, add a lightweight revalidation step that verifies cache hits before serving - Record embedding model version in cache metadata and design a migration strategy (gradual re-embedding or flush) for model updates - Add jitter (10-20% random spread) to cache TTLs to prevent thundering herd effects from mass expiration #AIAgents# #SoftwareArchitecture#