You're paying for the same tokens twice. Every API call reloads the same context: system prompts, docs, history. Charged in full. Every time. Context caching in Model Studio fixes that. Learn how today 👇 #AlibabaCloud# #ModelStudio# #CostOptimization#

Still defaulting to your top-tier model for every simple task? LangChain just published how they cut costs on their own coding agent without losing quality. Title: How to Build a Model Router in the Harness URL: Facing ballooning monthly costs on its Open SWE coding agent, LangChain built a model router directly into the harness to balance cost and quality. Three highlights stand out. 📊 Starting from task analysis Analyzing historical traces showed 22% of requests were new features and 17% bug fixes, with wide variation in token consumption. That diversity in task complexity is exactly what makes selective routing worthwhile. 🎯 Three models on the Pareto frontier They picked a fast tier (GLM-5.3-Flash), a balanced tier (GPT-5.6 Sol), and a performance tier (GPT-6 Astra) based on cost-intelligence tradeoffs. Routing happens once at the start of each thread, and that model is used throughout. 💰 64% cost reduction with no quality loss In an A/B test across 973 threads, merged-PR rate was essentially the same — 29.2% with the router versus 27.3% always-premium (p=0.49) — while median cost per thread dropped from $2.61 to $0.94. A parallel test using only the cheapest model was pulled after just one day, underscoring the value of finding that sweet spot instead of going all-cheap or all-premium. #AIAgents# #CostOptimization#

# Useful but Little-Known Features of Claude Agent SDK 🌍 Do you really know how much your agent runs cost? Detailed tracking makes it visible! Claude Agent SDK provides granular token usage and cost breakdowns for every query. 📌 Title: Detailed Cost and Usage Tracking 🔗 URL: 🧩 Overview `ResultMessage` includes `total_cost_usd` (estimated cost), `usage` (token breakdown), and `num_turns` (turn count), available on all result subtypes including errors. Additionally, `model_usage` (Python) / `modelUsage` (TypeScript) provides per-model cost and token breakdowns. When parallel tool calls occur, messages share the same ID with duplicate usage data, so ID-based deduplication is essential for accurate counting. 🛠 How to Use ```python # Python - Get total cost for a query from claude_agent_sdk import query, ResultMessage async for message in query(prompt="Summarize this project"): if isinstance(message, ResultMessage): print(f"Total cost: ${ or 0}") # Per-model breakdown if message.model_usage: for model, usage in message.model_usage.items(): print(f" {model}: ${usage['costUSD']:.4f}") print(f" Input tokens: {usage['inputTokens']}") print(f" Cache read: {usage['cacheReadInputTokens']}") ``` ```typescript // TypeScript - Accumulate costs across multiple queries import { query } from "@anthropic-ai/claude-agent-sdk"; let totalSpend = 0; for (const prompt of prompts) { for await (const message of query({ prompt })) { if (message.type === "result") { totalSpend += // Per-model breakdown for (const [model, usage] of Object.entries(message.modelUsage)) { console.log(`${model}: $${usage.costUSD.toFixed(4)}`); } } } } console.log(`Total: $${totalSpend.toFixed(4)}`); ``` 🏗 Integration into Production Systems - Send `total_cost_usd` to monitoring systems for real-time cost trend visualization - Analyze per-model costs via `model_usage` to optimize subagent cost allocation - Calls that `resume` the same session already include the session's earlier spend (Claude Code v2.1.277 and later), so read the latest result rather than summing, which double-counts; implement summation logic only for independent calls that share no session - Errors still incur costs, so always record costs from failed `ResultMessage` instances too 💡 Use Cases 💰 Report per-user usage costs in multi-tenant environments 📊 Analyze per-model cost breakdowns to find optimal model configurations 🔍 Monitor cache token ratios (`cache_read_input_tokens`) to measure prompt caching effectiveness ⚠️ Caveats - `total_cost_usd` is a client-side estimate and may differ from actual billing. Use the Usage and Cost API for authoritative billing data - Parallel tool calls produce duplicate messages with the same ID; deduplicate by ID to avoid inflated token counts - Setting `ENABLE_PROMPT_CACHING_1H` enables 1-hour TTL caching but increases write costs ✨ No cost optimization without cost visibility. Start by integrating tracking to understand your agent's spending patterns! #ClaudeAgentSDK# #AIAgent#

# Snowflake Features and Practical Usage 🚀 "Bumping the size up makes it faster, but what about cost?" Snowflake cost optimization hinges on answering that question correctly. Let's master virtual warehouse sizing and auto-suspend. 📌 Title and Feature URL Title: Working with Virtual Warehouses URL: 📝 Overview A virtual warehouse is a cluster of compute resources that supplies the CPU, memory, and temporary storage needed to run SQL queries and data operations such as INSERT, UPDATE, DELETE, and COPY. It consumes credits only while running and can be resized or auto-suspended flexibly. Designing size and auto-suspend per workload is the first step in Snowflake cost optimization. 🔧 How It Works Key facts about warehouse sizing and billing: ・Sizes range from X-Small to 6X-Large, and each step up doubles compute and credit consumption. X-Small=1, Small=2, Medium=4, Large=8, X-Large=16, 2X-Large=32 ... up to 6X-Large=512 credits/hour. ・Billing is per-second with a 60-second minimum each time a warehouse starts or resumes. For example, an X-Large running 61 seconds costs about 0.271 credits, while a full hour costs 16 credits. ・Larger warehouses speed up large, complex queries, but larger is not necessarily faster for small, basic queries. ・Besides standard warehouses, Snowpark-optimized warehouses target memory-heavy workloads like ML training. 🛠 Practical Usage ・Use AUTO_SUSPEND (on by default) to suspend after idle time and AUTO_RESUME (on by default) to resume when a statement arrives, preventing wasted credits while idle. ・Create with CREATE WAREHOUSE etl_wh WAREHOUSE_SIZE = XLARGE; for batch, and use WAREHOUSE_SIZE = SMALL AUTO_SUSPEND = 60 for ad-hoc analytics to "pay only for what you use." ・Add INITIALLY_SUSPENDED = TRUE to create the warehouse in a suspended state. ・Warehouses can be resized even while running, so you can temporarily scale up just before a heavy job. 🎯 Use Cases ・Run a daily batch on X-Large to finish fast. Since one size step roughly doubles speed and halves runtime, you cut wall-clock time at a comparable credit cost. ・Set an ad-hoc analytics warehouse to Small with AUTO_SUSPEND=60 so it costs nothing when nobody is querying. ・For data loading, small-to-medium sizes are often sufficient; tune based on file count and size rather than warehouse size. ⚠️ Caveats ・Every resume bills a 60-second minimum, so an extremely short AUTO_SUSPEND (a few seconds) can backfire by triggering frequent start/stop cycles. ・A large size is wasted on small queries. "Scale up for slow queries" is the rule — bigger is not universally better. ・Loading performance depends more on file count and size than on warehouse size. Consider parallelizing files before scaling up. #Snowflake# #DataEngineering#

A 2,500 GB database with a 20 GB hot working set doesn’t need 2,500 GB of RAM. That principle runs through Lakebase Postgres cost optimization: size around what your application actually needs. ✦ Sync only the working set your app queries ✦ Match Snapshot, Triggered or Continuous sync to freshness requirements ✦ Size compute so hot data fits in cache ✦ Set PITR and snapshots around actual recovery needs The full guide breaks down how to keep costs predictable while maintaining the performance, availability and developer experience applications need:

# Practical and Useful Patterns for OpenAI Agent SDK 🌍 Make your agent's token spending visible. Track usage at every level -- from total cost per tenant to per-step breakdown monitoring. The Usage API lets you capture total token consumption for an entire run and drill down into per-step breakdowns for tool calls, handoffs, and guardrails. 📌 Title: Usage 🔗 URL: 🧩 Overview The OpenAI Agent SDK's Usage feature tracks token consumption across agent runs. `result.context_wrapper.usage` gives you total tokens for the entire run, while `result.context_wrapper.usage.request_usage_entries` provides per-step breakdowns (tool calls, handoffs, guardrails). You can use RunHooks' `on_agent_end` for real-time logging, and enable usage reporting for third-party model adapters with `ModelSettings(include_usage=True)`. 🛠 How to Use Create an agent with `Agent(name="Assistant", instructions="Answer user questions.")` and run `result = await "Tell me about quantum computing")`. Access total, input, and output tokens via ` `result.context_wrapper.usage.input_tokens`, and `result.context_wrapper.usage.output_tokens`. Loop through `result.context_wrapper.usage.request_usage_entries` to see per-request breakdowns with `entry.input_tokens` and `entry.output_tokens`. 🏗 Practical Usage Patterns **Per-Tenant/User Cost Calculation** Record "how much each tenant spent" for billing and budget management in SaaS applications. In a `handle_request(tenant_id, user_id, message)` function, run `result = await message)`, compute cost with `calculate_cost(model="gpt-4.1", input_tokens=result.context_wrapper.usage.input_tokens, output_tokens=result.context_wrapper.usage.output_tokens)`, and record it with `await billing_db.record(tenant_id=tenant_id, user_id=user_id, total_tokens= cost_usd=cost, timestamp= **Per-Step Monitoring** Use `request_usage_entries` to identify which tool calls or handoffs consume the most tokens. After `result = await "Generate a sales report")`, loop through `result.context_wrapper.usage.request_usage_entries` with `enumerate` to check each step's `entry.input_tokens` and `entry.output_tokens`. Flag steps where `entry.input_tokens > 5000` with a warning to identify token-heavy operations. **Real-time Logging with RunHooks** Use `on_agent_end` hook to log usage when each agent completes. Create a `UsageLoggingHooks` class extending `RunHooks` with an `on_agent_end(self, context: RunContextWrapper, agent: Agent, output)` method that reads `context.usage` and logs with `"agent_completed", agent_name= total_tokens= ...)`. Pass it via `await "question", hooks=UsageLoggingHooks())`. **Enabling Usage for Third-party Models** When using non-OpenAI model adapters, explicitly enable usage reporting. For third-party models, explicitly enable usage reporting with `Agent(name="ThirdPartyAgent", instructions="...", model=my_custom_model_adapter, model_settings=ModelSettings(include_usage=True))`. 💡 Use Cases 💰 Record per-tenant/user token consumption in a database to build usage-based billing 📊 Visualize per-step token consumption with request_usage_entries to identify cost optimization bottlenecks 🔔 Log usage via RunHooks on_agent_end for real-time dashboard monitoring 🔌 Enable ModelSettings(include_usage=True) for third-party models to ensure usage tracking coverage ⚠️ Caveats - ` includes retries and guardrail re-executions, so it may be higher than the visible step count suggests. - Third-party models may return zero usage without `include_usage=True`. Check your adapter's documentation. - Cost calculations depend on model pricing. Ensure you accurately reflect per-model input/output token rates. ✨ Track token usage at three levels -- total, per-step, and via hooks -- and you'll have cost management and quality monitoring fully covered. Start with result.context_wrapper.usage, then add step-level and hook-based tracking as needed! #OpenAIAgentSDK# #AIAgent#

# Decision Points in AI Agent Development 🎯 The Hook Single LLM provider or multi-provider? "We should use multiple providers for resilience" sounds prudent, but have you priced in the real cost? Maintaining abstraction layers, tuning prompts per provider, running parallel quality evaluations. The overhead of multi-provider is routinely underestimated. On the other hand, are you comfortable betting everything on one vendor? 📋 Overview This is about whether your AI agent should use a single LLM provider (OpenAI, Anthropic, Google, Azure OpenAI, etc.) or multiple providers in parallel. "Provider" here means the API vendor, not different models within the same vendor. Each provider differs in API specification (tool calling format, streaming protocol, error codes), pricing, rate limits, and availability SLA. Moving to multi-provider ripples through infrastructure, prompts, and evaluation -- operational cost increases non-linearly. 🔍 Decision Points The key axes are provider trust and system availability requirements. 📌 SLA requirement is 99.9%+ and the provider's track record falls short → Consider multi-provider. LLM provider outages and rate limits tend to be more frequent and less predictable than traditional SaaS. 📌 Heavy dependency on provider-specific features (prompt caching, batch API, fine-tuning) → Stay single-provider. An abstraction layer reduces you to the lowest common denominator, losing provider-specific optimizations. 📌 Small team → Stay single-provider. If you lack the resources to maintain parallel prompt tuning and quality evaluation across providers, quality management on the secondary provider will suffer. 📌 Regulatory or data sovereignty requirements → Multi-provider. When certain requests must be routed to specific regions or providers, there is no alternative. 💡 Key Details 🔹 Single provider's greatest asset is depth of optimization. You can focus prompt tuning on one model family and leverage prefix caching and context caching for major cost and latency reductions. Output quality, style, and structured output precision remain stable. 🔹 Multi-provider's greatest asset is fault tolerance. When Provider A goes down, automatic failover to Provider B maintains availability. Task-based provider selection also enables cost optimization. The "always ready to switch" posture doubles as negotiation leverage. 🔹 The hidden costs of multi-provider: building and continuously maintaining adapters that absorb API differences (tool calling formats, streaming protocols, error codes) across providers. The same prompt performs differently across providers, requiring per-provider prompt tuning and evaluation. Pre-validating quality degradation during failover scenarios. ⚖️ Trade-offs Single provider gains depth of optimization and operational simplicity at the cost of vendor lock-in and availability risk. Multi-provider gains fault tolerance and strategic flexibility at the cost of abstraction layer maintenance and quality management complexity. 🏆 The proven approach: start with a single provider behind an abstraction layer. Use something like LiteLLM or an AI Gateway from day one, even with just one provider. When availability or cost issues surface, adding a second provider becomes a manageable migration rather than a rewrite. Direct integration without an abstraction layer makes future migration prohibitively expensive. 🛠️ Use Cases 🚀 Startup MVP → Single provider with abstraction layer. Prioritize development speed, but keep the door open for multi-provider later. 🏥 High-SLA systems (healthcare, finance) → Multi-provider. When downtime is unacceptable, pre-validate your failover target and combine with circuit breaker patterns for automatic switchover. 🌍 Global deployment with data sovereignty → Multi-provider. Route EU requests to EU-based providers, Japan requests to domestic regions. 📊 High-traffic systems → Multi-provider. Distribute traffic across providers for rate limit relief and cost optimization. #AIAgents# #SoftwareArchitecture#

$ASAN Asana CEO: Model routing is becoming a margin lever “We've gotten rather good at… figuring out which types of tasks should go to which types of model… to both solve for quality and cost optimization.”

.@grok estimates that sending 4 humans to summit Olympus Mons (~22km, the tallest volcano in the solar system) would cost $1.5–18B, mostly depending on how well @elonmusk does with Starship cost optimization. I think "we" should do it.