Register and share your invite link to earn from video plays and referrals.

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture ๆŠ•็จฟใฏๅ€‹ไบบใฎๆ„่ฆ‹ใงใ™ใ€‚
Joined May 2026
282 Following    431 Followers
# Practical and Useful Patterns for OpenAI Agent SDK ๐ŸŒ Make your agent's token spending visible. Track usage at every level -- from total cost per tenant to per-step breakdown monitoring. The Usage API lets you capture total token consumption for an entire run and drill down into per-step breakdowns for tool calls, handoffs, and guardrails. ๐Ÿ“Œ Title: Usage ๐Ÿ”— URL: ๐Ÿงฉ Overview The OpenAI Agent SDK's Usage feature tracks token consumption across agent runs. `result.context_wrapper.usage` gives you total tokens for the entire run, while `result.context_wrapper.usage.request_usage_entries` provides per-step breakdowns (tool calls, handoffs, guardrails). You can use RunHooks' `on_agent_end` for real-time logging, and enable usage reporting for third-party model adapters with `ModelSettings(include_usage=True)`. ๐Ÿ›  How to Use Create an agent with `Agent(name="Assistant", instructions="Answer user questions.")` and run `result = await "Tell me about quantum computing")`. Access total, input, and output tokens via ` `result.context_wrapper.usage.input_tokens`, and `result.context_wrapper.usage.output_tokens`. Loop through `result.context_wrapper.usage.request_usage_entries` to see per-request breakdowns with `entry.input_tokens` and `entry.output_tokens`. ๐Ÿ— Practical Usage Patterns **Per-Tenant/User Cost Calculation** Record "how much each tenant spent" for billing and budget management in SaaS applications. In a `handle_request(tenant_id, user_id, message)` function, run `result = await message)`, compute cost with `calculate_cost(model="gpt-4.1", input_tokens=result.context_wrapper.usage.input_tokens, output_tokens=result.context_wrapper.usage.output_tokens)`, and record it with `await billing_db.record(tenant_id=tenant_id, user_id=user_id, total_tokens= cost_usd=cost, timestamp= **Per-Step Monitoring** Use `request_usage_entries` to identify which tool calls or handoffs consume the most tokens. After `result = await "Generate a sales report")`, loop through `result.context_wrapper.usage.request_usage_entries` with `enumerate` to check each step's `entry.input_tokens` and `entry.output_tokens`. Flag steps where `entry.input_tokens > 5000` with a warning to identify token-heavy operations. **Real-time Logging with RunHooks** Use `on_agent_end` hook to log usage when each agent completes. Create a `UsageLoggingHooks` class extending `RunHooks` with an `on_agent_end(self, context: RunContextWrapper, agent: Agent, output)` method that reads `context.usage` and logs with `"agent_completed", agent_name= total_tokens= ...)`. Pass it via `await "question", hooks=UsageLoggingHooks())`. **Enabling Usage for Third-party Models** When using non-OpenAI model adapters, explicitly enable usage reporting. For third-party models, explicitly enable usage reporting with `Agent(name="ThirdPartyAgent", instructions="...", model=my_custom_model_adapter, model_settings=ModelSettings(include_usage=True))`. ๐Ÿ’ก Use Cases ๐Ÿ’ฐ Record per-tenant/user token consumption in a database to build usage-based billing ๐Ÿ“Š Visualize per-step token consumption with request_usage_entries to identify cost optimization bottlenecks ๐Ÿ”” Log usage via RunHooks on_agent_end for real-time dashboard monitoring ๐Ÿ”Œ Enable ModelSettings(include_usage=True) for third-party models to ensure usage tracking coverage โš ๏ธ Caveats - ` includes retries and guardrail re-executions, so it may be higher than the visible step count suggests. - Third-party models may return zero usage without `include_usage=True`. Check your adapter's documentation. - Cost calculations depend on model pricing. Ensure you accurately reflect per-model input/output token rates. โœจ Track token usage at three levels -- total, per-step, and via hooks -- and you'll have cost management and quality monitoring fully covered. Start with result.context_wrapper.usage, then add step-level and hook-based tracking as needed! #OpenAIAgentSDK# #AIAgent#
Show more