Register and share your invite link to earn from video plays and referrals.

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture ๆŠ•็จฟใฏๅ€‹ไบบใฎๆ„่ฆ‹ใงใ™ใ€‚
Joined May 2026
284 Following    451 Followers
# Practical and Useful Patterns for OpenAI Agent SDK ๐ŸŒ Unbounded history is the enemy of cost and latency! Use history control and compaction to keep context size optimal. ๐Ÿ“Œ Title: Sessions โ€“ Control history / Compaction ๐Ÿ”— URL: ๐Ÿงฉ Overview Long-running sessions cause history bloat, driving up token costs and latency. Use `SessionSettings(limit=N)` to cap history to the most recent N items, `session_input_callback` for custom trimming and reordering, `OpenAIResponsesCompactionSession` for automatic compression, and manual `run_compaction()` for idle-time optimization. ๐Ÿ›  Usage The simplest approach is `SessionSettings(limit=50)`. Create an agent with `Agent(name="assistant", instructions="Answer helpfully")` and a session with `SQLiteSession("user_12345")`, then pass the settings through `RunConfig`: `await "What's the latest?", session=session, run_config=RunConfig(session_settings=SessionSettings(limit=50)))`. Only the most recent 50 items are retrieved from the session. For advanced control, use `session_input_callback`. Define a `custom_history_filter(history, new_input)` function that keeps system messages via `h.role == "system"`, limits regular exchanges to `history[-20:]`, and returns `filtered + new_input` (`new_input` is a list). Pass it as `await "...", session=session, run_config=RunConfig(session_input_callback=custom_history_filter))`. For automatic compression, use `OpenAIResponsesCompactionSession`. Create `underlying = SQLiteSession("user_12345")`, wrap it with `OpenAIResponsesCompactionSession(session_id="user_12345", underlying_session=underlying)`, and run `await "Summarize our discussion so far", session=session)` โ€” long conversations are automatically summarized and compacted. For manual compaction, disable auto-compaction with `OpenAIResponsesCompactionSession(session_id="user_12345", underlying_session=underlying, should_trigger_compaction=lambda _: False)` and call `await "force": True})`. Running this during user idle time improves latency on the next request. ๐Ÿ— Practical Patterns `SessionSettings(limit=50)` is the simplest cost control. Only the most recent 50 items are retrieved and sent to the model, while the default `limit=None` retrieves everything. This works well for customer support where recent exchanges matter most. `session_input_callback` is for advanced control. For example, always retain system messages and VIP customer flags while limiting regular exchanges to the most recent 20. The key insight is that it only filters what's sent to the LLM โ€” storage remains unchanged. `OpenAIResponsesCompactionSession` performs automatic compression, summarizing long conversations into shorter representations to use the context window efficiently. For streaming-first applications, manually running `run_compaction()` during user idle time is effective. The next request starts from compacted history, improving time-to-first-token latency. ๐Ÿ’ก Use Cases ๐Ÿ’ฐ Token cost capping via limit settings ๐Ÿ”ง Selective context retention with custom callbacks ๐Ÿ“ฆ Context optimization for long sessions via auto-compaction โšก Streaming-first latency improvement with idle-time compaction โš ๏ธ Caveats - `limit` caps how many items are retrieved, not what is stored โ€” the session keeps the full history, but anything beyond the limit never reaches the model, so set the value carefully - `session_input_callback` runs on every request โ€” avoid heavy processing - Auto-compaction depends on summarization accuracy; exact numbers and proper nouns may be lost - `run_compaction()` involves API calls, so calling it too frequently increases costs โœจ Proper history control keeps cost and latency optimal even in long-running sessions! #OpenAIAgentSDK# #AIAgent#
Show more