Register and share your invite link to earn from video plays and referrals.

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
286 Following    468 Followers
# Practical and Useful Patterns with ADK 🧪 You can't hit production APIs during agent testing. ADK's Environment Simulation intercepts tool calls and replaces them with controlled responses, letting you reproduce edge cases without affecting production systems. 📌 Title: Environment Simulation 🔗 URL: 🧩 Overview ADK's Environment Simulator intercepts tool calls during agent execution and replaces them with controlled responses — all without modifying agent code. It supports error injection, fixed responses, conditional injection, probabilistic failures, and latency simulation. It integrates as a Callback or Plugin, with a decision priority of: injection configs, then mock strategy, then real tool execution. 🛠 Usage Integration as a Callback: Import `LlmAgent` from `google.adk.agents`, `EnvironmentSimulationFactory` from ` and `EnvironmentSimulationConfig`, `InjectedError`, `InjectionConfig`, `ToolSimulationConfig` from the `environment_simulation_config` submodule. Create an `EnvironmentSimulationConfig` with a list of `ToolSimulationConfig` entries in `tool_simulation_configs`. Each entry specifies a `tool_name` (e.g., `"get_user_profile"`) and `injection_configs` containing `InjectionConfig` instances -- for example, an `InjectedError` with `injected_http_error_code=503` and an `error_message`. Convert the config to a callback via `EnvironmentSimulationFactory.create_callback(config)` and pass it as the `before_tool_callback` parameter of `LlmAgent`. Fixed response injection: Set the `injected_response` parameter of `InjectionConfig` to a dictionary (e.g., `{"status": "ok", "order_id": "ORD-9999"}`) to return a fixed response for tool calls. Conditional injection based on arguments: Use `match_args` in `InjectionConfig` to specify an argument pattern (e.g., `{"item_id": "ITEM-404"}`). The injection triggers only when tool call arguments match. Pair it with `injected_error` set to `InjectedError(injected_http_error_code=404, error_message="Item not found.")` to return a 404 error for that specific argument. Probabilistic failure simulation: Set `injection_probability` on `InjectionConfig` to a value like `0.3` for a 30% failure rate, and `random_seed` (e.g., `42`) for reproducibility. Combined with `injected_error` set to `InjectedError(injected_http_error_code=500, error_message="Internal server error.")`, this randomly injects 500 errors at the specified probability. Latency simulation: Set `injected_latency_seconds` on `InjectionConfig` to the desired delay (e.g., `5.0` seconds) along with an `injected_response` (e.g., `{"result": "slow but successful"}`). The response is returned after the specified delay. Mock Strategy mode auto-generates responses from tool schemas: Import `MockStrategy` from `environment_simulation_config` and set `mock_strategy_type` to `MockStrategy.MOCK_STRATEGY_TOOL_SPEC` on a `ToolSimulationConfig` with the target `tool_name` (e.g., `"create_order"`). This auto-generates responses based on the tool's schema definition. 🏗 Practical Patterns **Testing Agent Resilience to API Failures**: Inject 503 or 500 errors to verify that agents handle retries and fallbacks correctly. Reproduce failure scenarios without touching production systems. **Edge Case Testing with Conditional Injection**: Use `match_args` to trigger errors only for specific argument patterns — "non-existent product ID," "out-of-stock item." Pinpoint-test agent behavior under specific conditions. **Realistic Testing with Probabilistic Failures**: Use `injection_probability` to simulate flaky APIs with, say, 30% failure rate. Fix `random_seed` for reproducible tests. **Realistic Mock Generation with Environment Data**: Provide DB snapshots or other context via `environment_data` so the Mock Strategy generates more realistic responses: Convert context data such as a DB snapshot (e.g., a dictionary of product records) to a JSON string via `json.dumps()` and pass it as the `environment_data` parameter of `EnvironmentSimulationConfig`. The Mock Strategy references this environment data to generate more realistic responses. **Combining Injections with Mock Strategy**: Inject errors for specific conditions while using Mock Strategy for everything else. Injection configs are evaluated first; unmatched calls fall through to the mock strategy. 💡 Use Cases 🔧 Verifying agent resilience to external API failures 🎯 Reproducing edge cases under specific conditions 📉 Probabilistic simulation of flaky API interactions ⏱ Verifying timeout handling with high-latency APIs ⚠️ Caveats - `injected_latency_seconds` has a maximum of 120 seconds. - `InjectionConfig` accepts either an error or a response, not both simultaneously. - `tool_name` must exactly match the registered tool name. Watch for typos. - Mock Strategy depends on tool schemas — insufficient schemas lead to lower-quality generated responses. ✨ Environment Simulation is a powerful testing foundation that reproduces edge cases without contaminating production. Combine error injection with conditional mocks to verify agent resilience against every failure scenario! #ADK# #AIAgent#
Show more
# Useful but Little-Known Features of Claude Agent SDK 🌍 Want to send messages while the agent is working? Streaming input mode enables interactive sessions! With streaming input mode, you can add messages, interrupt execution, and send images during an active session. 📌 Title: Streaming Input Mode 🔗 URL: 🧩 Overview Streaming input mode (the recommended, preferred mode) provides a persistent, interactive session using `AsyncGenerator`. Unlike single message input (one-shot), it supports message queueing, image attachments, and mid-execution interruption during a session. Messages are sent sequentially via `yield`, and follow-up messages can be dynamically generated based on conditions — ideal for chat interfaces and complex workflows. 🛠 How to Use ```python # Python - Interactive session with streaming input from claude_agent_sdk import ClaudeSDKClient, ClaudeAgentOptions, AssistantMessage, TextBlock import base64 async def message_generator(): yield { "type": "user", "message": {"role": "user", "content": "Analyze for security issues"}, } # Follow-up with image with open("diagram.png", "rb") as f: image_data = base64.b64encode( yield { "type": "user", "message": { "role": "user", "content": [ {"type": "text", "text": "Also review this architecture diagram"}, {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": image_data}}, ], }, } options = ClaudeAgentOptions(max_turns=10, allowed_tools=["Read", "Grep"]) async with ClaudeSDKClient(options) as client: await client.query(message_generator()) async for message in client.receive_response(): if isinstance(message, AssistantMessage): for block in message.content: if isinstance(block, TextBlock): print(block.text) ``` ```typescript // TypeScript - Send messages sequentially via AsyncGenerator import { query, type SDKUserMessage } from "@anthropic-ai/claude-agent-sdk"; async function* generateMessages(): AsyncGenerator { yield { type: "user", message: { role: "user", content: "Analyze this codebase for security issues" }, parent_tool_use_id: null, }; // Dynamic follow-up based on conditions await new Promise((resolve) => setTimeout(resolve, 2000)); yield { type: "user", message: { role: "user", content: "Now focus on authentication" }, parent_tool_use_id: null, }; } for await (const message of query({ prompt: generateMessages(), options: { maxTurns: 10, allowedTools: ["Read", "Grep"] } })) { if (message.type === "result" && message.subtype === "success") { console.log(message.result); } } ``` 🏗 Integration into Production Systems - Wire chat UI message sends to the generator's `yield` to pass user input to the agent in real time - Include `base64`-encoded image uploads in messages for visual analysis capabilities - Send interrupt signals to redirect the agent, enabling control over long-running tasks - Context is naturally maintained within the session, eliminating multi-turn conversation management overhead 💡 Use Cases 💬 Build interactive sessions between users and agents in custom chatbots 🖼 Dynamically send screenshots or diagrams during code review for analysis 🔄 Send additional instructions or shift focus areas during long-running code analysis ⚠️ Caveats - Single message input does not support image attachments, dynamic message queueing, or real-time interruption - For stateless environments (Lambda functions, etc.), use single message input instead - In Python, streaming input requires the `ClaudeSDKClient` context manager ✨ The real power of AI agents goes beyond one-shot queries. Build rich interactive experiences with streaming input! #ClaudeAgentSDK# #AIAgent#
Show more
Even with automated evals, only a person can tell why an agent failed. LangChain argues expert judgment is a strategic advantage 🧑‍⚖️ Why human insight is a strategic advantage for building better agents Automated evaluation can flag problems, but it can't explain the cause or the fix. The article lays out three principles for building human review into the evaluation system. 🎯 Principle 1: humans define what "good" looks like Reviewers supply context a trace doesn't contain, such as the standard of quality for the job and what the user was after. Scores with reasoning also expose gaps in the evaluators themselves. 🧵 Principle 2: context makes the judgment hold up Looking at one step misses contradictions with something said several turns earlier. For multi-turn agents, the whole conversation (a thread) is the unit of review. 🔁 Principle 3: turn judgment into evaluation data Labels become ground truth, used to align LLM judges with human scores, update online evaluators, and build golden datasets. In the case study, Fetch Rewards manually checks even product-card images and offer availability. Automation handles scale; the advantage comes from turning expert judgment into a reusable asset. #AIAgents# #LLMEvaluation#
Show more
Can dense, sparse and multi-vector approximate nearest neighbor search live in a single Rust codebase? Modular and Efficient Approximate Nearest Neighbors Search with Vectorium and kANNolo 🧩 Overview Two Rust libraries: Vectorium for data management and kANNolo for indexing. Encoders and datasets are compile-time type parameters, so each combination is specialized at build time. 🎯 The problem ・Faiss has a separate code path per index/quantizer combination, which is hard to extend ・Other libraries factor out only the graph or only the quantization ・None handles all three embedding types behind one interface 🛠 Method ・11 encoders (PQ, RaBitQ, sparse, multi-vector, and more) ・Hnsw and IVF, plus a two-stage RerankIndex that pairs any first-stage index with any rescoring ・A structure-of-arrays PQ table layout makes table building 4.5-5.2x faster 📊 Results ・Up to 3.26x hnswlib's throughput at 99% accuracy (kANNolo-Rerank, dense data) ・Smallest memory on every sparse collection ・But Glass wins on AgNews, Seismic is faster on sparse data, and builds are not fast #ANNSearch# #Rust#
Show more
# Practical and Useful Patterns for OpenAI Agent SDK 🌍 Customize where your traces go. Send alongside OpenAI to Langfuse or Datadog, or replace the backend entirely -- seamlessly integrate with your existing observability stack. Custom processors give you full control over trace routing, transformation, and filtering. 📌 Title: Tracing -- Custom processors / Ecosystem 🔗 URL: 🧩 Overview The OpenAI Agent SDK's tracing is fully extensible through custom processors. Use `add_trace_processor()` to add custom processing alongside the OpenAI backend, or `set_trace_processors()` to completely replace the destination with Langfuse, Datadog, MLflow, Arize, or Braintrust. Even when using non-OpenAI models, `set_tracing_export_api_key()` enables free trace visualization. 🛠 How to Use Import `add_trace_processor` and `TracingProcessor` from `agents`. Define a `MyProcessor` class extending `TracingProcessor` with `on_trace_start`, `on_trace_end`, `on_span_start`, `on_span_end`, and `shutdown` methods. Access ` `trace.duration_ms`, and ` for trace data. Register with `add_trace_processor(MyProcessor())` to run custom processing alongside the OpenAI backend. 🏗 Practical Usage Patterns **Parallel Export with add_trace_processor()** Keep the OpenAI Traces dashboard while also sending data to your own observability platform. Extend `TracingProcessor` to create a `DatadogProcessor`. In `on_trace_start`, call `dd_client.start_span(operation_name="agent.trace", resource= In `on_trace_end`, call `dd_client.finish_span(metadata={"duration_ms": trace.duration_ms})`. In `on_span_start`, use `operation_name=f"agent.{span.type}"`. In `shutdown`, call `dd_client.flush()`. Register with `add_trace_processor(DatadogProcessor(dd_client))` to send traces to Datadog in parallel with OpenAI. **Complete Backend Replacement with set_trace_processors()** When you don't want traces going to OpenAI, replace the processor entirely. Import `set_trace_processors` from `agents` to completely replace the trace destination. Use `set_trace_processors([LangfuseTracingProcessor()])` for Langfuse, `[MLflowTracingProcessor()]` for MLflow, `[ArizeTracingProcessor()]` for Arize, or `[BraintrustTracingProcessor()]` for Braintrust. This replaces all existing processors including the OpenAI backend. **Free Traces for Non-OpenAI Models with set_tracing_export_api_key()** Use trace visualization for free even when running non-OpenAI models (Gemini, Claude, etc.). Import `set_tracing_export_api_key` from `agents` and call `set_tracing_export_api_key("your-tracing-export-key")`. This enables free trace visualization even when using non-OpenAI models like Gemini or Claude. **Combining Multiple Processors** In production, it's common to send traces to multiple observability tools. Pass multiple processors as a list to `set_trace_processors`: `LangfuseTracingProcessor()` for detailed LLM analytics, `DatadogProcessor(dd_client)` for infrastructure monitoring integration, and `CustomAuditProcessor()` for internal audit logging -- all running simultaneously. 💡 Use Cases 📡 Use add_trace_processor() to send traces to Datadog in parallel while keeping the OpenAI dashboard 🔀 Switch destinations entirely to Langfuse/MLflow/Arize/Braintrust with set_trace_processors() 🆓 Visualize non-OpenAI model (Gemini, Claude) traces for free with set_tracing_export_api_key() 🔧 Build production observability pipelines combining multiple processors (Langfuse + Datadog + audit logs) ⚠️ Caveats - `set_trace_processors()` **replaces all** existing processors, including the OpenAI backend. Use `add_trace_processor()` if you want to keep it. - Always implement the `shutdown()` method in custom processors to flush any buffered data. - Multiple processors add overhead to trace delivery. Consider asynchronous sending to minimize impact. ✨ Custom processors let you pipe trace data directly into your existing observability stack. Start with add for parallel export, then optimize destinations with set once you're confident! #OpenAIAgentSDK# #AIAgent#
Show more
# OpenCode Features and Practical Usage 🤖 Turn OpenCode from "a tool you chat with in the terminal" into "a dev server you drive from code." The SDK plus a headless server let you embed AI coding into CI and custom UIs. 🏷️ Title: Type-safe JS client and headless server 🔗 URL: 📘 Overview OpenCode can run not only as a TUI but as a headless server you control over HTTP. The type-safe `@opencode-ai/sdk` JavaScript/TypeScript client connects to that server to create sessions, send prompts, and subscribe to events. All types are generated from the server's OpenAPI spec, so completion and type-checking work out of the box. ⚙️ How It Works ・Start the server: `opencode serve` launches a headless server, defaulting to `127.0.0.1:4096`. The SDK's `createOpencode()` can both start the server and hand you a client in one call. ・Connect a client: For an existing server, use `createOpencodeClient({ baseUrl: "http://localhost:4096" })`. ・Core APIs: Create/list/get/update/delete sessions, share/unshare, abort, and revert/unrevert messages. Send prompts with `client.session.prompt()`. ・Structured output: Pass `format: { type: "json_schema", schema: {...} }` to receive validated JSON matching your schema, ideal for downstream machine processing. ・Event stream: `client.event.subscribe()` gives you server-sent events as a stream to monitor long-running operations. 🛠️ Practical Usage Starting the server from code is a few calls: `createOpencode()` returns both a `client` and `server`, then `client.session.create()` opens a session and `client.session.prompt()` sends a prompt (passing a `format` of `json_schema` for structured output), and `server.close()` shuts it down. For CI, `noReply: true` is handy when you want to inject context without triggering an AI response. 💡 Use Cases ・A review bot fetches a structured findings JSON via the SDK (instead of `opencode run`) and posts a formatted GitHub PR comment. ・A custom web UI or editor extension uses the `tui` APIs to append/submit prompts and show toasts, building a bespoke frontend. ・A nightly batch sends the same analysis prompt across many repos and stores the resulting JSON in a data platform. ⚠️ Caveats ・SDK config overrides and merges with `opencode.json` rather than replacing it entirely, so watch for lingering unintended settings. ・Structured output retries on validation failure (default 2). If it still fails, ` holds a `StructuredOutputError`. Keep schemas small and focused for reliability. ・Calls like `client.session.get()` throw on an invalid ID, so wrap them in try-catch. #OpenCode# #AIAgents#
Show more
TL;DR: LangChain's Harrison Chase pushes back on the idea that the hard problems of building agents are solved. He sees six open challenges for always-on, goal-driven agents. New context engineering challenges in late 2026 Key points ・🔔 Proactivity: balancing useful updates against noisy notifications is unsolved ・🧠 Memory: the hard part is the intelligence layer that decides what to remember, not storage ・⏳ Async work: long-running tasks must run while the main agent stays responsive ・🎯 Goal tracking: avoid repeating actions and stay consistent over long horizons ・⚠️ Specification gaming: an agent told to remove all bugs deletes the codebase ・🌐 Browser use: screenshots or DOM, and context management when both are large A map of what comes after summarization, offloading, skills and subagents. Note there are no numbers or solutions; it's an opinion piece. #ContextEngineering# #AIAgents#
Show more
Life once upon a TIME is live 🎲 Pick your era, country, class and gender, or let fate roll them. 7 settings, pixel art, ~1 hour per life, 5 languages (EN/JA/ZH/KO/ES). No sign-up. #indiegame# #browsergame#
Show more
When an LLM picks from fixed options, wouldn't it be handy to adjust after the fact whether it thinks first? 🤔 ThinkJev: One Thought, Every Budget for Calibrated Typed Decisions ❓ What is a typed decision? 💡 A choice among a fixed set of options that code then acts on, such as routing a ticket or checking a rule. Reading option probabilities in one forward pass is fast, but it fails on multi-step decisions like date arithmetic. ❓ How does it solve this? 💡 With no training, one Qwen3 model plays two roles: System 1 answers without thinking, System 2 answers after thinking, both on the same cached prompt. Cutting one thought short and closing it also gives an accuracy curve for every budget. ❓ How well does it work? 💡 On Qwen3.5-27B, direct answers score 62.2% and always thinking scores 82.7%. The anytime policy reaches 81.6% using 72% of the thought tokens. At a 5% target, System 1 answers 24.5% of decisions with a realized error of 3.9%. ❓ Any caveats? 💡 The pre-registered savings criterion failed on every model. System 1 confidence is a weak signal for when to think, and calibration must be refit when the thinking budget changes. #LLMReasoning# #Calibration#
Show more
Posted a blog for development of "Life once upon a TIME"
You are not a king or a hero. You are one of the countless people history forgot. Be born a farmer in Edo, a citizen of Rome, a Sengoku foot soldier. Live a whole life in about an hour, then get graded F to A. Free, no login:
Show more
Even when an agent holds the right permissions, "should it take this action?" is a separate question 🛡️ Securing AI Agents: Lessons From Real-World Deployment Uber has shared the design and lessons of "Uber Secure AI," its defense-in-depth framework for protecting its own AI agents. Authorization answers who may act, but agents also need a layer that checks whether an action fits the request. Three highlights: 📜 Register a behavioral contract The AI Registry records each agent's persona, permitted intents, constraints, and risk tier. Argus derives actual capabilities from code and prompts and checks them against what was declared. ⚡ Decide at runtime with small models AI Guard evaluates proposed actions and enforces its verdict before they run. Because it sits in the execution path, it uses small specialized models hosted locally and running in parallel, not remote general-purpose LLMs. 🎯 Limiting intents and constraints worked best Restricting agents to explicit intents and constraints greatly reduced their ability to act on manipulated instructions, at the cost of some flexibility. Easy onboarding is treated as part of the control so developers don't bypass it. It's also useful that the authors say plainly that guardrails reduce risk but don't eliminate it. No quantitative results are given. #AIAgents# #AISecurity#
Show more
# Decision Points for Embedding AI Agents in Enterprise Systems # General-Purpose Large Model vs Small Fine-Tuned Model — Choosing the Inference Engine 🎯 The Hook For your agent's inference engine, do you go with a GPT-4-class general-purpose large model controlled via prompts, or a task-specific small fine-tuned (FT) model? "Bigger is smarter" is true, but "bigger is always the right answer" is not. The optimal choice shifts based on task diversity, call frequency, cost constraints, and rate of change. Most enterprises find the practical sweet spot in a staged migration: start with a large model, accumulate training data from logs, then replace high-frequency tasks with FT models one by one 🔑 📋 Overview General-purpose large model plus prompting shines when tasks are diverse and cannot be enumerated in advance, when business requirements change rapidly, and when you want to handle new tools or APIs by simply updating prompts. Small FT models are appropriate when the task is narrowly defined with stable input/output patterns, call frequency is high and cost reduction matters, or latency requirements are strict. These are not mutually exclusive — a router plus specialist model hybrid is the standard enterprise approach 📊 🔍 Decision Points Here is the decision flow: If task types number 10 or more with new ones added frequently, choose the general-purpose large model. The flexibility to handle new tasks with just a prompt change is a decisive advantage. If monthly request volume exceeds several hundred thousand, calculate the cost benefit of small FT. Compare on total cost of ownership — not just API call costs, but also training costs, GPU inference infrastructure, and training data management overhead. If the latency requirement is P95 under 500ms, prioritize small FT. Large model inference times may not meet this bar. If you cannot prepare and continuously update several thousand or more training examples, stay with the large model. FT on insufficient training data will underperform the general-purpose model. If business requirements change more frequently than monthly, the re-training cost often makes maintaining the large model the better choice ⚡ 💡 Key Details Three hybrid patterns to understand: Router plus specialist models — Use the large model to classify request intent, then route routine tasks to small FT models. Intent classification cost is low, and specialist task inference cost drops dramatically. A Slack Bot takes company-wide questions, routes ticket classification to a small FT, and handles complex investigations with the large model. FT for drafts, large model for finishing — Generate drafts quickly with a small FT model, then verify and refine with the large model only for quality-critical cases. A practical pattern that balances cost and quality. Staged migration — Start with everything on the large model, accumulate training data from call logs, and replace high-frequency tasks with small FT models one by one. This is the lowest-risk approach and the one most enterprises adopt 🔄 ⚖️ Trade-offs Premature FT investment is the most common failure. Building an FT model before sufficient logs accumulate produces accuracy below the general-purpose model. Wait for at least three months of logs before considering FT. Reaching for FT during the PoC phase is premature 😩 FT model staleness is equally serious. Business rules and product lineups change, but training data is not updated, causing the model to answer with outdated knowledge. Automating the re-training pipeline is a mandatory investment for operating FT models. Cost comparison errors are frequent. Comparing only API call costs while overlooking FT training costs, GPU inference infrastructure costs, and training data management costs leads to flawed decisions. Compare on TCO (total cost of ownership) or the analysis is meaningless. Missing quality evaluation is another blind spot. Without regularly evaluating FT model accuracy, model drift goes undetected. Continuous A/B testing against the general-purpose model is the foundation of quality assurance ⚠️ 🛠️ Use Cases Company-wide help desk agent (Slack Bot): Handles broad queries across IT, admin, HR, and accounting. The large model classifies intent, then routes ticket categorization and prioritization (tens of thousands per month) to a small FT. Complex investigations and novel question types are handled by the large model 📚 Ticket classification agent (Zendesk integration): Automatically classifies category, priority, and assigned team for tens of thousands of support tickets daily. High call frequency and stable I/O patterns make small FT the optimal choice. Training data is updated monthly, and accuracy is monitored via A/B testing against the large model 🏢 Product review analysis agent (Shopify integration): Sentiment analysis and tagging of product reviews are batch-processed by a small FT. Customer insight summaries derived from reviews are generated by the large model. Routine classification goes to FT, creative analysis goes to the large model — a clear division of responsibility 🔧 Practical tip: "Start with the large model and accumulate logs" is the golden rule. Identify high-frequency tasks from three or more months of logs, build training data, validate FT effectiveness via A/B testing against the large model, then migrate. Make the FT transition a data-driven decision 💪 #AIAgents# #EnterpriseArchitecture#
Show more
# Useful but Little-Known Features of ADK 2.0 🌍 What if your agent's prompts could improve themselves automatically, instead of requiring manual tuning? ADK 2.0's optimization feature combines samplers and agent optimizers to automatically refine your agent's instructions based on evaluation results. 📌 Title: Optimization 🔗 URL: 🧩 Overview The optimization feature consists of two components: Samplers (which evaluate candidates) and Agent Optimizers (which improve instructions). `LocalEvalSampler` runs candidate agents against evaluation datasets and computes scores. Two optimizers are available: `GEPARootAgentPromptOptimizer`, which uses the GEPA algorithm to optimize the root agent's prompt, and `SimplePromptOptimizer`, which iterates through an execute-evaluate-critique-rewrite cycle. Results are returned as an `OptimizerResult` containing optimized agents with their scores. 🛠 How to use it Run optimization from the CLI with `adk optimize`. ```python import asyncio from google.adk.evaluation.eval_config import EvalConfig from google.adk.evaluation.local_eval_sets_manager import LocalEvalSetsManager from google.adk.optimization.local_eval_sampler import LocalEvalSampler from google.adk.optimization.local_eval_sampler import LocalEvalSamplerConfig from google.adk.optimization.simple_prompt_optimizer import SimplePromptOptimizer from google.adk.optimization.simple_prompt_optimizer import ( SimplePromptOptimizerConfig, ) # Set up the sampler sampler_config = LocalEvalSamplerConfig( eval_config=EvalConfig(criteria={"response_match_score": 0.75}), app_name="my_agent", train_eval_set="train_eval_set", ) sampler = LocalEvalSampler(sampler_config, LocalEvalSetsManager(agents_dir=".")) # Run optimization optimizer = SimplePromptOptimizer(config=SimplePromptOptimizerConfig()) result = sampler)) # Get the optimized agent and score best = result.optimized_agents[0] best_agent = best.optimized_agent print(f"Score: {best.overall_score}") ``` 🏗 Building it into production ・Run optimization periodically as evaluation datasets evolve, keeping prompts continuously refined ・Use `SimplePromptOptimizer` for quick iterations and `GEPARootAgentPromptOptimizer` for thorough exploration ・Compare pre- and post-optimization scores and only update prompts when improvement is confirmed ・Build evaluation datasets from real user interactions for practical optimization 💡 Use cases 📈 Automatically improve agent prompts based on evaluation scores 🔬 Systematically compare multiple prompt candidates and select the best one 🔄 Re-optimize prompts after model changes or API specification updates 🎯 Fine-tune prompts for specific tasks like code generation, summarization, or classification ⚠️ Watch out `GEPARootAgentPromptOptimizer` only optimizes the root agent's prompt. It does not cover sub-agent prompts, so optimize those separately if needed. Also, optimization requires multiple evaluation runs, so keep an eye on LLM API call costs. ✨ With optimization, you can move from trial-and-error prompt tuning to data-driven automation. Set up regular optimization cycles to continuously raise your agent's quality bar. #ADK# #AIAgent#
Show more
TL;DR: Keep working on long tasks in the background while still answering the user right away. LangChain proposes async subagents for this, since in the standard ReAct loop a single long tool call blocks everything else. Agents Are Going Async (LangChain) Key points ・🧠 Agents need to be responsive (react immediately) and productive (advance many long tasks at once) ・🚧 In a ReAct loop, one long tool call makes every other task wait ・🔀 The design is one orchestrator that stays with the user, plus background workers ・🛠️ Five management tools: start, check, update, cancel, and list async_task ・♻️ Follow-ups use interrupt (default, for corrections) or enqueue (for additions) ・🔔 Results are picked up before the next model call if busy, or wake the orchestrator if idle ・🧪 Available in Deep Agents and Managed Deep Agents, in beta The post has no benchmarks and is a LangChain product write-up, but the design ideas are useful. #AIAgents# #LangChain#
Show more
TL;DR: Uber Eats now offers autonomous robot delivery alongside couriers in select cities. For restaurants, the only change is that the handoff goes to a robot. Robot Deliveries: A New Way to Keep Your Orders Moving Key points ・🤖 Fully electric, autonomous robots travel on sidewalks and avoid obstacles with onboard sensors ・🤝 Partners are Avride, Cartken, Coco, and Serve ・📈 Hundreds of thousands of deliveries across more than 10 cities ・⏱️ The robot waits up to 10 minutes at pickup, then leaves on a direct route once the lid is closed ・🔒 Insulated compartment that only the ordering customer can unlock ・🍕 Fits about six pizzas plus drinks, moving at walking speed (3-5 mph) The scale is notable, but the post gives no city names or measured delivery-time data. Low operational burden for restaurants could be the key to adoption. #UberEats# #DeliveryRobots#
Show more
TL;DR Uber Freight supports search across 40+ microservices and nearly 100 data sources by composing one shared index from independently owned fragments. Title: Freight Search: Composing a Shared Search Index from Independently Owned Fragments URL: Key points ・🧩 Each index field comes from exactly one fragment, so ownership never conflicts ・🧱 Fragments merge into one denormalized document, so queries need no joins ・⚙️ Spark handles bootstrap; Flink with CDC handles live ingestion, with changes searchable in seconds ・🏷 Mappings are generated from Java annotations, so the class is the single source of truth ・🔗 Declare @/LinkedEntity or a fold function and the platform handles parent-child denormalization and 1:N aggregation ・🔄 The SIA-to-OpenSearch migration touched only a few places and applied to every entity ・📈 About 300 fragments across 20+ indices, thousands of queries per second, p99 in hundreds of milliseconds Separating business definitions from platform internals is what let the design survive an engine migration. #Uber# #SearchInfrastructure#
Show more
You are not a king or a hero. You are one of the countless people history forgot. Be born a farmer in Edo, a citizen of Rome, a Sengoku foot soldier. Live a whole life in about an hour, then get graded F to A. Free, no login:
Show more
Add one new skill, and a different skill that used to be picked correctly stops being selected. That is what can happen as an agent's skill library grows 🧩 The usual fixes are retraining the router or collecting labeled routing errors for every skill. This paper shows that you can leave the router and the existing skills alone and rewrite only the incoming skill's name and description, once, at insertion. The authors call the failure routing interference: forgetting that happens at access time, not in model weights. 🔧 Skill++ rewrites the description so it stands apart from similar skills, adds a "does not cover" scope exclusion, and qualifies the name if it clashes. No training and no labels are needed. As the library grows from 27 to 731 skills, the share of queries where all required skills land in the top 5 rises from 36.4% to 76.3% (Claude Haiku 4.5). Errors caused by substitution fall from 46.0% to 6.5%. Skill++: Towards Lifelong Agents via Routing-Aware Skill Refinement Numbers come from the authors' own benchmark, and the study is text-only. #AIAgents# #LLM#
Show more
Born somewhere in world history. Live one whole life. Die. "Life once upon a TIME" is out: a free browser game. Live an ordinary life in Edo Japan, Imperial Rome, medieval England and more. Step in at up to 3 turning points. No sign-up. #indiegame#
Show more
A schema tells you what exists, not what it means. Neo4j has released an open ontology specification as its answer for AI agents 🧭 The Neo4j Ontology Graph Specification: A pragmatic, open, AI-first approach It is an open-source specification for describing domain knowledge in a single machine-readable JSON or YAML file. It is Apache 2.0 licensed, with SDKs for JVM, JavaScript, and Go. Highlights 🛠️ Pragmatic Define nodes, relationships, properties, constraints, and indexes in one file. It validates against a published JSON Schema and fits existing development and operations workflows. 🔗 Interoperable It builds on W3C standards such as RDFS, OWL, SKOS, and SHACL, and documents how RDFS and SHACL concepts map onto the spec. A reference field on every element can link to external standard terms. 🤖 AI-first Descriptions and aliases tell an agent whether "Account" means a bank account, what else it is called, and which identifier (such as an IBAN) applies. One ontology can serve as constraints, data derivation, and agent context. The idea is to write the meaning once and use it in both the database and the agents. There is no RDF serialization yet, and the post includes no performance evaluation. #Neo4j# #GraphRAG#
Show more