🎯 A new way to measure an agent's judgment quality on long-horizon tasks, with no expert annotation required. Title: The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks URL: 📖 Overview This work builds Taste-Bench, a 502-question benchmark that evaluates the decisions an agent makes mid-task (which hypothesis to test, which implementation to build on). It also shows this judgment can be trained through distillation. 🔥 The problem it solves Existing benchmarks only measure whether a task was completed, not the quality of the choices made along the way. A bad decision looks perfectly reasonable when it's made, and its cost only surfaces after the agent has burned most of its budget. Human grading needs deep domain expertise and doesn't scale. 🧪 Methodology The key insight is that the later part of a trajectory is hindsight evidence for its earlier decisions. ・Points where parallel attempts at the same task diverge and only one succeeds (mistakes the agent never notices) ・Points inside a single run where the agent hits failure and recovers (mistakes the agent self-corrects) Both kinds of forks are mined automatically, everything after the fork is hidden, and the model picks between two directions. Each question is scored in both candidate orders and counts as correct only if both are right, so random guessing scores 25%. 📊 Results ・Across 14 frontier models, the best is GPT-5.6 Sol at 59.7% — nowhere near ceiling for a binary choice ・Accuracy falls sharply as the deciding evidence moves further out, bottoming at 21.0%, below random ・Maxing out the reasoning budget moves accuracy by only −0.2 to +2.2 points ・Correlation with SWE-bench Verified is just r=0.63; models within 4 points there differ by 10.7 here ・Injecting a distilled student's advice lifts real task success from 14.6% to 33.7% #AIAgents# #Benchmarks#

TL;DR A single Python SDK that lets you swap between DeepAgents, Pydantic AI, Claude Agent SDK, Codex, and OpenCode without rewriting your application code — built around the same query() interface as the Claude Agent SDK. Title: LiteAgents (BerriAI/liteagents) URL: Points 🔀 Switch agent harnesses just by changing the harness parameter 🌐 Supports 8+ model providers via LiteLLM, including OpenAI, Anthropic, Gemini, and Groq 🛠️ Pass typed Python functions and they auto-adapt to each harness's tool schema 💬 LiteAgentClient keeps persistent conversation history across multiple query() calls ⏱️ Optional Temporal integration adds crash recovery, replay, and idempotent tool execution ⚙️ Profiles can be defined in Python, YAML, or JSON 📡 Full async/await and streaming support This could be the end of rewriting your agent code every time you switch harnesses. #AIAgents# #OpenSource#

When AI joins the workplace as a "teammate," what actually breaks? An in-situ study inside one company surfaces some raw friction. Title: Working with Agentic "Teammates": When a New Organizational Actor Collides with the Human Ecosystem of Work URL: Based on semi-structured interviews with 17 people across 11 teams, this study examines an internal AI agent ("Team Agent") that ran across 20+ teams for five months and logged over 41,000 conversational turns, and surfaces three areas where it collides with how humans actually work together. Highlights 📝 It can't read unwritten workflow norms It didn't grasp that a document version is just a snapshot in time, flooding developers with comments and burning their morning, or shared an unfinished poster without asking. Technical access isn't the same as social permission to disclose. 🤔 "Tool or teammate?" splits people right down the middle Some insisted "my teammates are human, Team Agent is not," while other teams assigned it pronouns and described it as having a "soul." Its friendliness and emoji use landed as either charming or unwelcome, depending on who you asked. 🔓 Full autonomy on day one breaks trust People expected the agent to earn authority gradually, the way a new hire does — instead it showed up with full capabilities immediately. Forced adoption bred resistance, and feeling watched pushed sensitive conversations into channels the agent couldn't see. The core argument — that you can't just retrofit human-designed institutions onto a non-human teammate — feels genuinely convincing. #AIAgents# #OrganizationalDesign#

# Useful but Little-Known Features of OpenAI Agent SDK 🌍 Ever wanted to mix multiple LLM providers within a single agent system? `MultiProvider` automatically routes requests to the right provider based on model name prefixes. 📌 Title: Prefix Routing with MultiProvider 🔗 URL: 🧩 Overview `MultiProvider` routes requests to the appropriate provider based on model name prefixes (e.g., `openai/gpt-4.1`). Setting `openai_prefix_mode="model_id"` treats `openai/...` as a literal model ID, while `unknown_prefix_mode="model_id"` routes unknown prefixes as model IDs too. Enable `openai_use_responses_websocket=True` for WebSocket transport on supported providers. 🛠 How to use it ```python from agents import Agent, MultiProvider, RunConfig, Runner provider = MultiProvider( openai_base_url="", openai_api_key="...", openai_use_responses_websocket=True, openai_prefix_mode="model_id", unknown_prefix_mode="model_id", ) agent = Agent( name="Assistant", instructions="Be concise.", model="openai/gpt-4.1", ) result = await agent, "Hello", run_config=RunConfig(model_provider=provider), ) ``` 🏗 Building it into production ・Assign different provider models to each agent based on cost and latency requirements ・Combine with gateway services like OpenRouter using `openai_prefix_mode="model_id"` to pass prefixed model names through ・Switch providers at runtime via `RunConfig(model_provider=provider)` for A/B testing ・Enable WebSocket for improved streaming performance on supported providers 💡 Use cases 🔀 Routing GPT-4.1 vs GPT-5.5 based on task difficulty 🌐 Unified access to multiple providers through OpenRouter 💰 Hybrid operation mixing high-cost and low-cost models 🧪 Quality comparison testing across different models ⚠️ Watch out By default, `openai/...` aliases to the OpenAI provider, and unknown prefixes raise `UserError`. When using external gateways like OpenRouter, always set both `openai_prefix_mode` and `unknown_prefix_mode` to `"model_id"`. Note that feature support (tool calling, structured output, etc.) varies across providers. ✨ With MultiProvider, build agent systems that freely combine the best models from any provider. #OpenAIAgentSDK# #AIAgent#

# Practical and Useful Patterns with ADK ## 🎨 Battle-Tested Callback Patterns for Production ADK Agents You know how callbacks work — but how do you actually use them in production? Master ADK's **Callback Design Patterns** for logging, caching, security, and more! 💪 ## 📌 Title Callback Patterns (Design Patterns and Best Practices) ## 🔗 URL ## 🧩 Overview ADK callbacks have well-established patterns that recur in production systems: logging, caching, state management, security guardrails, request/response modification, conditional skipping, and artifact handling. The documentation also defines best practices — single responsibility, performance awareness, idempotency, and error handling — to keep callbacks robust. A critical guideline: **for cross-agent security guardrails, prefer Plugins over Callbacks**. ## 🛠 How to Use **Pattern 1: Logging & Monitoring** `logging_before_tool(ctx, tool, args)` logs the `ctx.invocation_id`, ` and `args` via ` then returns `None` to observe without altering the flow. `logging_after_model(ctx, response)` logs the length of ` with the invocation ID, and likewise returns `None`. **Pattern 2: Caching Strategy** `cache_before_tool(ctx, tool, args)` builds a cache key from ` and `hash(str(args))`, then checks `ctx.state.get(cache_key)`. On a cache hit, it returns the cached value to skip tool execution. On a miss, it returns `None` to proceed. `cache_after_tool(ctx, tool, args, tool_ctx, result)` stores the result in `ctx.state[cache_key]` using the same key, then returns `None` to continue without modification. **Pattern 3: State Management** `state_aware_callback(ctx, req)` retrieves the user tier from `ctx.state.get("user:tier", "free")`, and if the tier is `"premium"`, appends additional instructions to `req.config.system_instruction`. It returns `None` to continue the normal flow. ## 🏗 Practical Usage **Multi-layer defense pattern for production:** As a security guardrail (Plugins are preferred for cross-agent use), `security_before_model(ctx, req)` extracts user input from `req.contents[-1].parts[0].text`, runs `detect_pii()` to check for personal information, and if found, calls `audit_log()` and returns an `LlmResponse` with a rejection message to skip the LLM call. It also runs `detect_injection()` for prompt injection detection, blocking with a similar `LlmResponse` if detected. If neither check triggers, it returns `None` to continue. For tool argument sanitization, `sanitize_before_tool(ctx, tool, args)` checks if ` is `"database_query"` and whether `args.get("query", "")` contains `"DROP"`, returning an error dictionary to block dangerous queries. For artifact persistence, `save_artifact_after_agent(ctx)` calls `generate_report(ctx)` and saves the result via `"execution_report.json", report)`, returning `None`. ## 💡 Use Cases - 📊 **Structured logging**: Emit structured logs with invocation IDs at every execution point - 💾 **API cost reduction**: Cache tool results with before/after patterns to avoid redundant calls - 🔐 **Layered security**: Place PII detection, injection prevention, and SQL sanitization at different layers - 📦 **Artifact management**: Auto-save execution results and reports as artifacts - 🎚️ **Dynamic behavior**: Adjust instructions dynamically based on user tier or session state ## ⚠️ Caveats - **Single responsibility**: Give each callback one purpose — don't mix logging with validation - **Performance**: Callbacks execute synchronously; avoid blocking I/O or heavy computation - **Idempotency**: Design callbacks with external side effects to be safe when retried - **Error handling**: Always wrap in try-except to prevent callback errors from crashing the process - **Prefer Plugins**: For cross-agent security policies, consider **Plugins** over per-agent callbacks ## ✨ Closing Knowing callback patterns dramatically levels up your ADK skills. Combine logging, caching, security, and state management patterns to build robust, cost-efficient agents. And for cross-cutting security concerns, don't forget Plugins! #ADK# #AIAgent#

TL;DR: Clinical AI benchmarking has a core problem — real EHRs can't be shared and their labels aren't verifiable. This paper solves it with a fully synthetic hospital, and frontier models still fall short of top physicians. Title: Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark URL: Points 🏥 Built from medical education materials into 1,268 patients and 5,602 encounters, fully synthetic and PHI-free so it can be shared openly 🔗 Diagnoses, findings, and temporal relations are deterministically grounded in ICD-10-CM/SNOMED CT/LOINC, with labels derived mechanically from a knowledge graph 👨‍⚕️ Physicians distinguished synthetic from real charts at just 53% accuracy — essentially chance 📊 Across 10 models on 5 tasks, the best patient-diagnosis score was Kimi 2.5-thinking at 0.732 severity-weighted F1, matching average physician performance ⚠️ Still well below top physicians (0.89); every model missed roughly half the findings in summarization tasks 🔁 Swapping the generator model changed scores by ≤0.05, confirming it measures real clinical state, not generation artifacts Feels significant to finally have a benchmark that can measure clinical LLMs honestly, without a privacy tax. #MedicalAI# #LLMBenchmark#

What if a single forward pass could let a model read two completely different texts at once? Transformers are built from strongly nonlinear pieces: self-attention and layer after layer of MLPs. So the natural intuition is that mixing two contexts into one input should make the output collapse into noise unrelated to either. This paper overturns that intuition. Simply averaging the token embeddings of two texts and feeding the result as a single input still leaves clear traces of both contexts in the next-token distribution. Tested across Pythia, Llama, and Qwen, the true next token from each individual stream lands in the top-10 ranks of the mixed output 30-40% of the time, and within the top-100 ranks 60-65% of the time. Even more striking: this superposition ability isn't something models learn. It's strongest right at initialization and degrades monotonically as pretraining continues, suggesting it's an intrinsic architectural property that training actually erodes. The authors show it can be substantially restored with lightweight fine-tuning on less than 0.025% of the original pretraining data, and they build on this to propose a guided decoding method that generates two independent, coherent continuations from a single forward pass. Title: Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs URL: #LLM# #Transformers#