TL;DR Reliable agentic AI comes not from "a better model or prompt" but from explicitly engineering context and the orchestration harness—lessons from PRINCE, a Bayer × Thoughtworks pharma system. 🧪
Title: Building Reliable Agentic AI Systems
URL:
Highlights
🧭 Sequential agents with pause points: clarify intent → Think & Plan → Researcher → Reflection → Writer, verifying step by step
🔁 Three reflection loops: process (trajectory), data (evidence sufficiency), draft (output completeness) catch distinct failures
🔎 Hybrid retrieval: query expansion n=5, weighting 0.7 semantic + 0.3 keyword, bge-reranker cutting ~20 → 7 chunks
🗃️ Structured data via Text-to-SQL: SELECT-only, up to 3 self-corrections, ≤50 rows per query
🛟 Harness engineering: state persisted in PostgreSQL/DynamoDB, resume from failure point, automatic provider fallback
📌 Sentence-level citations plus RAGAS and Langfuse evals; daily batch checks on live traffic catch hallucinations
🏷️ NER extracts entities from study PDFs; high-confidence fields auto-update, low-confidence quarantined for human review
The "even with big context windows, selectivity still matters" stance rings true to anyone shipping this stuff.
#
AIAgents# #
LLMOps#