The OCI Agent Evaluation Framework goes beyond final-answer scoring to evaluate prompts, RAG, tools, trajectories, state changes, and production behavior across the agent lifecycle. Learn more:

📚 Wouldn't it be great to run ReAct, RAG, and Tree of Thoughts behind one API and compare them, instead of wrangling scattered per-paper implementations? This repo delivers exactly that, with all 35 patterns in one place. Title: FareedKhan-dev/all-agentic-architectures URL: 📦 Overview This is a Python library and a "living textbook" implementing 35 production-grade agentic AI patterns. Every architecture exposes the same .run(task) method and returns an identical result shape, so you can swap patterns without touching downstream code. ❓ Challenges Solved Agentic design patterns have been scattered across papers, each with its own implementation and conventions. The real value here is unifying them under a single interface so you can try them side by side. 💡 Core Idea & Approach The central idea is the "deterministic-picker discipline." ・Instead of handing scoring entirely to the LLM, it first has the LLM commit to categorical features like booleans and enums ・The final decision is then composed in Python logic This mitigates the flat-band pathology of LLM-as-Scorer, and it appears in 13 of the 35 architectures. 🎯 Coverage & Use Cases It spans eight families: reasoning and reflection (Reflection, Self-Discover), search (Tree of Thoughts, LATS), RAG (Corrective/Self/Adaptive/GraphRAG), memory (MemGPT, Voyager), tools and actions (ReAct, SWE-Agent), and multi-agent (Debate, STORM). Each pattern ships with an executed Jupyter notebook, giving reproducible references grounded in real LLM output. 📊 Highlights ・Built on LangGraph, with support for Nebius, OpenAI, Anthropic, Ollama and more, switchable via a single env var ・283 passing pytest tests ・On a 17-task benchmark it recently scored 33/42 correct (78%), with Reflection and Self-Consistency among the strongest #AIAgents# #LangGraph#

RAG's "more precision means more latency" dilemma, tackled by using topics as a compass 🧭 Title: MCompassRAG: Topic Metadata as a Semantic Compass for Paragraph-Level Retrieval URL: 🧭 Overview A metadata-guided retrieval framework that uses topic-level signals as a "semantic compass" to select relevant evidence at the paragraph level. It aims to improve precision and efficiency at the same time. ❓ Challenges Solved RAG faces a precision-vs-efficiency trade-off. ・Fine-grained chunks raise precision but increase candidates, latency, and cost ・Larger chunks reduce candidates but introduce semantic noise from mixed topics This is acute in deep-research tasks needing fast, precise retrieval over large datasets. 💡 Methodology & Proposed Approach ・It enriches chunk representations with topic metadata within the same embedding space ・It uses LLM-teacher distillation to train a lightweight retriever ・This enables topic-aware retrieval with no additional LLM calls at inference time The core is combining metadata with dense embeddings and distilling into a lightweight retriever. 📊 Experimental Results ・Information efficiency: 8.24% average improvement across six benchmarks ・Latency: over 5x lower than the strongest efficient RAG baselines ・Code is available in a public repository It achieves this precision-and-speed balance without extra LLM calls at inference. #RAG# #Retrieval#

Great RAG paper from IBM. There are some really good ideas on how to solve common RAG issues. It's well known that retrievers chunk long documents by length, which discards the hierarchy the document already has. So they propose using a table of contents. A table of contents helps to encodes exactly the global structure that chunking throws away. STAIR uses that table of contents as the addressing scheme for a generative retriever, so the model stores and retrieves information from its own parameters against a structure the corpus supplies. On SearchTome, it reaches Recall@1 of 82.6 percent against 76.9 percent for a fine-tuned Differentiable Search Index, a statistically significant gap, with BM25 at 59.5 percent and DPR at 68.7 percent. Hallucination stays below 0.05 percent, which is the standing objection to generative retrieval and the reason grounding the address space in a real hierarchy is worth the extra structure. The ablations also show it generalizes where very few training samples exist. Paper:

Jev for RAG in almost all cases you trust the semantic matching capability of Jev more than dot product similarity very useful as the direct similarity metric in small data cases and a great reranker with big data

The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. 2️⃣ A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within @llama_index to help any agent do two-pass document processing with higher accuracy and lower cost. 1️⃣ We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types 2️⃣ We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses to achieve SOTA in accuracy and cost across various doc parsing and extraction tasks. It can be called from any agent harness as an MCP or skill. It takes in page numbers as input, so that the agent can choose to run LlamaParse over a subset of the doc instead of the full doc as a “zoom-in” pass. Come check it out! LiteParse: LlamaParse: All the relevant docs, including MCP, are here: