Build Voice AI agents locally! HuggingFace's speech-to-speech is a fully modular voice agent pipeline that runs entirely on open-source models. Every component is swappable and every stage runs in its own thread connected by queues. The pipeline exposes an OpenAI Realtime-compatible WebSocket API. Any existing OpenAI Realtime client can point at this server instead of OpenAI by changing one URL. Here is how it works: 1. Voice Activity Detection - Silero VAD v5 detects speech boundaries and turn-taking 2. Speech to Text - transcribes the user's turn with optional live partial transcripts 3. Language Model - generates the response, streaming text and tool calls 4. Text to Speech - synthesizes audio and streams it back to the client Every stage has multiple interchangeable backends you can select via CLI flags. For a fully local setup, run Gemma 4 via llama.cpp alongside the pipeline with Parakeet TDT for STT and Qwen3-TTS for speech output. No API keys needed. Key capabilities: • OpenAI Realtime-compatible WebSocket API - drop-in replacement • Fully local with no API keys via llama.cpp + Parakeet TDT + Qwen3-TTS • STT: Parakeet TDT, Whisper, Faster Whisper, Paraformer • TTS: Qwen3-TTS, Kokoro, Pocket TTS, ChatTTS, MMS TTS • Multilingual with automatic language detection • Four run modes: Realtime, Local, Raw WebSocket, TCP Socket 100% open source. I've shared the link in the replies!

Andrew Ng open-sourced an AI coworker! OpenWorker is a desktop app that delivers finished work, not just chat. You describe the outcome you want - a customer brief, a triaged inbox, a calendar update, a release status across Jira and GitHub - and it works across your files, terminal, and connected apps to produce the actual deliverable. The design principle: it reads freely but writes only after you approve. Before anything consequential - sending a message, making a calendar change, running a shell command - it pauses and shows you exactly what it's about to do. Unattended scheduled runs park approval-needed actions in an inbox rather than acting on their own. 25+ integrations out of the box: GitHub, Slack, Jira, Notion, Linear, HubSpot, Gmail, Google Calendar, Outlook, monday, and your terminal and local files. Any MCP tool plugs in too. Works from Slack directly. Mention @ OpenWorker in a channel, the work happens on your desktop with your tools, and the answer comes back as a thread reply. Bring your own model. OpenAI, Anthropic, Google Gemini, DeepSeek, Kimi, Qwen, MiniMax, Mistral, Grok, and fully local via Ollama. Everything runs on your machine with your own API keys. Built on aisuite, Andrew Ng's own unified LLM provider library. Currently in open beta on macOS and Windows. Key capabilities: • Delivers finished deliverables: documents, reports, Slack replies, updated calendars • Approval-gated writes, sends, and shell commands • 25+ integrations including GitHub, Slack, Jira, Notion, Gmail, and MCP tools • Scheduled automations for recurring work • Works from Slack via @ OpenWorker mention • Bring your own model: any provider or fully local via Ollama • Local-first: everything runs on your machine OpenWorker shows what happens when you get the loop right. Define the goal, the agent executes autonomously, approval gates control what runs. That's loop engineering in practice. Wrote a full breakdown on how to design loops you can actually trust to run without you. I've shared the link to the Github Repo in the replies!

Every API call gives you a choice: wait for the result now, or collect it later. Agentic Document Extraction (ADE) works both ways when you run a Parse or Extract. The first way is synchronous. You send the request and wait on the line until the result comes back. Simple, and right when someone needs the answer immediately. The second is asynchronous. You submit the work, get a ticket, and pick up the result once it is done. Nothing sits waiting on an open line. The result is identical either way. Same Markdown from Parse, same fields from Extract. You are only choosing how long you wait. So why ever hand it off? Two reasons. Big jobs need it. A single asynchronous Parse Job takes files up to 1 GB or 6,000 pages, far more than one live request can hold. It also costs less. Waiting on the line runs at the top rate, since something is blocked on it. Hand the job off and you drop to a cheaper tier for the same result. The rule of thumb is one question. If a person or an agent is waiting on the answer, run it synchronously. If nothing is waiting, submit a job and pay less. Full breakdown, plus the service tier companion, in the comments.

Stop optimizing tokens. Index your context instead! Retrieval quality is the foundation of context engineering. Karpathy described it best: the heavy cognitive work should happen at ingestion, not at query time. When knowledge is properly structured before retrieval, the model's job becomes reasoning, not sorting. Most AI systems focus on compressing what the model sees. The more important problem is what gets retrieved before the model sees anything. Token efficiency starts at retrieval. When context is properly indexed and prepared, the model spends its tokens on reasoning. When retrieval is weak, the model spends those same tokens sorting through noise and filling gaps from its own weights. That's the silent failure mode. Retrieval returns topically correct but incomplete context. The model completes the gaps from parametric knowledge and streams it out the same way as grounded content. No signal in the output tells you which parts came from retrieved context and which came from weights. The answer looks confident. It just isn't complete. The common assumption is that hallucination is the main failure mode. It's not. Models handle off-domain questions reasonably well now. If nothing in the retrieved context looks relevant, there's no material to build an answer on. The harder failure is partial coverage. The right document was retrieved. But not the full picture. Coverage gaps don't produce error messages. They produce confident answers with pieces missing. This gets worse when sources stay isolated. The same person might appear across multiple tools and systems. If those sources are indexed separately, the model has to figure out they refer to the same entity on its own. That's work that should happen before the model starts reasoning. Glean's system of context is built around this problem: • Unified index across all connected applications, not each source kept separate • Specialized indexes for different types of information: company data, code, experts, profiles, tools, and calendars • Multiple retrieval methods - semantic when meaning matters, lexical when exact terms matter, structured when fields and relationships need to stay intact • Enterprise Graph that maps relationships across people, teams, customers, and projects so relevance reflects how the company actually works • Memory that carries context forward across sessions and tasks • Tools that let the model act on what it finds The gap between finding information and understanding it is where most AI systems fall short. I've shared the link in the replies!

Open-source framework for building real-time voice AI agents! Pipecat is a Python framework for orchestrating audio, video, AI services, transports, and conversation pipelines. Voice-first architecture with pluggable components. What you can build: voice assistants, AI companions, multimodal interfaces, interactive storytelling, business agents (customer support, intake), and complex dialog systems. The framework handles speech recognition, text-to-speech, conversation logic, and real-time interaction. WebRTC and WebSocket transport built in. Ultra-low latency for natural conversations. Why Pipecat: • Voice-first: Integrates STT, TTS, and conversation handling in one framework • Pluggable: Supports multiple AI service providers for each capability • Composable pipelines: Build complex behavior from modular components • Real-time: Low-latency interaction with streaming audio/video Supported services: • Speech-to-Text: Deepgram, AssemblyAI, OpenAI Whisper, Groq, Azure, AWS, Google, and more • LLMs: OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama, AWS, Azure, and more • Text-to-Speech: OpenAI, ElevenLabs, Deepgram, Cartesia, Azure, AWS, Google, and more • Speech-to-Speech: OpenAI Realtime, Gemini Multimodal Live, AWS Nova Sonic, Ultravox, Grok Voice Agent I've wrote a detailed tutorial on building a production customer support voice agent recently - covering turn detection, interruption handling, telephony codecs, and how to inject live business context into every call. I've quoted the article!

The model humans prefer the most is not always the most accurate one! Arena just added factuality scoring to their leaderboard alongside human preference. The idea: human preference measures whether a response felt good to read. Factuality measures the correctness of claims in a model's response. These are two different things. Most benchmarks test models on fixed, predetermined questions in a lab setting. What Arena is doing differently: measuring factuality in real, open-ended user conversations at scale. Over 2 million claims extracted from actual conversations, verified against the web, across 130k Text Arena battles and 40k Search Arena battles. The way it works: they randomly sample battles, extract web-verifiable claims from each model response, verify them against the web, and calculate a factuality score per model. The final ranking is a weighted combination of human preference and factuality score. When factuality weight is enabled, the shifts are significant. In Text Arena, GPT-5.5 jumps 13 spots. Muse Spark drops 13. Claude Fable 5 moves down slightly. In Search Arena, GPT-5.5-search takes the top spot while Gemini grounding drops from 7 to 13. I've shared the full methodology blog in the replies!

Pytest for Google ADK agents! Testing AI agents is fundamentally different from testing regular code. There's no single output to assert against. Every run can produce a different response. And when something goes wrong, it's rarely obvious which step caused it. AI agents built using Google ADK run across multiple steps. A model call plans the action. A tool executes it. Another model call generates the final response. Any of these can quietly go wrong without an obvious error. DeepEval is an open-source LLM evaluation framework, for evaluating large-language model systems. DeepEval's Google ADK integration brings Pytest to this problem. One call to "instrument_google_adk()" and every agent run is automatically traced. Every model call, tool invocation, and agent step becomes a component span you can evaluate independently. Testing works at two levels. End-to-end evaluation scores the full agent run on task completion. Component-level evaluation attaches metrics to individual LLM calls or tool spans, so you know exactly which step failed when a test breaks. The pytest integration works the same way as any other DeepEval framework. Parametrize your test with Goldens, run the agent inside the test function, call "assert_test()" at the end. A failing metric fails the test, which fails the build. Run it with "deepeval test run". You can also run evals outside CI. Loop through Goldens in a script, run the agent, and score each resulting trace without touching pytest at all. Key capabilities: • Auto-instrumentation with one function call • Trace, agent, LLM, and tool spans all independently evaluable • Native pytest integration with assert_test() for CI/CD • Metrics: TaskCompletion, AnswerRelevancy, Faithfulness, G-Eval and more • Script-based eval outside of CI • Optional Confident AI dashboard for trace visualization 100% open source. I've shared the link in the replies!

3x Model：GPT-5.6 Sol、Terra、Luna 5x Effort：Light、Medium、High、Extra High、Ultra 2x Speed：Standard、Fast ChatGPT Work and Codex 里，有 30 种不同的搭配方式！！！怎么选呢？总不能每个任务都把模型和强度、速度都拉满吧，额度也不够用啊？！ 看看 OpenAI Developer 最新发布的 GPT-5.6 最佳实践和迁移指南，从官方指导找一些确定性规则： 先看 GPT‑5.6 模型的三个等级 · gpt-5.6-sol：旗舰能力 · gpt-5.6-terra：能力与价格之间的平衡 · gpt-5.6-luna：面向高吞吐、效率优先的工作负载 对于推理强度 如果你之前用 GPT-5.5 或 5.4 已经有常用的推理强度，在使用 GPT-5.6 时，可以尝试： · GPT‑5.6 与相同推理强度 · GPT‑5.6 与低一档推理强度 原因是 GPT‑5.6 可能用更少 token 达到相同或更好的效果。但这只是需要验证的可能性，并非保证。medium 是标准和 Pro 模式的默认平衡点；high、xhigh 和 ultra/max 只应在真实评测显示质量提升时使用。 “更短的提示词”不是“少给必要信息” OpenAI 报告，在其内部评测中，将冗长系统提示词缩减为更精炼的版本，分数提高约 10%–15%，总 token 减少 41%–66%，成本减少 33%–67%。但这是厂商内部评测，没有披露完整任务构成、样本规模和评测方法，因此不应直接外推到所有业务。 应该保留的是：目标、必要背景、关键约束、授权边界、所需证据、成功标准和输出格式。 要处理长期堆积的提示词债务，例如： · 重复表达同一限制； · 大量已经成为模型默认行为的指令； · 不必要的角色设定和风格示例； · 与当前任务无关的工具； · 冗长但不精确的工具描述。 模型越主动，权限边界越应清楚 GPT‑5.6 更擅长推断用户意图，也更主动、持续。但“更懂意图”并不等于可以省略关键边界。 文档建议明确区分： · 回答、解释、审查、诊断：可以检查材料并报告结果； · 修改、构建、修复：可以执行范围内的本地改动和非破坏性验证； · 外部写入、删除、购买、发布或明显扩大范围：必须确认。 深层含义是：授权规则已经不只是提示词风格，而是智能体系统的安全接口。 不必在每段提示词里反复写“先询问”，但需要一次性把可自主动作和确认边界定义清楚。

There's a fundamental gap in every AI agent stack today! Agents can plan, reason, and call APIs. But the moment they need a tool that requires an account, a subscription, or an API key, autonomy breaks. A human has to step in. That's the gap x402 closes. HTTP 402 "Payment Required" was reserved in the original HTTP spec and never formally used. Coinbase formalized it as a protocol for agentic payments. When an agent hits an endpoint and gets a 402 back, it authorizes a payment in USDC on Base from its wallet, and the tool runs. No account, no API key, no human in the loop. Apify Actors are serverless programs built for web scraping, automation, and data extraction. Roughly 20,000 of them are now callable via x402 today. An agent can discover the right tool, pay for it per call, and keep going entirely on its own. Most x402 services today are single purpose-built endpoints. This opens an entire catalog of community-built web automation tools to any autonomous agent. Key capabilities: • Agents discover, pay for, and run tools autonomously • No account, subscription, or API key required • Payment in USDC on Base via x402 by Coinbase • ~20,000 tools callable via x402 today • Accessible via MCP server or "npx awal" CLI I've shared the link in the replies!

Google DeepMind just dropped the Gemma 4 Technical Report! Gemma 4 is a family of open-weight, natively multimodal models ranging from 2.3B to 31B parameters, covering both dense and Mixture-of-Experts architectures. The most interesting architectural decision is in the 12B model. Instead of separate vision and audio encoders, it uses a unified encoder-free design. The 550M vision encoder is replaced by a single matrix multiplication at 35M parameters. The audio encoder is discarded entirely. Raw image patches and audio chunks are projected directly into the LLM embedding space. No separate encoder pipelines, no memory fragmentation. The efficiency numbers are compelling. A 2.3B effective parameter model (E2B) roughly matches Gemma 3 27B performance with 10x fewer parameters. The quantized 12B fits in 7.65 GB. KV cache footprint is reduced by up to 37.5% through local-to-global attention ratios, positional encoding, and key-value reuse in global attention layers. The audio encoder shrank from 390MB to 87MB after quantization, a 78% reduction, with improved transcription and translation performance. Gemma 4 31B sits at rank 43 on Arena Text as of June 2026, the top open dense model on the leaderboard. Models ranked above it are all MoE architectures with significantly more total parameters. All models ship with a thinking mode that generates a reasoning trace before responding. Benchmark results in thinking mode for the 31B: AIME 2026 at 89.2%, GPQA Diamond at 84.3%, LiveCodeBench v6 at 80.0%, and a Codeforces Elo of 2150. I've shared the link to the report the replies! Also put together a complete guide on fine-tuning Gemma 4 locally with Unsloth. I've quoted the article!

The meta-harness for all your AI coding agents! Omnigent is an open-source orchestration layer that sits above your AI coding agents. Orchestrates Claude Code, Codex, Cursor, OpenCode, Hermes, and Pi through a single interface. Swap or combine harnesses without rewriting anything. The problem with using multiple coding agents: each lives in its own terminal, has its own setup, its own commands. When you want to switch or combine them, you start over. Omnigent removes that. One layer above all of them, consistent interface regardless of which agent is running underneath. Sessions follow you across devices. Start in your terminal, continue in the browser, pick it up on your phone. Messages, sub-agents, terminals, and files stay in sync across every surface. Multi-agent orchestration works within a single session. Point one agent at another's work. Delegate tasks in parallel across Claude Code, Codex, and Pi running in separate git worktrees, then route the diffs to a reviewer from a different vendor. All from one session. Policy enforcement runs at three levels: server, agent, and session. Approve before shell commands execute, cap spend, limit tool calls. Stacks consistently across every harness you plug in. Custom agents via YAML. Define your own with tools, MCP servers, and sub-agents. Two ship out of the box: Polly, a multi-agent coding orchestrator, and Debby, a dual-head Claude and GPT brainstorming agent. Key capabilities: • Orchestrates Claude Code, Codex, Cursor, OpenCode, Hermes, Pi, and custom agents • Sessions follow you across terminal, browser, and phone • Multi-agent collaboration within a single session • Policy enforcement at server, agent, and session level • Cloud sandboxes via Modal, Daytona, E2B, and Kubernetes • Custom agents defined in YAML with tools and MCP servers • Real-time team collaboration and session sharing 100% open source. I've shared the link in the replies! Been going deep on what the harness layer actually needs to look like for agents to work independently in production - identity, memory, proactivity, accountability, and the context layer that ties it all together. Wrote a detailed breakdown on this recently. I've quoted the article!

5 Open Source No-Code LLM, RAG and AI Agent Builders! 1. AutoAgent - A fully automated, zero-code framework. You simply state a high-level goal in natural language, and it handles the planning, task decomposition, and execution automatically. It effectively turns a prompt into a running agent system. 👉 Github Repo: 2. AnythingLLM The best all-in-one solution for internal tooling. It combines RAG, agent workflows, and document management into a single, self-hosted workspace. It is privacy-focused and designed for both technical and non-technical teams to build tools around their own data. 👉 Github Repo: 3. LangChain Open Agent Platform A specialized UI built on top of LangGraph. Instead of hiding the logic, it makes the agent's flow explicit with nodes and edges. This gives you granular control over routing, loops, and multi-agent coordination without needing to write the underlying graph code. 👉 Github Repo: 4. Sim A visual workflow builder with an AI Copilot. You design agent pipelines as executable graphs, but you can use the built-in AI to generate or modify the flows for you. It includes detailed execution tracing, making it much easier to debug complex chains. 👉 Github Repo: 5. Dify A production-ready platform that focuses on observability. It supports prompt management, complex RAG pipelines, and agent logic, but adds the runtime monitoring you need for real applications. If you are deploying to actual users, this is the standard. 👉 Github Repo: If you're going deeper on how these agents actually work in production, I wrote a detailed breakdown on independent agents recently - covering identity, memory, proactivity, accountability, and the context layer that ties it all together. I've quoted the article below

Build and Train your own Diffusion Language Models! dllm is an open-source library that lets you build, train, and evaluate diffusion-based language models without setting up complex pipelines or writing custom training loops. Most language models today are autoregressive. They generate token by token, which makes training and inference fast but also leads to problems like exposure bias and difficulty maintaining global coherence. Diffusion language models flip this approach. Instead of predicting the next token, they denoise corrupted text over multiple steps until the full output is reconstructed. This gives them better global reasoning, fewer cascading errors, and stronger robustness on long-form tasks. The challenge has always been tooling. Diffusion LMs require more complex training loops, noise schedules, and evaluation setups than typical transformers. dllm solves that part. It gives you a structured, reproducible pipeline for training and evaluating diffusion LMs without writing custom scaffolding. Key Features: • Full training workflow for diffusion LMs using clean configs • Support for LoRA, DeepSpeed, and FSDP for scaling and efficiency • Modular model components so you can test new diffusion architectures • Simple dataset loading and experiment management • Built-in evaluation utilities for comparing runs and ablations Github Repo →

Fine-tuning massive LLMs used to be painfully slow, but not anymore! 4 open source libraries that accelerate fine-tuning of Large Language Models 1. Unsloth AI • Fine-tune models like Qwen3, Llama 4, and Gemma 3 up to 2× faster with 70% less VRAM • Uses optimized Triton kernels and manual backprop for exact accuracy • Supports low-resource setups and runs on consumer GPUs or even Colab/Kaggle with ~3 GB VRAM GitHub repo → 2. LLaMA Factory • Fine-tune over 100 models (LLaMA, Mistral, Gemma, etc.) using a simple CLI or WebUI • Supports LoRA, QLoRA, full or frozen fine-tuning across 2–8‑bit precision • Includes built-in dataset templates, training monitors, and model export options GitHub repo → 3. DeepSpeed • Built for large-scale distributed fine-tuning with ZeRO and FSDP • Optimized for multi-GPU and multi-node training with advanced memory management • Trusted in production environments for scalable LLM training GitHub repo → 4. Axolotl • Yaml-based setup for fine-tuning, LoRA/QLoRA, DPO, GRPO, and multimodal workflows • Includes kernel optimizations for memory-efficient training • Actively maintained with support for Hugging Face, model export, and inference GitHub repo →

The hard part of RAG isn't finding the right chunk! When you chunk a document for RAG, each chunk lands in the index on its own, disconnected from the section it came from. So two chunks with identical text, but from completely different parts of the document, look exactly the same to a flat index. That becomes a problem the moment a question needs context from more than one section. The chunks come back, but how they relate to each other gets lost. ADE Section fixes this. It reads the parsed document, builds the actual hierarchy, and figures out where every chunk falls within it. That gets attached to the chunk before it's embedded. Once that's in place, a broad question can stay at the section level. A specific one can drop into a sub-chunk. You can scope a search to one part of a document instead of the whole thing. Citations get more accurate too. The model knows exactly which section a fact came from. Run ADE Parse, then run ADE Section.