Build Voice AI agents locally! HuggingFace's speech-to-speech is a fully modular voice agent pipeline that runs entirely on open-source models. Every component is swappable and every stage runs in its own thread connected by queues. The pipeline exposes an OpenAI Realtime-compatible WebSocket API. Any existing OpenAI Realtime client can point at this server instead of OpenAI by changing one URL. Here is how it works: 1. Voice Activity Detection - Silero VAD v5 detects speech boundaries and turn-taking 2. Speech to Text - transcribes the user's turn with optional live partial transcripts 3. Language Model - generates the response, streaming text and tool calls 4. Text to Speech - synthesizes audio and streams it back to the client Every stage has multiple interchangeable backends you can select via CLI flags. For a fully local setup, run Gemma 4 via llama.cpp alongside the pipeline with Parakeet TDT for STT and Qwen3-TTS for speech output. No API keys needed. Key capabilities: • OpenAI Realtime-compatible WebSocket API - drop-in replacement • Fully local with no API keys via llama.cpp + Parakeet TDT + Qwen3-TTS • STT: Parakeet TDT, Whisper, Faster Whisper, Paraformer • TTS: Qwen3-TTS, Kokoro, Pocket TTS, ChatTTS, MMS TTS • Multilingual with automatic language detection • Four run modes: Realtime, Local, Raw WebSocket, TCP Socket 100% open source. I've shared the link in the replies!

Andrew Ng open-sourced an AI coworker! OpenWorker is a desktop app that delivers finished work, not just chat. You describe the outcome you want - a customer brief, a triaged inbox, a calendar update, a release status across Jira and GitHub - and it works across your files, terminal, and connected apps to produce the actual deliverable. The design principle: it reads freely but writes only after you approve. Before anything consequential - sending a message, making a calendar change, running a shell command - it pauses and shows you exactly what it's about to do. Unattended scheduled runs park approval-needed actions in an inbox rather than acting on their own. 25+ integrations out of the box: GitHub, Slack, Jira, Notion, Linear, HubSpot, Gmail, Google Calendar, Outlook, monday, and your terminal and local files. Any MCP tool plugs in too. Works from Slack directly. Mention @ OpenWorker in a channel, the work happens on your desktop with your tools, and the answer comes back as a thread reply. Bring your own model. OpenAI, Anthropic, Google Gemini, DeepSeek, Kimi, Qwen, MiniMax, Mistral, Grok, and fully local via Ollama. Everything runs on your machine with your own API keys. Built on aisuite, Andrew Ng's own unified LLM provider library. Currently in open beta on macOS and Windows. Key capabilities: • Delivers finished deliverables: documents, reports, Slack replies, updated calendars • Approval-gated writes, sends, and shell commands • 25+ integrations including GitHub, Slack, Jira, Notion, Gmail, and MCP tools • Scheduled automations for recurring work • Works from Slack via @ OpenWorker mention • Bring your own model: any provider or fully local via Ollama • Local-first: everything runs on your machine OpenWorker shows what happens when you get the loop right. Define the goal, the agent executes autonomously, approval gates control what runs. That's loop engineering in practice. Wrote a full breakdown on how to design loops you can actually trust to run without you. I've shared the link to the Github Repo in the replies!

Stop optimizing tokens. Index your context instead! Retrieval quality is the foundation of context engineering. Karpathy described it best: the heavy cognitive work should happen at ingestion, not at query time. When knowledge is properly structured before retrieval, the model's job becomes reasoning, not sorting. Most AI systems focus on compressing what the model sees. The more important problem is what gets retrieved before the model sees anything. Token efficiency starts at retrieval. When context is properly indexed and prepared, the model spends its tokens on reasoning. When retrieval is weak, the model spends those same tokens sorting through noise and filling gaps from its own weights. That's the silent failure mode. Retrieval returns topically correct but incomplete context. The model completes the gaps from parametric knowledge and streams it out the same way as grounded content. No signal in the output tells you which parts came from retrieved context and which came from weights. The answer looks confident. It just isn't complete. The common assumption is that hallucination is the main failure mode. It's not. Models handle off-domain questions reasonably well now. If nothing in the retrieved context looks relevant, there's no material to build an answer on. The harder failure is partial coverage. The right document was retrieved. But not the full picture. Coverage gaps don't produce error messages. They produce confident answers with pieces missing. This gets worse when sources stay isolated. The same person might appear across multiple tools and systems. If those sources are indexed separately, the model has to figure out they refer to the same entity on its own. That's work that should happen before the model starts reasoning. Glean's system of context is built around this problem: • Unified index across all connected applications, not each source kept separate • Specialized indexes for different types of information: company data, code, experts, profiles, tools, and calendars • Multiple retrieval methods - semantic when meaning matters, lexical when exact terms matter, structured when fields and relationships need to stay intact • Enterprise Graph that maps relationships across people, teams, customers, and projects so relevance reflects how the company actually works • Memory that carries context forward across sessions and tasks • Tools that let the model act on what it finds The gap between finding information and understanding it is where most AI systems fall short. I've shared the link in the replies!

Open-source framework for building real-time voice AI agents! Pipecat is a Python framework for orchestrating audio, video, AI services, transports, and conversation pipelines. Voice-first architecture with pluggable components. What you can build: voice assistants, AI companions, multimodal interfaces, interactive storytelling, business agents (customer support, intake), and complex dialog systems. The framework handles speech recognition, text-to-speech, conversation logic, and real-time interaction. WebRTC and WebSocket transport built in. Ultra-low latency for natural conversations. Why Pipecat: • Voice-first: Integrates STT, TTS, and conversation handling in one framework • Pluggable: Supports multiple AI service providers for each capability • Composable pipelines: Build complex behavior from modular components • Real-time: Low-latency interaction with streaming audio/video Supported services: • Speech-to-Text: Deepgram, AssemblyAI, OpenAI Whisper, Groq, Azure, AWS, Google, and more • LLMs: OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama, AWS, Azure, and more • Text-to-Speech: OpenAI, ElevenLabs, Deepgram, Cartesia, Azure, AWS, Google, and more • Speech-to-Speech: OpenAI Realtime, Gemini Multimodal Live, AWS Nova Sonic, Ultravox, Grok Voice Agent I've wrote a detailed tutorial on building a production customer support voice agent recently - covering turn detection, interruption handling, telephony codecs, and how to inject live business context into every call. I've quoted the article!

The model humans prefer the most is not always the most accurate one! Arena just added factuality scoring to their leaderboard alongside human preference. The idea: human preference measures whether a response felt good to read. Factuality measures the correctness of claims in a model's response. These are two different things. Most benchmarks test models on fixed, predetermined questions in a lab setting. What Arena is doing differently: measuring factuality in real, open-ended user conversations at scale. Over 2 million claims extracted from actual conversations, verified against the web, across 130k Text Arena battles and 40k Search Arena battles. The way it works: they randomly sample battles, extract web-verifiable claims from each model response, verify them against the web, and calculate a factuality score per model. The final ranking is a weighted combination of human preference and factuality score. When factuality weight is enabled, the shifts are significant. In Text Arena, GPT-5.5 jumps 13 spots. Muse Spark drops 13. Claude Fable 5 moves down slightly. In Search Arena, GPT-5.5-search takes the top spot while Gemini grounding drops from 7 to 13. I've shared the full methodology blog in the replies!

Pytest for Google ADK agents! Testing AI agents is fundamentally different from testing regular code. There's no single output to assert against. Every run can produce a different response. And when something goes wrong, it's rarely obvious which step caused it. AI agents built using Google ADK run across multiple steps. A model call plans the action. A tool executes it. Another model call generates the final response. Any of these can quietly go wrong without an obvious error. DeepEval is an open-source LLM evaluation framework, for evaluating large-language model systems. DeepEval's Google ADK integration brings Pytest to this problem. One call to "instrument_google_adk()" and every agent run is automatically traced. Every model call, tool invocation, and agent step becomes a component span you can evaluate independently. Testing works at two levels. End-to-end evaluation scores the full agent run on task completion. Component-level evaluation attaches metrics to individual LLM calls or tool spans, so you know exactly which step failed when a test breaks. The pytest integration works the same way as any other DeepEval framework. Parametrize your test with Goldens, run the agent inside the test function, call "assert_test()" at the end. A failing metric fails the test, which fails the build. Run it with "deepeval test run". You can also run evals outside CI. Loop through Goldens in a script, run the agent, and score each resulting trace without touching pytest at all. Key capabilities: • Auto-instrumentation with one function call • Trace, agent, LLM, and tool spans all independently evaluable • Native pytest integration with assert_test() for CI/CD • Metrics: TaskCompletion, AnswerRelevancy, Faithfulness, G-Eval and more • Script-based eval outside of CI • Optional Confident AI dashboard for trace visualization 100% open source. I've shared the link in the replies!

There's a fundamental gap in every AI agent stack today! Agents can plan, reason, and call APIs. But the moment they need a tool that requires an account, a subscription, or an API key, autonomy breaks. A human has to step in. That's the gap x402 closes. HTTP 402 "Payment Required" was reserved in the original HTTP spec and never formally used. Coinbase formalized it as a protocol for agentic payments. When an agent hits an endpoint and gets a 402 back, it authorizes a payment in USDC on Base from its wallet, and the tool runs. No account, no API key, no human in the loop. Apify Actors are serverless programs built for web scraping, automation, and data extraction. Roughly 20,000 of them are now callable via x402 today. An agent can discover the right tool, pay for it per call, and keep going entirely on its own. Most x402 services today are single purpose-built endpoints. This opens an entire catalog of community-built web automation tools to any autonomous agent. Key capabilities: • Agents discover, pay for, and run tools autonomously • No account, subscription, or API key required • Payment in USDC on Base via x402 by Coinbase • ~20,000 tools callable via x402 today • Accessible via MCP server or "npx awal" CLI I've shared the link in the replies!

Google DeepMind just dropped the Gemma 4 Technical Report! Gemma 4 is a family of open-weight, natively multimodal models ranging from 2.3B to 31B parameters, covering both dense and Mixture-of-Experts architectures. The most interesting architectural decision is in the 12B model. Instead of separate vision and audio encoders, it uses a unified encoder-free design. The 550M vision encoder is replaced by a single matrix multiplication at 35M parameters. The audio encoder is discarded entirely. Raw image patches and audio chunks are projected directly into the LLM embedding space. No separate encoder pipelines, no memory fragmentation. The efficiency numbers are compelling. A 2.3B effective parameter model (E2B) roughly matches Gemma 3 27B performance with 10x fewer parameters. The quantized 12B fits in 7.65 GB. KV cache footprint is reduced by up to 37.5% through local-to-global attention ratios, positional encoding, and key-value reuse in global attention layers. The audio encoder shrank from 390MB to 87MB after quantization, a 78% reduction, with improved transcription and translation performance. Gemma 4 31B sits at rank 43 on Arena Text as of June 2026, the top open dense model on the leaderboard. Models ranked above it are all MoE architectures with significantly more total parameters. All models ship with a thinking mode that generates a reasoning trace before responding. Benchmark results in thinking mode for the 31B: AIME 2026 at 89.2%, GPQA Diamond at 84.3%, LiveCodeBench v6 at 80.0%, and a Codeforces Elo of 2150. I've shared the link to the report the replies! Also put together a complete guide on fine-tuning Gemma 4 locally with Unsloth. I've quoted the article!

The meta-harness for all your AI coding agents! Omnigent is an open-source orchestration layer that sits above your AI coding agents. Orchestrates Claude Code, Codex, Cursor, OpenCode, Hermes, and Pi through a single interface. Swap or combine harnesses without rewriting anything. The problem with using multiple coding agents: each lives in its own terminal, has its own setup, its own commands. When you want to switch or combine them, you start over. Omnigent removes that. One layer above all of them, consistent interface regardless of which agent is running underneath. Sessions follow you across devices. Start in your terminal, continue in the browser, pick it up on your phone. Messages, sub-agents, terminals, and files stay in sync across every surface. Multi-agent orchestration works within a single session. Point one agent at another's work. Delegate tasks in parallel across Claude Code, Codex, and Pi running in separate git worktrees, then route the diffs to a reviewer from a different vendor. All from one session. Policy enforcement runs at three levels: server, agent, and session. Approve before shell commands execute, cap spend, limit tool calls. Stacks consistently across every harness you plug in. Custom agents via YAML. Define your own with tools, MCP servers, and sub-agents. Two ship out of the box: Polly, a multi-agent coding orchestrator, and Debby, a dual-head Claude and GPT brainstorming agent. Key capabilities: • Orchestrates Claude Code, Codex, Cursor, OpenCode, Hermes, Pi, and custom agents • Sessions follow you across terminal, browser, and phone • Multi-agent collaboration within a single session • Policy enforcement at server, agent, and session level • Cloud sandboxes via Modal, Daytona, E2B, and Kubernetes • Custom agents defined in YAML with tools and MCP servers • Real-time team collaboration and session sharing 100% open source. I've shared the link in the replies! Been going deep on what the harness layer actually needs to look like for agents to work independently in production - identity, memory, proactivity, accountability, and the context layer that ties it all together. Wrote a detailed breakdown on this recently. I've quoted the article!

5 Open Source No-Code LLM, RAG and AI Agent Builders! 1. AutoAgent - A fully automated, zero-code framework. You simply state a high-level goal in natural language, and it handles the planning, task decomposition, and execution automatically. It effectively turns a prompt into a running agent system. 👉 Github Repo: 2. AnythingLLM The best all-in-one solution for internal tooling. It combines RAG, agent workflows, and document management into a single, self-hosted workspace. It is privacy-focused and designed for both technical and non-technical teams to build tools around their own data. 👉 Github Repo: 3. LangChain Open Agent Platform A specialized UI built on top of LangGraph. Instead of hiding the logic, it makes the agent's flow explicit with nodes and edges. This gives you granular control over routing, loops, and multi-agent coordination without needing to write the underlying graph code. 👉 Github Repo: 4. Sim A visual workflow builder with an AI Copilot. You design agent pipelines as executable graphs, but you can use the built-in AI to generate or modify the flows for you. It includes detailed execution tracing, making it much easier to debug complex chains. 👉 Github Repo: 5. Dify A production-ready platform that focuses on observability. It supports prompt management, complex RAG pipelines, and agent logic, but adds the runtime monitoring you need for real applications. If you are deploying to actual users, this is the standard. 👉 Github Repo: If you're going deeper on how these agents actually work in production, I wrote a detailed breakdown on independent agents recently - covering identity, memory, proactivity, accountability, and the context layer that ties it all together. I've quoted the article below

Build and Train your own Diffusion Language Models! dllm is an open-source library that lets you build, train, and evaluate diffusion-based language models without setting up complex pipelines or writing custom training loops. Most language models today are autoregressive. They generate token by token, which makes training and inference fast but also leads to problems like exposure bias and difficulty maintaining global coherence. Diffusion language models flip this approach. Instead of predicting the next token, they denoise corrupted text over multiple steps until the full output is reconstructed. This gives them better global reasoning, fewer cascading errors, and stronger robustness on long-form tasks. The challenge has always been tooling. Diffusion LMs require more complex training loops, noise schedules, and evaluation setups than typical transformers. dllm solves that part. It gives you a structured, reproducible pipeline for training and evaluating diffusion LMs without writing custom scaffolding. Key Features: • Full training workflow for diffusion LMs using clean configs • Support for LoRA, DeepSpeed, and FSDP for scaling and efficiency • Modular model components so you can test new diffusion architectures • Simple dataset loading and experiment management • Built-in evaluation utilities for comparing runs and ablations Github Repo →

Fine-tuning massive LLMs used to be painfully slow, but not anymore! 4 open source libraries that accelerate fine-tuning of Large Language Models 1. Unsloth AI • Fine-tune models like Qwen3, Llama 4, and Gemma 3 up to 2× faster with 70% less VRAM • Uses optimized Triton kernels and manual backprop for exact accuracy • Supports low-resource setups and runs on consumer GPUs or even Colab/Kaggle with ~3 GB VRAM GitHub repo → 2. LLaMA Factory • Fine-tune over 100 models (LLaMA, Mistral, Gemma, etc.) using a simple CLI or WebUI • Supports LoRA, QLoRA, full or frozen fine-tuning across 2–8‑bit precision • Includes built-in dataset templates, training monitors, and model export options GitHub repo → 3. DeepSpeed • Built for large-scale distributed fine-tuning with ZeRO and FSDP • Optimized for multi-GPU and multi-node training with advanced memory management • Trusted in production environments for scalable LLM training GitHub repo → 4. Axolotl • Yaml-based setup for fine-tuning, LoRA/QLoRA, DPO, GRPO, and multimodal workflows • Includes kernel optimizations for memory-efficient training • Actively maintained with support for Hugging Face, model export, and inference GitHub repo →