LLM Compressor v0.14.0 is out, and GPTQ just got its biggest speedup since launch. A new Triton kernel makes quantization ~15x faster end to end. Batching layers that share a shape pushes that to ~30x on some MoE workloads. Even the old eager path is 1.5-2x faster. Also new: expanded MSE/iMatrix observers that beat GPTQ for NVFP4 on internal benchmarks, REAP pruning with distributed DDP, and support for GLM 5.3 and Qwen3.8. Full release notes:

llm-d flow control, chapter 3: where control starts. Throughput can plateau while latency keeps rising. The admission point is where flow control decides to keep a request in the queue or admit it to the model server pod. Using a load test, you can find that cutoff point for your model and workload.

llm-d flow control, chapter 2: shared inference under burst pressure. A GPU pool can have spare capacity on average and still run out during a traffic burst. With @_llm_d_ flow control enabled, excess requests are queued until capacity becomes available.

llm-d flow control interactive learner, chapter 1: capacity utilization. Here’s the same traffic in separate tenant-GPU reservations versus a shared pool. Sharing lets tenants use capacity that would otherwise sit idle.

LLM bills spiral when debugging sessions or multi-agent workflows run away. Omnigent warns first, then downgrades, doesn't hard-stop mid-task. 👇 🛡️ trivial_gate routes trivial requests off Opus, Fable, GPT-5 💰 Progressive budgets block expensive tiers; work continues on Sonnet/Haiku 📊 /usage breaks spend by harness, model, and session Phased rollout: visibility → routing → budgets Read more: #Omnigent# #LLMOps# #GenAI#

LLM providers are trying to build enterprise guardrails directly into their products, but it won't work. Companies don't run on a single LLM any more, they're often using a mix of frontier models plus open-source and specialized vertical models spread across various use cases. Model providers can improve their guardrails, but that won't fix the fragmentation problem. The industry tried this before! In the early microservices era, every team baked auth, rate limiting, and logging directly into their apps, which worked fine until you had hundreds of services and no consistent enforcement or unified observability. We abstracted the connectivity logic to the traffic layer back then, and you need to do it again with AI. Effective AI governance (token budgets, permissions, compliance, etc) can't live inside a single model. You need a neutral layer that sits in front of everything. A "Switzerland for AI" as Khozema Shipchandler called it. A single control tower that manages all traffic regardless of source.