Lossless quantization has usually meant giving up inference speedup. This paper changes that. SLQ (Statistically-Lossless Quantization) reaches task-lossless compression at 3.3 bits per parameter, and distribution-lossless at 5-6 bpp where the output distribution is practically indistinguishable from the original. 1.7 to 3.6x throughput over BF16 in @vllm_project. Beats FP8 while staying lossless. From Michael Helcig, @_EldarKurtic, and @DAlistarh.

legendary drop from NVIDIA: ModelOpt 0.45.0 biggest additions: - New NVFP4 (W4A16) weight-only quantization format that requires no calibration - Better MoE support (including mixed NVFP4 + FP8 recipes for models like Nemotron) - Easy MXFP4 → NVFP4 conversion for models like DeepSeek V4 and GPT-OSS - Various improvements for large-scale PTQ and Megatron workflows looks like NVIDIA is pushing harder on making 4-bit inference more practical and calibration-free.

new: i8 vectors f32: 4 bytes/dim i8: 1 byte/dim 4x fewer bytes → 75% lower storage and query costs + faster queries when embedded with a quantization-aware model (e.g. voyage-4-large) trained on i8 vectors, recall loss can be ~0! docs:

As an AI Engineer. Please learn: - Harness engineering, not just prompt engineering - Prompt caching vs. semantic caching tradeoffs - KV cache management at scale - Speculative decoding vs quantization - Structured output failures & fallback chains - Evals (LLM-as-judge + human evals) - Cost attribution per feature, not just per model - Agent guardrails & loop budgets - LLM observability as a first-class discipline - Model routing & graceful fallback logic - Knowing when to fine-tune vs. in-context learning

weekend project: 2x3090/vllm cyankiwi/Qwen3.6-27B-AWQ-BF16-INT4 200k context. swival as my coding agent. As long as models keep getting more powerful via RL, distillation, and quantization, GPU depreciation will be much slower than expected. Even a 3090 will remain very useful

Maxim Kontsevich, born August 25, 1964, in Khimki, Russia, is a Russian-French mathematician and mathematical physicist. He won the Fields Medal in 1998 for proving Witten's conjecture on moduli spaces of curves, creating the Kontsevich integral for knot invariants, and his work on deformation quantization of Poisson manifolds. His ideas on homological mirror symmetry have also been hugely influential, blending string theory concepts with algebraic geometry. He's a professor at IHÉS in France and the University of Miami. Kontsevich's career reminds us how crossing boundaries between math and physics can lead to profound discoveries. How do you think proving Witten’s conjecture early on shaped the direction of his later work on quantization and mirror symmetry?

Here’s this week’s shipping recap 👇 — Nano Banana 2 & Nano Banana Pro are now GA and available via the Gemini Enterprise Agent Platform, Gemini API, and in @GoogleAIStudio —Co-Scientist, our new multi-agent system for structured scientific thinking, generates and refines novel hypotheses to solve complex scientific problems — dreambeans from @GoogleLabs works overnight to curate a personalized daily collection of topics that are relevant to you based on your connected Google apps — @GoogleGemma 4 12B, our unified encoder-free model, brings powerful multimodal intelligence straight to your laptop fully offline — Gemma 4 models and their drafters are now optimized with Quantization-Aware Training (QAT) to cut memory requirements and maximize on-device performance — @GoogleMagenta RealTime 2 is our open-weights live music model that you can play like an instrument using a MIDI keyboard, text prompts, and gestures

New course on serving LLMs efficiently -- how do you serve models to many concurrent users at low latency and reasonable cost? This short course is built with @RedHat and taught by @cedricclyburn. Efficient LLM serving requires efficient memory management. A 70B-parameter model takes ~140 GB just to load the weights. On top of that, every active request needs its own chunk of GPU memory, the KV cache, to store the token context it has built up so far. In this course, you'll learn to reduce a model's memory footprint with quantization and serve it using vLLM, which handles many concurrent requests efficiently through smart memory management. Skills you'll gain: - Quantize a model and measure the accuracy tradeoff - Serve a model with vLLM and watch it handle concurrent requests efficiently - Benchmark your deployment and make informed tradeoffs between speed, cost, and accuracy Join and learn to serve LLMs efficiently: