Static embeddings offer unmatched throughput, but they also suffer from accuracy loss compared to traditional embedding models. Can we make them better for retrieval? We tried several things: ✅️ raw maxsim scoring on the per-token embeddings ✅️ training a small adapter model ✅️ changing the distillation training target and teacher While none of these saw the results we wanted, it's an excellent dive into static embeddings and how they do (and don't) work! Blog:

Physical information is sitting inside video embeddings, but models can't quite pull it out. This paper tests that gap with 8,000 controlled physics simulations. Title: World Embedding Benchmark URL: Built from 80 families across fluid mechanics, solid mechanics, dynamics, and optics, this benchmark measures how much physics video embeddings actually "understand." Three highlights stand out. 🔍 Physical information exists, but is hard to extract Pre-trained models show weak retrieval (Recall@10 of just 0.6-7.3%), yet V-JEPA2 achieves strong physical-property regression (nRMSE ~0.94-1.02) despite having no text alignment at all. The physics is encoded — it's just not well connected to language. ⚖️ A tradeoff between alignment and recoverability Physics-specific contrastive training improves retrieval and pair classification, but actually hurts physical-property regression. Reorganizing the representation space for similarity matching comes at the cost of preserving quantitative physical information. 🎬 A real payoff in retrieval-augmented generation Using a physics-adapted model to retrieve reference videos lifted the average generation score from 0.60 to 0.66, with the biggest fidelity gains in optics and thermal domains. The takeaway for world-model design: alignment and recoverability need to be evaluated together, not optimized one at a time. #WorldModels# #VideoGeneration#

Recent research has highlighted the promise of scaling memory embeddings in LLM training. While Engram and STEM index memory by token identity or local n-grams, can we design more flexible memory routing that captures how each token’s meaning changes with context? This paper introduces Mixture-of-Memory Embeddings (MoME), which uses a learned router to sparsely select among multiple memory slots for each token. The architecture outperforms strong memory baselines across three model families. Interestingly, both qualitative and quantitative analyses show that the learned routers’ activations correlate with the context-dependent senses of polysemous words. The authors also train a sub-billion-parameter model that achieves competitive CORE-22 performance against similarly sized base models, including Qwen3-0.6B and Llama 3.2-1B. Code and pretrained models are publicly available!

🔎 A provocative paper: "agentic search needs no embeddings or vector DB—just grep the raw corpus directly." Title: Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction URL: ❓ What is Direct Corpus Interaction (DCI)? 💡 A retrieval paradigm where the agent explores the raw corpus directly via grep, find, and shell commands—no embedding model, vector index, or retrieval API. It needs no offline indexing and adapts naturally to evolving local corpora. ❓ Why aren't conventional retrievers enough? 💡 Sparse or dense, they compress the corpus through a fixed similarity interface into top-k before reasoning begins. That makes exact lexical matching, combining weak clues, and checking local context hard—and evidence filtered out early can't be recovered no matter how strong the downstream reasoning. That's fatal for multi-step agent search. ❓ Can it really win without a retriever? 💡 Yes. On BrowseComp-Plus, swapping the retriever for DCI on the same Sonnet 4.6 lifts accuracy 69.0 → 80.0% (+11.0) while cutting cost 29.4%. Multi-hop QA averages 83.0 (+30.7 over the strongest baseline); IR ranking hits 68.5 NDCG@10 (+21.5). Even the lightweight GPT-5.4 nano version beats many baselines. ❓ What's actually driving it? 💡 The authors call it "retrieval interface resolution." Trajectory analysis shows the edge comes less from surfacing more gold documents and more from high-resolution local search and verification after reaching them. #AIAgents# #RAG#

Introducing EmbeddingGemma 2, our new open embeddings model for on-device use cases! 👀 Code, image, video, audio, and text embeddings 🤏 Modular, going from 270m to 740m parameters 🪆 Matryoshka embeddings for storage efficiency 🤗 Apache 2 License

“Budget” and “financials” are different words, but embeddings understand they’re related. That’s the foundation behind semantic search and one of the core building blocks of modern multimodal systems. Learn how embeddings power retrieval across text, audio, images, and video in Building Multimodal Data Pipelines:

Enterprise RAG needs more than vectors. Hologres stores searchable text with embeddings, uses HGraph for vector retrieval, queries relationships with Cypher, and composes results with SQL. Break free from single-retrieval limits. Deliver a complete "decision evidence package" to significantly boost search efficiency in complex scenarios and elevate business decision-making. 🚀📊 #RAG# #Hologres# #MultiModalSearch#

We’re releasing EmbeddingGemma 2, our first natively multimodal open model engineered for on-device embeddings. Built on the Gemma 4 architecture and released under an Apache 2.0 license, it goes beyond text to unify images, video, audio, and code in a single embedding space.

Me in 2012 instead of learning: DSA System Design AI / ML Generative AI LLMs AI Agents RAG Vector Databases Embeddings Fine-tuning Prompt Engineering MCP AI Coding Agents Python Java C C++ Go Rust JavaScript TypeScript HTML CSS React Next.js Vue Angular Node.js Express.js NestJS Spring Boot Django FastAPI REST APIs GraphQL WebSockets SQL PostgreSQL MySQL MongoDB Redis Elasticsearch Kafka RabbitMQ Docker Kubernetes Terraform AWS Azure GCP Linux Git GitHub CI/CD DevOps Microservices Serverless Cloud Cybersecurity Data Engineering Data Science MLOps Data Analytics Blockchain Web3 Smart Contracts Solidity Edge Computing Distributed Systems Observability Prometheus Grafana OpenTelemetry Supabase Firebase Vercel Cloudflare PostgreSQL + pgvector LangChain LlamaIndex PyTorch TensorFlow Hugging Face Ollama OpenAI APIs Claude APIs Gemini APIs Model Context Protocol AI Infrastructure GPU Computing CUDA