🎨 Every studio has a "final_final_v7.png." It's often faster to remake old work than to find it, and AI helps with that from an unexpected angle. In creative work, assets pile up faster than anyone can organize them. The catch is that keyword search fails: search "blue environmental concept" and you get nothing if the file is named ENV_ALTSTYLING_DARK_V4. The words and the filenames simply don't share a vocabulary. So flip the approach. Convert images and footage into vector embeddings that represent meaning as numbers. Now "things that mean something similar, however differently phrased, sit close together in vector space," so you can search in natural language. Built on Weaviate, hybrid search combines vector proximity with metadata filters (project, date, type) to land on the right asset. Search tens of thousands of concept images for "a cold, forested biome that feels visually distinct," pull B-roll by shot characteristics, dig up an old drum texture, all without knowing the naming convention. Here, AI isn't the star that generates new content; it's the infrastructure layer that makes existing work accessible, returning to creation the hours lost to organizing and searching. Building Foundry: AI isn't replacing creativity, it's removing friction makes the case for that quiet but essential value. 🔗 #VectorSearch# #CreativeAI#

Here are 5 distance metrics in vector search. But how do you choose the right one? • L1 (Manhattan): sum of absolute differences, exact kNN only with no HNSW support • L2 (Euclidean): straight-line distance, the safe default for most models • Cosine similarity: angle between vectors, magnitude ignored • Dot product: same ranking as cosine on normalized vectors, less compute • Max inner product: dot product without the normalization constraint Most teams default to cosine and move on. That works until your model outputs non-normalized vectors, and suddenly dot product or max inner product is the better fit. Scoring formulas and config details in the blog.

Reliability Sprint Token usage grew 3.5x in a quarter and led to reliability issues across our databases, vector search, job queues, and document processing infra. We implemented database sharding and tighter incident detection, and migrated to horizontally scalable systems like Turbopuffer and Temporal among many other efforts. Resolved all major reliability issues in two months, now Harvey is scaling as fast as before but much more reliably for our customers.

The multi-model advantage for the agentic era: Spanner is a unified multi-model database that natively integrates relational, key-value, graph, full-text search, and vector search on a single, globally consistent engine. In the 2025 Gartner® Critical Capabilities for Cloud Database Management Systems for Operational Use Cases report, Google (Spanner) ranked #1# in the Lightweight Transactions Use Case for the second consecutive year—in our opinion, proving it is the most efficient engine for modern microservices and event-driven architectures. Learn more →

# Practices for Embedding AI Agents in Software # Context Budget Allocator 🎯 The Hook "Just put everything in the context window" sounds reasonable until your costs spike, your system instructions get pushed out, and the LLM ignores the most important retrieved documents buried in the middle. 🔥 The Problem In RAG-powered agents, search results, conversation history, system instructions, and long-term memory all compete for the same finite token window. More input means higher cost but not necessarily better output. As conversations grow, system instructions shrink proportionally and behavior degrades. The "Lost in the Middle" phenomenon means information placed in the center of a long context gets less attention than content at the beginning or end. 💡 The Pattern Divide the context window into named slots (system instructions, retrieval, history, memory) each with a maximum token ratio and priority. Reserve system instructions as a non-compressible fixed slot at 10-20%. Cap retrieval results at a reranked top-k of 3-8 documents. Compress conversation history via summarization when window usage exceeds a threshold. Arrange content to counter Lost in the Middle: critical information first, recent user input last. The higher the cost sensitivity, the tighter the top-k, the lower the compression threshold, and the shorter the history retention. ✅ When to Use Use when: - RAG or memory is active and candidate content could exceed 50% of the model's context window - Cost sensitivity is medium or higher, with token volume affecting both cost and inference latency - Multi-turn conversations accumulate history that crowds out other content types Don't use when: - Input is just system instructions plus a single user message, fitting within 30% of the window - Using a long-context model with input under 20% of the window and low cost sensitivity ⚠️ Pitfalls - Never compress system instructions. Losing tool definitions or safety rules breaks agent behavior entirely - Raw top-k without reranking has low signal density. Retrieve 20 candidates, rerank to 3-8 with a cross-encoder - Summarization is lossy. Key decisions and proper nouns can vanish. Combine with keyword extraction to preserve critical terms 🔧 Implementation Approach - Model the context window as named slots (system/user/retrieval/history/memory) with a struct defining max token ratio, priority, and compressibility per slot - Reserve system instructions as the highest-priority non-compressible fixed allocation, then distribute remaining budget to other slots in descending priority order - Cap retrieval content by reranking vector search candidates with a cross-encoder before fitting within the slot budget, maximizing signal density - Trigger summarization compression on the history slot when it exceeds budget, combining with keyword extraction to prevent loss of critical terms #AIAgents# #SoftwareArchitecture#

📄 Upload your documents, get a knowledge graph — Neo4j's new "Document Intelligence" feature in Aura makes it that simple! Title: Introducing Document Intelligence: From documents to a knowledge graph, right inside Aura URL: 📦 Overview Neo4j has added "Document Intelligence" to its fully managed graph database service Aura. Upload unstructured documents like PDFs, contracts, or technical docs, and the platform automatically extracts entities, resolves duplicates, and constructs a knowledge graph — all without writing a single line of code. ❓ Challenges Solved Most enterprise data is trapped in unstructured documents. Building knowledge graphs from this data previously required complex NLP pipelines, LLM-based entity extraction, entity resolution, and graph schema design — limiting knowledge graph adoption to specialized technical teams. 💡 Methodology & Proposed Approach The feature uses LLM-based entity extraction to identify people, organizations, concepts, and their relationships. Entity resolution merges duplicate references across documents into unified graph nodes. The system infers graph schemas from extracted relationships and stores everything in AuraDB, ready for immediate use with Aura Agent, GenAI Copilot, and GraphRAG pipelines. 🛠 Use Cases - Legal teams automating compliance checks by extracting obligation networks from contract collections - Pharmaceutical companies structuring molecule-disease relationships from clinical trial reports - Enhancing RAG accuracy by supplementing vector search with structural relationships that graphs uniquely capture #KnowledgeGraph# #Neo4j#