💾 Billion-scale vector search without keeping everything in memory. 📰 Title: HFresh: Memory-Efficient Vector Search 🔗 URL: Weaviate introduces HFresh, a new disk-based vector index built to handle billion-vector datasets on constrained memory budgets. Highlights 🧩 Partition-based architecture Instead of connecting every vector in one giant graph like HNSW, HFresh splits vectors into small regions called postings. An in-memory centroid HNSW narrows down the right region, then only the relevant postings are read from disk. 📦 Two-stage quantization Centroids use RQ8, cutting memory about 4x while preserving routing accuracy. Postings use RQ1, compressing up to 32x versus 32-bit floats to minimize disk I/O and storage cost. 🔄 Rebuild-free background maintenance Split, Merge, and Reassign, based on the LIRE protocol, keep the index fresh continuously with no full rebuilds required. On the 1M-vector DBpedia dataset, HFresh used just 239MB of heap versus 6.67GB for uncompressed HNSW, about 28x less. It has also been proven at 1 billion 256-dim vectors, making it a strong fit for memory-constrained, large-scale deployments. #VectorSearch# #Weaviate#