turbopuffer(@turbopuffer):tpuf BM25 index tracks matching doc IDs per term: [1, 8, 32, ... 1048576] instead of IDs (big), we store the deltas (small): [1, 7, 24, ... 16] we prefix sum the deltas at query time to rebuild IDs applying @lemire's SIMD speedup gave us up to 13% faster text search on ARM

turbopuffer

@turbopuffer

{vector, full-text} search engine built on object storage. fast, cheap, 1T scale. powers Anthropic, Cursor, Notion, and more

Joined October 2023

5 Following 14.5K Followers

turbopuffer@turbopuffer

2026.07.24 15:16

tpuf BM25 index tracks matching doc IDs per term: [1, 8, 32, ... 1048576] instead of IDs (big), we store the deltas (small): [1, 7, 24, ... 16] we prefix sum the deltas at query time to rebuild IDs applying @lemire's SIMD speedup gave us up to 13% faster text search on ARM