TL;DR Uber Freight supports search across 40+ microservices and nearly 100 data sources by composing one shared index from independently owned fragments.
Title: Freight Search: Composing a Shared Search Index from Independently Owned Fragments
URL:
Key points
・🧩 Each index field comes from exactly one fragment, so ownership never conflicts
・🧱 Fragments merge into one denormalized document, so queries need no joins
・⚙️ Spark handles bootstrap; Flink with CDC handles live ingestion, with changes searchable in seconds
・🏷 Mappings are generated from Java annotations, so the class is the single source of truth
・🔗 Declare @/LinkedEntity or a fold function and the platform handles parent-child denormalization and 1:N aggregation
・🔄 The SIA-to-OpenSearch migration touched only a few places and applied to every entity
・📈 About 300 fragments across 20+ indices, thousands of queries per second, p99 in hundreds of milliseconds
Separating business definitions from platform internals is what let the design survive an engine migration.
#
Uber# #
SearchInfrastructure#