Register and share your invite link to earn from video plays and referrals.

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture æŠ•çšżăŻć€‹äșșăźæ„èŠ‹ă§ă™ă€‚
Joined May 2026
287 Following    473 Followers
TL;DR: The common claim that dense embeddings beat BM25 in code search depends heavily on how you evaluate. Against a fair BM25 the premium shrinks, and fusing dense with BM25 is the safer choice. Embeddings vs. Grep: Rethinking the Dense Advantage in Code Search Key points ・đŸ§Ș Proposes RepoPremium, a repo-level benchmark (core: 1,265 queries over 10 Python repositories) ăƒ»đŸ“ Switching BM25 from chunks to whole files shrinks the Cur-Pre premium from +0.121 to +0.034 ăƒ»đŸ”€ With generate-then-grep, where an LLM writes identifiers first, BM25 scores 0.613 vs 0.532 for dense ăƒ»đŸ€ RRF fusion of dense and BM25 scores 0.609, above dense alone at 0.550 ăƒ»đŸ§­ About 30 routers were tried; none met the preregistered criterion ăƒ»đŸ€– In agent runs, swapping the search tool moves exact-match by only about 0.04 Caveats: mostly 10 Python repositories and localization-only evaluation; other languages use one repository each. #CodeSearch# #RAG#
Show more