註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
加入 May 2026
287 正在關注    473 粉絲
TL;DR: The common claim that dense embeddings beat BM25 in code search depends heavily on how you evaluate. Against a fair BM25 the premium shrinks, and fusing dense with BM25 is the safer choice. Embeddings vs. Grep: Rethinking the Dense Advantage in Code Search Key points ・🧪 Proposes RepoPremium, a repo-level benchmark (core: 1,265 queries over 10 Python repositories) ・📏 Switching BM25 from chunks to whole files shrinks the Cur-Pre premium from +0.121 to +0.034 ・🔤 With generate-then-grep, where an LLM writes identifiers first, BM25 scores 0.613 vs 0.532 for dense ・🤝 RRF fusion of dense and BM25 scores 0.609, above dense alone at 0.550 ・🧭 About 30 routers were tried; none met the preregistered criterion ・🤖 In agent runs, swapping the search tool moves exact-match by only about 0.04 Caveats: mostly 10 Python repositories and localization-only evaluation; other languages use one repository each. #CodeSearch# #RAG#
顯示更多