登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Avi Chawla
@_avichawla
Daily tutorials and insights on DS, ML, LLMs, and RAGs • Co-founder @dailydoseofds_ • IIT Varanasi • ex-AI Engineer @ MastercardAI
参加 September 2019
152 フォロー中    80.5K ファン
RAG vs. Vectorless RAG, clearly explained! In a standard RAG setup, documents are split into chunks, embedded, and stored in a vector DB. The query gets embedded too, and the top-k closest chunks go to the LLM. The problem is that similarity and relevance answer different questions. For instance, five annual reports from one company can likely follow the same template every year, and only the numbers can change. So a FY2022 question can lie just as close to an FY2025 chunk, and the LLM might answer from the wrong year. Cross-references introduce another problem. If a chunk mentions "see Appendix G" but looks nothing like the table in Appendix G, that table will never make it to the top-k. Vectorless RAG drops the embedding model and the vector DB, and replaces similarity search with LLM reasoning instead. The preprocessing step parses each document's layout into a ToC tree index. Every node holds a section title, its page range, and a short summary, and the whole tree fits in the LLM's context. At query time, the LLM gets the query along with the tree. It reads the section titles and summaries and picks the few sections most likely to hold the answer. For example, for a question about debt, it might pick the "Financial Summary" section and also "Appendix G", because its summary says the debt details are there. The system then pulls those full sections and checks whether the evidence is enough. ↳ If it is, the LLM answers from the full sections, with page citations. ↳ If not, it goes back to the tree and opens more nodes. The LLM also sees the chat history. So if you ask about assets and then follow up with "what about liabilities?", it goes back to the same report. The trade-off is that every query now runs LLM calls over the tree. Top-k vector search will be cheaper in that same run (although not guaranteed to be as accurate), and the tree only helps when documents have real structure. So the two suit different data: ↳ Use a tree for long, structured documents like filings, contracts, and manuals. ↳ Stick with a vector DB for lots of short, unrelated text. The diagram below depicts both flows. If you want to dive deeper, I have also written a full breakdown of MIT's Recursive Language Models. It covers why LLMs get worse as context grows and how RLMs let the model split a long document and work through it piece by piece. Read it below.
もっと見る