Jev for RAG, clearly explained!
Hybrid search gives you a shortlist. It does not decide which passages contain evidence, which are merely adjacent, or whether the evidence is strong enough to answer.
That missing judgment is where Jev fits.
Jev sits between retrieval and generation. It does not replace BM25, embeddings, or the LLM. It evaluates the candidates they produce before those candidates enter the context window.
→ Retrieve wide
Combine dense and keyword search, then merge the results with reciprocal rank fusion. This gives Jev a broad candidate pool, such as the top 20 passages shown in the visual.
Retrieval still sets the ceiling. If the right passage is missing from this shortlist, Jev cannot recover it.
→ Judge every candidate together
Send the query as Jev’s state. For each candidate, ask a typed yes-or-no question such as “Does C7 help answer this query?”
Jev evaluates every question in one packed request and returns a calibrated probability for each passage. This avoids making a separate model call for every query-passage pair.
→ Let code apply the threshold
Your application compares each probability with a threshold. Candidates above it continue to the LLM. Everything below it is removed before generation.
Jev makes the fuzzy judgment. Code remains responsible for the actual decision.
→ Gate the entire answer
The same request can check whether the retained passages make the query answerable. It can also flag signs of prompt injection inside a candidate.
If answerability falls below the threshold, the application can skip the LLM and return “not in the documents.” The injection score should remain a filtering signal, not a security boundary.
The result is a cleaner division of work.
Hybrid search retrieves broadly. Jev reranks, filters, and decides whether sufficient evidence exists. The LLM writes only from the passages that survive.
Jev’s value here is not simply moving passages up or down a list. It turns relevance into an explicit probability that your application can inspect, threshold, and act on.
To summarise:
- Retrieval finds the candidates.
- Jev decides what deserves context.
- The LLM writes the grounded answer.
----
I also built an open-source project showing how to use Jev as a judge for AI observability with Comet Opik.
It evaluates support traces for groundedness, request coverage, action honesty, and helpfulness, then records the results as an auditable experiment.
You can explore the project here:
My article on how Jev works is quoted below.