RAG Reranking: Filtering Retrieval Results by Relevance

#The scenario

Your RAG pipeline retrieves 20–50 candidate chunks per query. Vector similarity is a blunt instrument: half the candidates are topically adjacent but don't actually answer the question. Stuffing all of them into the LLM's context costs tokens, dilutes attention, and invites hallucinated synthesis from irrelevant material. Classic cross-encoder rerankers help, but add a model to host and maintain.

#Why Jev fits

"Does this chunk help answer this question?" is a binary relevance judgment — the Noul primitive, returning a probability you can sort and threshold on. It runs per chunk, at retrieval time, on structured input (query + chunk text + metadata). Because output tokens are free under early-access pricing and inputs are a reported $0.042/M (verify at jev.com), judging 50 chunks per query is cheap enough to do on every request rather than sampling.

#Judgment design example

Structured input per candidate chunk:

Judgments to define:

1. Noul per chunk: "Does this passage contain information that directly helps answer the query?" → probability. Sort descending, keep top-k (e.g. 5), drop the rest.
2. Noul on the survivors (optional): "Is the combined evidence sufficient to answer confidently?" → no → escalate to a broader retrieval pass or a human, instead of letting the LLM improvise.
3. Score (alternative to Noul): "Relevance 1–5" when you want graded ranking rather than a binary keep/drop — useful when the boundary between relevant and adjacent is fuzzy in your domain.

#Where the LLM sits

Strictly downstream, on a diet. The LLM receives only the top-k chunks that survived judgment — smaller prompt, higher signal density, fewer chances to confabulate from noise. This is the retrieval-quality application early coverage demonstrated with an "AI memory retrieval" test: Jev grades what memory worth surfacing; the generative model only reads the honor roll. See the front-filter recipe.

#Watch-outs

• Judge query+chunk, not chunk alone. Relevance is relational; the same passage is gold for one query and noise for another. Always include the query in the state.
• Latency budget. N sequential judgments add up. Fire them concurrently (the SDKs handle 429 backoff) and keep k and the candidate pool small enough to stay inside your p95 target — reported latency is 70–500 ms per call (launch-blog figure, no SLA).
• Recall safety net. If Jev kills everything below the threshold, have a fallback (pass the top vector-score chunks anyway) — an empty context is worse than a noisy one.

#Sources

• jev101.com — application-scenario coverage of RAG result filtering (RAG 筛选) as a flagship judgment-model workload.
• jevai.wiki — API reference pages for the request shape (state + typed questions) used in the design table.
• "Jev: The New AI Model That's Breaking The Internet (Full Tutorial)" — Moritz (~43K views) — includes the "Testing Jev for AI memory retrieval" chapter this pattern generalizes.
• learnjev.com — Three Primitives tutorial (Noul returns a bare probability; no confidence field) and cost/benchmarks context.

Unofficial fan-made handbook. Not affiliated with TypeSafe AI or jev.com.