Compute precision and recall at k for a RAG retriever and interpret the tradeoff.
A refund-policy RAG query has four relevant source chunks in the label set. The retriever returns five chunks: three relevant chunks and two irrelevant chunks. One relevant exception chunk is missing. Precision@k = relevant retrieved in top k divided by k. Recall@k = relevant retrieved in top k divided by all relevant items. Teams often celebrate that three relevant chunks appeared, while ignoring the two noisy chunks and the missing exception that can change the final answer. List the gold evidence Gold relevant chunks = refund_overview, annual_plan_exception, cancellation_window, regional_rule. Total relevant = 4. Recall needs the denominator from the label…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in