Build an Evidence-Centered Rubric
Create a benchmark rubric that scores observable evidence instead of vague quality.
Benchmark a compliance chatbot without rewarding confident but unsupported answers. Evidence-centered design: claim -> evidence -> task -> score levels. A vague helpfulness rubric lets reviewers reward tone even when the answer misses compliance evidence. Score 1-5 for helpfulness, clarity, and completeness. Score citation currency, exception handling, unsupported-claim refusal, and escalation of ambiguity. Claim The chatbot gives compliance-safe answers to employee policy questions. The claim names the competence being benchmarked. Evidence Current policy cited; exception identified; unsupported advice refused; ambiguity escalated. Each signal can be observed in the answer. Task Ask policy questions with current rules, retired rules, ambiguous exceptions,…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in