Design a grader rubric with smooth scores, partial credit, and anti-hacking checks.
A reimbursement-answer grader gives high scores to answers that sound policy-aware but cite the wrong evidence. Decompose quality into weighted criteria and test the grader against shortcuts. A single holistic grader prompt rewards polished wording and lets the model win with policy-sounding language. Before Score 1 if the answer is professional and mentions policy; score 0 otherwise. After Score factual match, citation support, policy boundary, required format, and tone separately; run hack probes before training. Name the skill The target is not "professional." It is correct reimbursement guidance grounded in a policy excerpt. A grader must reward the behavior that…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in