Turn FMEA Into Benchmark Cases

A scheduling assistant may miss contraindications hidden outside the main request. FMEA translates severe, hard-to-detect failure modes into targeted benchmark slices. Leaving the risk as misses critical context makes the benchmark too vague to test. Benchmark only ordinary scheduling requests and report overall booking accuracy. Add risk slices for attachment allergies, conflicting instructions, and urgent escalation triggers with stricter pass gates. Failure mode Assistant misses contraindication in attachment. Specific enough to source and label. Effect Unsafe booking and clinical review failure. Severity justifies targeted coverage. Trigger Medication allergy appears in PDF attachment, not message body. The benchmark must reproduce the…

Sign up free — one personalized lesson every day, matched to your role and goals.

Already have an account? Sign in