Use DMAIC to decide whether to improve prompts, retrieval, or fine-tuning data after eval failures.
Failed pilot The assistant failed 38 of 120 eval cases. The team wants an architecture change before classifying the failures. DMAIC turns a noisy failure sheet into an improvement plan. DMAIC Define -> Measure -> Analyze -> Improve -> Control Each phase prevents a different architecture mistake. Shortcut Jump to fine-tuning because failures feel embarrassing Fine-tuning becomes a targeted move, not a panic button. The fix should match the measured defect bucket. D Define M Measure A Analyze I Improve
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in