Sort Output Failures by the Control That Fixes Them

Place each output failure under the first control lever you would investigate. Prompt or instruction Retrieval or source context Decoding or sampling Validation or evaluation The model answers in prose even though the workflow expects strict JSON. The answer gives an old refund rule that is absent from the retrieved policy chunks. The answer is accurate but too casual for regulated customer emails. The same prompt produces wildly different wording across runs. The answer includes a fact but no quoted source span. The source document contains the answer, but the retriever selected unrelated chunks. The assistant ignores the requested role…

Sign up free — one personalized lesson every day, matched to your role and goals.

Already have an account? Sign in