Sort the LLM Failure Signal
Categorize common LLM production symptoms by likely failure surface.
Sort each production symptom by the failure surface you should investigate first. Prompt or instruction Retrieval or source data Tool or permission Model behavior Human workflow Answer cites a policy paragraph removed from the help center last week Agent tries to create a ticket but receives a 403 from the workflow API All answers became shorter after a prompt merge that removed the caveat section Same prompt and sources now refuse benign internal questions after a model upgrade Reviewers keep approving drafts even when the UI shows a low-confidence warning Bot invents a refund rule only when retrieved documents include…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in