Dialogue: “Should We Fine-Tune the FAQ Bot?”
Coach a stakeholder toward retrieval when the FAQ answers depend on changing documents.
Mara MP Product manager Slack thread Mara says, “Can we just fine-tune the FAQ bot so it finally knows our shipping rules?” The shipping rules changed last Tuesday. The team is about to spend effort on training when the more likely problem is stale source material. 2 public wrong answers Weekly rule changes Coach frame Use the ladder Prompting vs retrieval vs fine-tuning Coach with three tests: instruction quality, evidence freshness, and stable repeated behavior. Common trap Treating “custom” as a synonym for “fine-tuned.” The stakeholder hears a reasoning chain, not a flat no. Architecture should follow the failure mode.
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in