Start Every Agent Eval With GQM
Write an agent evaluation goal before choosing metrics.
Do not start by asking what to measure. Start by asking what decision the measurement must support. Goal A useful agent eval goal names the agent, the workflow, the purpose of the evaluation, the quality focus, the stakeholder viewpoint, and the deployment context. "Evaluate the agent" is not a goal. "Decide whether the billing-agent draft can safely resolve tier-1 duplicate-charge tickets in production from the support-ops viewpoint" is a goal because it points to a decision. Question Questions translate that goal into observable behavior. Good questions do not ask whether the agent is "good." They ask whether the agent detects…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in