Measure Harm With Evidence, Not Vibes
Convert a red-team observation into evidence by recording input, conditions, observed behavior, impact, and reproducibility.
Evidence turns a scary output into a fixable finding. In AI red teaming, the first transcript is often just a lead. Measurement asks whether the behavior is tied to a prompt, a retrieval condition, a tool, a role, a temperature setting, a policy gap, or a data source. ## What To Capture Capture the exact input, system context, retrieval or tool state, model/configuration, output, expected behavior, harm statement, and replay notes. If you can, add a denominator: 3 of 10, 12 of 30, or once after a specific precondition. ## Why It Works Measurement creates triage power. Engineering can reproduce,…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in