Pre-Mortem an A/B Test Readout
Check an experiment readout for metric, randomization, sample, and guardrail validity before launch.
trust-checklist before-after stat-tile-trio Pricing-page treatment shows a 3.1% signup lift, but mobile traffic is 62% of treatment and 49% of control. Metric -> assignment -> sample -> guardrail -> decision The common trap is to treat a positive lift as a launch decision before checking whether the comparison was trustworthy. Decision metric Primary metric: signup conversion. Decision: ship the pricing page if conversion lifts without guardrail harm. A pre-named metric prevents post-hoc cherry-picking and ties the test to the launch decision. Assignment Treatment mobile share = 62%; control mobile share = 49%. The groups are imbalanced on a major behavior…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in