Zum Hauptinhalt springen

Pre-Mortem an A/B Test Readout

Check an experiment readout for metric, randomization, sample, and guardrail validity before launch.

data-fluency5 min read

trust-checklist before-after stat-tile-trio Pricing-page treatment shows a 3.1% signup lift, but mobile traffic is 62% of treatment and 49% of control. Metric -> assignment -> sample -> guardrail -> decision The common trap is to treat a positive lift as a launch decision before checking whether the comparison was trustworthy. Decision metric Primary metric: signup conversion. Decision: ship the pricing page if conversion lifts without guardrail harm. A pre-named metric prevents post-hoc cherry-picking and ties the test to the launch decision. Assignment Treatment mobile share = 62%; control mobile share = 49%. The groups are imbalanced on a major behavior…

Read the full lesson

Sign up free: one personalized lesson every day, matched to your role and goals.

← Back to library

Kontakt aufnehmen