Worked Walkthrough: Write The Final Decision
Use hypothesis result, sample adequacy, interval, and guardrails to write a defensible experiment decision.
Activation is +0.9 pp, p=0.061, powered MDE was +1.5 pp, interval is -0.05 to +1.85, guardrails are clean. Decision = significance result + interval meaning + guardrail health + next-test value The common trap is rounding p=0.061 into a win because it is "close enough" or killing the idea because it missed 0.05. Respect alpha The planned alpha was 0.05; p=0.061 does not meet it. Do not move the threshold after seeing data. The test missed the pre-committed statistical significance rule. Read interval The interval from -0.05 to +1.85 pp includes near-zero and meaningful positive effects. The test did not…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in