Activation is +0.9 pp, p=0.061, powered MDE was +1.5 pp, interval is -0.05 to +1.85, guardrails are clean. Decision = significance result + interval meaning + guardrail health + next-test value The common trap is rounding p=0.061 into a win because it is "close enough" or killing the idea because it missed 0.05. Respect alpha The planned alpha was 0.05; p=0.061 does not meet it. Do not move the threshold after seeing data. The test missed the pre-committed statistical significance rule. Read interval The interval from -0.05 to +1.85 pp includes near-zero and meaningful positive effects. The test did not…
Read the full lesson
Sign up free: one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in