
Experimentation & Evidence Decisions · Module 3 of 4
The Trustworthy Readout
You are Maya, Senior PM at NorthPeak. The re-run on the instrument you chose last quarter has closed, and it is the best result of the year: 24-hour setup completion up 9.2% relative on 20,000 users, a tight interval, every guardrail inside tolerance. You run this course's two checks and both return green. The effect clears the ship bar by 1.84 times, and the test could resolve an effect of 2.59% against a 5.00% bar. The readout is still not data. On the bench: two glass columns holding the assigned users side by side, and they look identical. Engage the balance and the gauge at the fulcrum reads 5.94 sigma, because the split was 10,420 against 9,580: a 2.1-point tilt that reads as rounding and that a fair coin does not do. Draw the rake across a pre-assignment attribute and one band does not match. Grip it, lift it out of both columns, and everything moves at once: the beam settles, the gauge falls to zero, the columns mirror, and the headline drops from 9.2% to 6.4% in your hand. The number gets smaller and better, and it still clears the bar. Then roll the readout through one of four doors: accept, discount, salvage, or void and re-run. Discover why accepting is the worst door here and the one a green dashboard invites, why a hedge that shades the number cannot change its sign, and why the identical 5.94 sigma reading demands salvage in one world and a re-run in another.
0 of 22 items completed0%~91 min remaining
Material · 22 items