Gates
Cleared, or not cleared
A threshold, a measurement, and whether the thing was actually promoted. The gauge draws the interval as a band rather than a point — a measurement whose interval straddles the threshold has not cleared it however good the mean looks, and the card says inconclusive instead of pass.
Goal reach rate
Passinterval clears the thresholdHeld-out accuracy
Inconclusiveinterval straddles the thresholdWall-contact rate
Failthe card says so out loudProvenance
Where the number came from
A result and a story about a result look identical once they are both a percentage on a slide. The strip travels with the figure: which run, which commit, how many samples, and whether the measurement was taken in one pass or assembled afterwards from the attempts that went well.
Single run, all 500 episodes, no rerun.
one run, complete, at this commitRe-measured the next morning after a harness fix. Compare only against other reruns.
re-measured later — compare only against other rerunsSpliced from 3 runs — run-2026-08-14-2204, run-2026-08-15-0912, run-2026-08-15-1730. This figure was never measured end to end.
The failing episodes were re-run and the better attempt kept. This figure was never promoted, and should not be quoted as a result.
spliced from three runs — not a measurement of the systemSeeds & splits
Which seed carried the mean
A mean over five seeds can be one lucky seed and four ordinary ones. The matrix shows every measurement against the threshold and refuses to average away a disagreement. The split badge answers the other half: whether the score is evidence of generalisation or of memorisation.
Seeds
4/5 clear ≥ 0.85Median 0.91 · spread 0.154/5 clear — report per seed, not the meanSeeds
3/3 clear ≥ 0.85Median 0.94 · spread 0.023/3 clear, tight spreadworkspace/evaluation/holdout-badge.tsx