A holdout set you keep opening for "just one more idea" is no longer a holdout. Teams that seal the final slice and look once can defend a number. Teams that nibble it weekly are grading their own homework.

Train/tune split, sealed holdout, one look, leakage audit — Eval hygiene: sacred means access-controlled, not vibes-controlled.

Protect the slice

Separate train, validation/tune, and final holdout with written owners.
Log every access to the holdout — curiosity counts as contamination.
Audit for leakage (duplicates, near-dupes, prompt overlap) before bragging.

Public habit: refuse procurement scores without a holdout policy sentence.

Sources: Public ML evaluation methodology notes; data leakage primers; buyer RFP reproducibility asks. Soft-lander essay — no product claim.