A model that shouts 0.97 on everything is not confident — it is uncalibrated. Decision-makers need reliability diagrams or at least honesty about intervals. Shipping on uncalibrated swagger is how demos become incident reports.

Raw score / calibrate / interval / decision — Eval hygiene over confident theater.

Before you trust a high score

Do you have any calibration plot or equivalent?
What happens to precision at that confidence threshold?
Who is allowed to override a confident wrong answer?

Soft rule: if you cannot calibrate, do not decorate.

Sources: Public ML calibration discussions (high level). Soft lander — blog_public=0.