The Illusion of Validity

Confidence in a judgment that tracks how coherent the evidence looks rather than how much of it there is. The phrase reaches this site through the online ratings literature.
What it is
A tidy number, a clean pattern or a consistent story produces confidence that does not depend on whether the underlying data could support it. The feeling of knowing arrives before the check that would justify it.