The Four Psychometric Standards

The Four Psychometric Standards

Reviewed by Faisal Zaidi

The four psychometric standards are Benjamin Hardy's framing of what a proposed personality theory has to satisfy before anyone uses its results: are the categories reliable, are they valid, are they independent, and are they comprehensive. Applied to the Myers-Briggs Type Indicator his answer is no on all four. It is a compact and genuinely useful checklist, and the four items are the ordinary vocabulary of test theory rather than an invention of his. The conclusion is defensible; his pages do not carry it.

What it is

Run the Myers-Briggs through the four and the shape of the problem is clear.

On reliability, roughly half of respondents receive a different four-letter type on retest four to six weeks later, with independent estimates across studies ranging from about thirty-nine per cent to about seventy-six per cent changing at least one letter over five weeks. The mechanism is not mysterious. Each of the four scales splits a continuous distribution near its median, so anyone scoring near the cut flips category on small measurement noise.

On independence and comprehensiveness, Randy Stein and Alexander Swan, writing in Social and Personality Psychology Compass in 2019, found the four dimensions continuous in the data rather than bimodal. There are no two humps. The theory requires a true type delineated across four dichotomies and the data do not show the dichotomies. Their conclusion is that the theory lacks agreement with known facts and data, lacks testability, and possesses internal contradictions.

The sourcing problem in the book has to be recorded alongside the verdict, because it is the more instructive half. The four-standards judgement, which is the substance, is stated and not cited. What is cited is Merve Emre's trade history The Personality Brokers, quoted for an industry revenue figure and for the history, a remark by Adam Grant, a personality psychologist quoted on horoscopes, and a podcaster's joke about being one type before coffee and another after. A reader who wanted to check the strongest paragraph in the book would be sent to a trade history, a quotation and a joke. The retest figures and the Stein and Swan paper above come from this publication's own verification pass rather than from the book.