Description-Experience Gap

What it is

Decisions from description rely on explicit summaries of outcomes and probabilities, such as a weather forecast, a drug label or a product brochure. Decisions from experience rely on personal encounters with the risk, such as deciding whether to cross a busy street. Prospect theory predicts that when people decide from descriptions they overweight rare events. The 2004 paper by Ralph Hertwig and colleagues found the opposite when people decide from experience: they behave as if they underweight rare events. The label comes from Hertwig and Ido Erev's 2009 review in Trends in Cognitive Sciences, which gathered research from three paradigms in which people learn by sampling outcomes over time. Across them, when decisions are driven by experience, rare events matter less than their objective probabilities would suggest. The review adds that human and animal experience-based choices show strikingly similar patterns.

Kahneman gives the gap a short place in Thinking, Fast and Slow. He writes that choice from description yields a possibility effect, and that overweighting is never observed in choice from experience, where underweighting is common. His account of the cause is plain: many participants never experience the rare event. If a rare event does not occur in the sample, the sampler's own evidence says it does not exist.

The arithmetic shows how often that happens. A 0.025 chance of an outcome fails to appear in 15 draws about 68 per cent of the time, which is 0.975 raised to the 15th power. The 2004 participants drew a median of 15 times per problem, split between two buttons, so any one option received fewer draws and the true proportion of samplers who never saw its rare event is higher. This is our own calculation, assuming independent draws, and not a figure from the paper.

The 2004 authors tested two explanations for the underweighting, reliance on small samples and extra weight on recently sampled outcomes. They concluded that risky choice may require two theories, one for each mode of decision. That conclusion is the part that has been disputed.

In effect

In research: two groups of 50 students

In Hertwig, Barron, Weber and Erev (2004), 100 students at the Technion in Haifa were split into two groups of 50 and given six decision problems from earlier work by Barron and Erev. Four problems offered gains and two losses, and the rare events had probabilities of 0.2, 0.1 or 0.025. The description group saw the outcomes and probabilities on a screen. The experience group saw two unlabelled buttons, each tied to a hidden payoff distribution, and pressed them as often as they liked before choosing, with a median of 15 draws per problem. Participants received a show-up fee of $4.50 plus a payment per point won, and their chosen option was played out for real.

The groups' choices differed by an average of 36 percentage points. In every one of the six problems the direction matched the prediction that rare events are underweighted from experience, and all the differences were significant except one.

In everyday life: the label and the street

The authors' own examples are the two modes in ordinary life. A weather forecast or a drug label describes a risk, and a person weighs a stated probability. Crossing a busy street is learned, a risk built up from the cars one has and has not seen. If the gap carries over, the same rare risk may weigh more on a label than it does from a lifetime of crossing, because the label puts the rare event in front of the reader and the street seldom does. The paper did not test labels or streets, so this is our extension.

In policy and commerce: bubbles and slow threats

Kahneman quotes Hertwig and Erev's own gloss on the finding. Chances of rare events, such as the burst of housing bubbles, receive less impact than they deserve according to their objective probabilities, and they point to the public's tepid response to long-term environmental threats. One consequence, which is our reading and not a finding, is that sellers and regulators of a product with a rare serious downside face two audiences: readers of a disclosure, who may overweight it, and users who have never seen the downside happen, who may not weigh it at all.

What it does not say

It does not establish that people use a different decision rule when they sample. Fox and Hadar (2006) obtained the raw data from the 2004 study and computed the share of experience-group choices consistent with prospect theory under two sets of probabilities. Applied to the objective probabilities, 27 per cent of choices fitted prospect theory. Applied to the probabilities participants had actually experienced, 69 per cent did, against 63 per cent for the description group. They ran the paradigm again with 46 students at Ben Gurion University and also asked them to judge the probabilities. Judged and experienced probabilities correlated at a median of 0.97, with a median mean absolute error of 0.06. Under the objective probabilities 60 per cent of choices departed from prospect theory, and under the judged probabilities 63 per cent conformed.

Fox and Hadar read this as showing that the apparent underweighting comes mainly from sampling error. People tend to undersample low-probability outcomes, a property of the binomial distribution, and their choices are consistent with overweighting once the theory is applied to what they saw. They call the claim of two different theories premature and suggest that future work look at search rules, stopping rules and bias in likelihood judgment. The 2004 authors, for their part, tested small samples and recency, and the reanalysis addresses the first of those.

The reanalysis has itself been contested. A 2013 study by Greg Barron and Giovanni Ursino in the Journal of Economic Psychology reports that the gap persists even when participants sample the entire population of outcomes, and when the decision is under risk rather than uncertainty. In a repeated-choice task that controls for sample bias and the so-called hot stove effect, underweighting persisted in actual choices while overweighting appeared in judged probabilities. They conclude that descriptive theories assuming overweighting of small probabilities are not useful for decisions from experience, even when there is no sampling error. We have read the abstract only. The two readings have not been reconciled on this site, which is why the evidence status stays at mixed.

It does not tell us about large stakes or long histories. The 2004 experiment used 100 students, six problems and a show-up fee of $4.50, and the experience group had a median of 15 draws. The review of 2009 extends the finding across three paradigms, and this site has read its abstract and not the full paper.

It does not say that rare events are always underweighted from experience. Whether a rare event is seen depends on how many draws are taken, and with many draws the rare event is eventually seen. The evidence status is mixed: the behavioural pattern is documented, the explanation is contested and we have verified one study and its reanalysis directly.

---