What Outcome Bias Does to a Good Call
People rate a decision by how it turned out, even when told the decision maker knew only what they know. Five studies, from medical vignettes to betting prices, follow the habit into courtrooms, workplaces and markets.
Answer to the question
Why do we judge a decision by its result?
Because the result is the visible part of a decision, and Baron and Hershey partly attribute the bias to outcomes making the arguments for or against it more salient. In their 1988 experiments, identical medical decisions were rated higher when the treatment succeeded, although judges were told the decision maker had the same information they had. The habit is called outcome bias.
Key takeaways
- Jonathan Baron and John Hershey found in 1988 that medical decisions were rated higher when the treatment succeeded, with the probabilities held constant and the judges told the decision maker had the same information.
- In their first experiment, the decisions followed by success scored 0.70 points higher on average, and the seven of the last eight subjects who said outcomes should not count showed the same pattern.
- Kim Kamin and Jeffrey Rachlinski found in 1995 that about three-quarters of the foresight group judged a flood too unlikely to justify precautions, while most of the hindsight group faulted the same decision. A debiasing procedure made no significant difference.
- Andy Brownback and Michael Kuhn found in 2019 that principals judged agents on luck even when effort was observable, and that independent third parties showed the same bias.
- Raphael Flepp and colleagues found in 2024 that betting prices overrated teams that had beaten their expected goals, so competitive markets do not fully erase outcome bias.
Seven of the last eight people in the first experiment said that how a medical decision turned out should not change how they rated it. Taken together, those seven rated the cases with a good outcome higher anyway. Jonathan Baron and John Hershey named the pattern outcome bias, after five experiments in which undergraduates judged decisions made by other people with the result already known.
The Case for Judging by Results
A result is the part of a decision that anyone can see. The reasoning happens in one head and is rarely written down, while the result arrives as a fact: the patient recovered, the investment fell, the river flooded. A rule that grades results needs no access to anyone's thinking, cannot be argued with, and is likely to match how the people who live with the consequences will judge.
There is a statistical version as well. Over a long enough record, a person who keeps producing poor results is probably reasoning poorly, so even a single result is weak evidence about the reasoner. Baron and Hershey's own subjects offered versions of this. In the fourth experiment some said outcomes were inevitable or showed luck or skill, and others simply asserted that outcomes mattered.
A result is the part of a decision that anyone can see.
Fifteen Cases With the Odds Held Fixed
The design took away any difference in knowledge between the judge and the person judged. Subjects were told the decision maker had the same information they had, and the probabilities in every case were stated and held fixed.
In 1988, twenty undergraduates at the University of Pennsylvania rated fifteen medical decisions on a scale from minus three to plus three for the quality of the thinking behind them. One set described a 55-year-old man with heart disease weighing bypass surgery with an 8 per cent risk of dying in the operation. Others concerned liver surgery, a foot-infection test and a choice between two diagnostic tests. Each decision appeared with the treatment succeeding or failing.
The decisions followed by success scored 0.70 points higher on average. Across 140 matched pairs, the success was rated higher in 44.3 per cent, the failure in 9.3 per cent, and the two were tied in 46.4 per cent. That gap is outcome bias in its plainest form, and the seven who said outcomes should not count were part of it. It was not confined to the decision itself: the people behind a decision followed by a good outcome were also rated as more competent, and as more deserving of the next decision.
A third experiment used gambles. A prize winner chose between a sure amount and a risky one, and the result of the gamble was shown whether or not it had been taken. The better the outcome of the chosen option, the higher the rating, in 60.0 per cent of 160 pairs against 11.9 per cent the other way. A worse result on the option not taken also lifted the rating of the choice, in 46.5 per cent of 159 pairs against 13.2 per cent. The authors partly attribute the pattern to outcomes making the arguments for each side more or less salient.
Every subject in the five experiments was a student, and nobody tested people judging their own decisions. Across them the authors concluded that understanding the contamination was not enough to remove it, and a fifth experiment, with 111 members of a decision-making class taught by the first author, found the same pattern in whether people would let another student choose for them.
Follow the Rule Into a Courtroom
Daniel Kahneman gives the version with legal weight in Chapter 19 of Thinking, Fast and Slow. People were asked whether a city should have paid for a full-time bridge monitor. Among those shown only the evidence available at the time, 24 per cent said yes. Among those who were also told that debris had blocked the river and caused a flood, 56 per cent said yes, although they had been told not to let hindsight distort their judgement.
The study behind the figures is a 1995 paper by Kim Kamin and Jeffrey Rachlinski, which compared foresight, hindsight and a hindsight condition built to reduce the bias. Foresight participants reviewed the evidence in an administrative hearing, and hindsight participants reviewed parallel materials in a trial. About three-quarters of the foresight group found the flood too unlikely to justify further precautions, and most of the hindsight group judged the same decision negligent. The debiasing procedure made no significant difference. Strictly, the design measures hindsight bias in judgements of negligence, the sibling of outcome bias.
Kahneman lists the people who are judged this way after the event, among them physicians, financial advisers, third-base coaches, chief executives, social workers, diplomats and politicians. His account of what outcome bias costs is that heavier accountability pushes decision makers towards bureaucratic solutions and extreme risk aversion, with defensive medicine the example, and that the same machinery rewards risk takers who happen to be lucky.
Effort and Markets Pay the Bill
In a 2019 experiment on outcome bias in a work setting, by Andy Brownback and Michael Kuhn, principals judged agents on outcomes that luck had influenced, even though the agents' effort was perfectly observable. The authors report that this weakens the incentive to work hard. Two remedies failed: letting principals or agents control the information about luck, and handing the judgement to independent third parties, who also showed outcome bias. Elicited beliefs showed that lucky agents were seen as harder workers than identical unlucky ones.
Money does not remove the effect either. Raphael Flepp and colleagues asked whether outcome bias survives when money is at stake. Using European football data and a betting exchange, they reported in 2024 that prices overestimated the winning chances of teams that had beaten their expected goals and underestimated those that had fallen short. Bets on the overperformers returned less than zero and bets on the underperformers returned more. Competitive markets, the authors conclude, do not fully erase outcome bias. A result records how things went and only the reasoning records whether the call was good, yet in the experiments, the courtroom study, the work setting and the betting prices the two were graded as one.
Common questions
Why do we judge a decision by its result?
Because the result is the visible part, and Baron and Hershey partly attribute the bias to outcomes making the arguments for or against a decision more salient. In Baron and Hershey's 1988 experiments, identical medical decisions were rated higher when the treatment succeeded, although the judges were told the decision maker had the same information they had. The habit is called outcome bias.
How is outcome bias different from hindsight bias?
Hindsight bias is the feeling, once you know what happened, that it was predictable. Outcome bias is the verdict that follows, in which the decision maker is rated by the result. In the bridge-monitor study the measured effect was hindsight in negligence judgements, which makes outcome bias the connected effect and not the same one.
Does knowing about outcome bias stop it?
The studies here say not reliably. Seven of the last eight subjects in Baron and Hershey's first experiment said outcomes should not count and still rated successes higher, and Kamin and Rachlinski's debiasing procedure made no significant difference to hindsight judgements.
Can a result ever be good evidence about a decision?
When the reasoning cannot be seen, a result is some evidence about the person. In these studies the judges were told the decision maker had the same information they had, so the result had nothing further to say about the quality of the reasoning.
Sources
Baron, J., & Hershey, J. C. (1988). 'Outcome bias in decision evaluation.' Journal of Personality and Social Psychology, 54(4), 569-579. doi:10.1037/0022-3514.54.4.569
Kamin, K. A., & Rachlinski, J. J. (1995). 'Ex post ≠ ex ante: Determining liability in hindsight.' Law and Human Behavior, 19(1), 89-104. doi:10.1007/BF01499075. Read at the level of the published abstract.
Brownback, A., & Kuhn, M. A. (2019). 'Understanding outcome bias.' Games and Economic Behavior, 117, 342-360. doi:10.1016/j.geb.2019.07.003. Read at the level of the published abstract.
Flepp, R., Merz, O., & Franck, E. (2024). 'When the league table lies: Does outcome bias lead to informationally inefficient markets?' Economic Inquiry, 62(1), 414-429. doi:10.1111/ecin.13163. Read at the level of the published abstract.
Kahneman, D. (2011). Thinking, Fast and Slow. New York: Farrar, Straus and Giroux. Chapter 19, 'The Illusion of Understanding'; the bridge-monitor figures, the list of professions and the passage on accountability, cited by chapter.