Preregistration

What it is

The problem preregistration addresses is not lying. It is latitude. A researcher analysing data has a long series of defensible choices to make: when to stop collecting, which participants to exclude for inattention or non-compliance, which of several measured outcomes to treat as the primary one, whether to control for age, whether to log-transform. Each choice is arguable on its own, and a researcher who already believes the hypothesis will tend to resolve the arguable ones in its favour without noticing. That accumulated latitude is what separates "my effect replicated" from "it didn't" with nobody intending to cheat. Registering the plan in advance does not remove the choices. It forces them to be made before anyone can see which way they would cut.

The second thing it does is put nulls on the record. A registered study exists as a public document whether or not it works, which is the part that speaks directly to publication bias. A literature in which nulls are registered and shared is a literature where the absence of an effect leaves a trace, and a count of published supporting studies starts to mean something again.

There is a pointed piece of evidence for why this is needed, drawn from the replication literature rather than from any one experiment. The strongest predictor of whether a replication attempt succeeded was not sample size and not study design. It was whether one of the original authors sat on the replication team. That is not an accusation of misconduct. It is the clearest available measurement of how much unconscious latitude there is in the process, and preregistration is the mechanism designed to take exactly that out of play.

The split it produces shows up cleanly in the ego-depletion literature. The 2010 meta-analysis of unregistered work put the effect at d = 0.62. The preregistered multi-laboratory tests that followed returned 0.04 across 23 laboratories, 0.06 across 36, and 0.10 across 12. Martin Hagger, who produced the 0.62 figure, ran the first of those replications himself and published the result, which is worth noting in a debate usually described in terms of camps.

In effect

The tally that settled a rescue theory

When ego depletion began to fail, the field already had a replacement: willpower mindset, Veronika Job, Carol Dweck and Gregory Walton's 2010 finding that depletion appeared only in people who believed willpower was a limited resource. In June 2023 Nathan Carruth, Jason Ramos and Akira Miyake published a preregistered replication in PLoS ONE, using the same depletion and outcome tasks, with the letter cancellation materials obtained from the original authors, and a sample of 187 against the original's 60.

They found neither the depletion effect nor its moderation by willpower belief.

One replication proves little, which is why they also reviewed the wider literature. Of thirteen qualifying studies, five reported some supporting evidence and eight did not. The five supportive studies had a median sample of 96 participants; the eight nulls had a median of 275. Only one of the five gave clean statistical support for the original claim, and two studies that did find a moderating effect found it running the opposite way, with more depletion among people who believed willpower was not limited. Seven of the eight nulls were preregistered with publicly shared data. One of the five supportive studies was. The authors requested the original dataset and did not receive it. Their conclusion is unusually blunt for a journal: the finding "is not a replicable finding that can provide a solid basis for rigorous theoretical development."

The check a reader can actually perform

Nobody outside research is going to assess a p-curve. Preregistration status is different, because it is a fact about a paper rather than a judgment about its statistics, and it is usually stated in the abstract or the methods section in as many words, often with a link to a public registration and a shared dataset. Applied to a claim before repeating it, the question is short: was this registered in advance, and did the people who found nothing get to publish? It is close to a proxy for whether a finding sits in the durable tier, and it costs about a minute. The failure mode it prevents is the common one, which is not believing a fraudulent study but believing a real, honestly conducted, badly filtered literature.

A publication that has to decide what to build on

For anyone writing about psychology commercially the practice becomes an editorial gate rather than a research standard. This site carries an evidence_status field on every concept page, decided when the note is written and shown to the reader, and preregistration status is one of the things that decides it. A concept whose supporting figure has never been traced to a paper is tagged as such and may not appear in a published article until someone opens the paper. That is a low bar. It is also higher than most of the industry clears, and it exists because the alternative is discovering in public that a claim you built a piece around was an artefact of which studies happened to get printed.

What it does not say

Preregistration constrains the analysis. It does not make a study good. A registered study can be underpowered, can use an unvalidated protocol, and can measure the wrong thing with perfect procedural discipline. Roy Baumeister's objection to the 2016 multi-laboratory ego-depletion replication is precisely this: the protocol was new and untested and failed its own manipulation check on fatigue, which would mean the independent variable was never successfully manipulated in the first place. That is a fair objection rather than a dodge, and a preregistration does nothing to answer it.

A preregistered null in an underpowered study is weak evidence in either direction. Carruth and colleagues say so about their own paper, and the honesty is instructive: 187 participants is a substantial improvement on 60 and still not enough to establish an absence on its own. What carries their argument is the pattern across thirteen studies, not the single replication, and anyone citing them should cite the pattern.

It is a record, not a lock. Registration makes departures from the plan visible to a reader who checks the registration against the paper. It does not prevent them, and it only helps the reader who looks.

Absence of preregistration is not evidence that a study is wrong. Most of psychology predates the practice becoming normal, which happened in earnest through the 2010s, and treating every unregistered paper as suspect would discard the durable tier along with the casualties. The signal is comparative rather than absolute: it matters when the registered and unregistered studies in the same literature disagree, and it matters most when they disagree in the direction the incentives predict.

Finally, the willpower-mindset tally is narrower than it is usually reported. Carruth and colleagues confine their critique to the application of willpower belief to the ego-depletion paradigm. They say plainly that willpower mindset remains a useful concept in self-control research and that other findings built on it are not implicated by their result. The tally is strong evidence about one rescue theory for one failing effect, and it is not a verdict on the idea that beliefs about self-control matter.

---