P-0025 ● Active

“Most published psychology findings would fail to replicate.”

Community stance
No votes yet
§
Meaning & context

In 2015 a large collaboration repeated 100 studies from top psychology journals and found only about a third produced a statistically significant result again. The "replication crisis" that followed changed how psychology is done.

Whether "most" findings would fail is contested. Replication rates in later projects range from roughly 40% to over 60% depending on field and method, and critics argue that some failed replications were unfaithful or underpowered. Reforms such as pre-registration seem to be improving reliability, but they apply to new work, not to the existing literature.

Published psychology findingsContested
▶

Effects reported in peer-reviewed psychology journals. Social and personality psychology have been studied most; cognitive psychology tends to replicate better.

Fail to replicateContested
▶

A close, well-powered repetition does not find a statistically significant effect in the same direction, or finds a much smaller effect. Other definitions use effect-size comparisons or Bayesian criteria.

Filter by type:
↕
Arguments
6 total
↑ Supporting3
Empirical

When 270 researchers repeated 100 studies from three leading journals, only about a third produced significant results, and effect sizes averaged half the originals.

The Open Science Collaboration found 36% of replications had significant effects in the same direction, compared with 97% of originals, and replication effect sizes averaged about half the original estimates.

Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.

Limitation

Gilbert and colleagues argued that some replications differed substantially from the originals and that the expected failure rate due to chance was underestimated.

Empirical

Publication bias makes the published literature unrepresentative. When journals commit to publish before results are known, the share of positive results drops from almost all to under half.

Scheel, Schijen and Lakens compared standard psychology papers with Registered Reports: 96% of standard papers reported positive results for their first hypothesis, versus 44% of Registered Reports.

Scheel, Schijen & Lakens (2021). An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports. Advances in Methods and Practices in Psychological Science, 4(2).

Limitation

Registered Reports may test riskier hypotheses, so the gap overstates the rate of false positives.

Empirical

Several famous effects have collapsed under large pre-registered replications, including ego depletion and facial-feedback effects, suggesting even well-known findings are unreliable.

A 23-lab registered replication of ego depletion (2,141 participants) found an effect close to zero; the original facial-feedback "pen in mouth" effect failed in a 17-lab replication.

Hagger et al. (2016). A Multilab Preregistered Replication of the Ego-Depletion Effect. Perspectives on Psychological Science, 11(4), 546–573; Wagenmakers et al. (2016). Perspectives on Psychological Science, 11(6), 917–928.

Limitation

High-profile failures are not a random sample, and later work on facial feedback found a small effect under different procedures.

No arguments of this type on this side.
↓ Opposing3
Empirical

Across large replication projects, rates are closer to half or more, not a clear majority of failures, and much higher in some areas such as cognitive psychology.

Many Labs 2 replicated 14 of 28 classic and contemporary findings (50%). The Social Sciences Replication Project replicated 13 of 21 experiments published in Nature and Science (62%), with effects about half the original size.

Klein et al. (2018). Many Labs 2. Advances in Methods and Practices in Psychological Science, 1(4), 443–490; Camerer et al. (2018). Nature Human Behaviour, 2, 637–644.

Limitation

Even a 50–60% success rate means a large share of findings are unreliable, and effect sizes typically shrink by half.

Empirical

Researchers and prediction markets can predict which studies will replicate quite accurately, which means unreliable findings are identifiable, not a hidden majority.

In the Social Sciences Replication Project, prediction markets among researchers correctly anticipated replication outcomes for most studies; studies with larger samples and stronger original evidence replicated far more often.

Camerer et al. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2, 637–644.

Limitation

Predictability doesn't change how many published findings would fail. It shows the problem is detectable.

Causal

Psychology has reformed. Pre-registration, larger samples, open data and Registered Reports are now widespread, so findings published in recent years are more likely to replicate than those from before 2015.

Analyses of psychology publications show steep increases in sample sizes, pre-registration and data sharing since the mid-2010s, and journals increasingly offer Registered Reports.

Nosek et al. (2022). Replicability, Robustness, and Reproducibility in Psychological Science. Annual Review of Psychology, 73, 719–748.

Limitation

A high-profile 2023 paper claiming near-perfect replicability with rigorous methods (Protzko et al., Nature Human Behaviour) was retracted in 2024, a reminder that reform claims also need scrutiny.

No arguments of this type on this side.
≈
Areas of agreement
3 points
Empirical

Original published effect sizes in psychology are, on average, inflated.

Causal

Publication bias and flexible analysis contributed to the problem.

Empirical

Replicability varies substantially across subfields and methods.

P-0025 · Last edited Oct 26, 2026 · 1 revision