Largest-ever reproducibility test finds half of social science claims don't replicate

Overview

Updated May 30

For decades, social science findings shaped everything from classroom teaching methods to criminal sentencing guidelines—yet no one had systematically checked whether those findings held up. A seven-year project involving 865 researchers, nearly 3,900 papers, and 62 journals across 11 disciplines found that about 55% of published claims successfully replicate and 54% of studies are precisely computationally reproducible.

The project, called SCORE and funded by the United States Defense Advanced Research Projects Agency (DARPA), is the largest and most comprehensive assessment of research reliability ever conducted. Within 24 hours of publication on April 2, major research funders began citing the findings to justify tightening data-sharing and pre-registration requirements.

When researchers shared raw data and code, reproducibility jumped from 54% to 77%; more recent papers replicated at higher rates. Journals requiring data sharing showed dramatically better results, and fields like economics and political science, which adopted transparency mandates earlier, hit 85% computational reproducibility. The message: the reforms adopted in response to the replication crisis are working at scale.

Why it matters

Half the research informing public policy, education, and medicine may not hold up—but proven fixes exist and are spreading.

Questions about this story

Free account needed to ask — your question is kept and asked for you right after sign-up. Answers are public.

No questions yet — be the first to ask.

Play on this story Voices Debate Who Said What? Predict Timeline Connections

Key Indicators

55%

Claims that successfully replicated

Of 274 tested claims from 164 papers, 151 showed statistically significant results matching the originals.

54%

Papers precisely computationally reproducible

Of 600 papers tested, only about half could be exactly reproduced from shared data and code.

91%

Reproducibility when data and code are shared

Approximate reproducibility nearly doubled when original materials were available, up from 73.5% overall.

865

Researchers involved

The SCORE project mobilized researchers worldwide across 11 social and behavioral science disciplines.

24%

Papers with openly shared data

Only 144 of 600 papers published between 2009 and 2018 made their data publicly available.

85%+

Reproducibility in economics and political science

Fields with mandatory data and code sharing policies showed dramatically higher reproducibility.

Voices

Curated perspectives — historical figures and your fellow readers.

Ever wondered what historical figures would say about today's headlines?

Play

Exploring all sides of a story is often best achieved with Play.

Most of these play right now — no account needed. Sign up to save scores, keep a streak, and unlock Debate and Predict. Log in Sign Up

Predict 5 ways this could play out. Back the one you believe — contrarian picks score more when a scenario has a resolution date. Log in to play

People Involved

Brian Nosek

Executive Director, Center for Open Science; Professor of Psychology, University of Virginia

Principal investigator on SCORE; co-author of the 2026 commentary

John Ioannidis

Professor of Medicine and Statistics, Stanford University

Foundational figure in metascience; not directly involved in SCORE

Balazs Aczel

Professor, Eotvos Lorand University, Budapest

Led the SCORE analytical robustness study

Gustav Nilsonne

Researcher, Karolinska Institutet

Co-authored all three main SCORE papers

Organizations Involved

Center for Open Science

Nonprofit Research Organization

Lead coordinating body for the SCORE program

The nonprofit behind the Open Science Framework and most major replication initiatives, including SCORE.

Defense Advanced Research Projects Agency (DARPA)

Federal research agency

Funder of the SCORE program

The Pentagon's advanced research arm funded SCORE with approximately $8 million, reflecting a national security interest in knowing which social science findings are actually reliable.

Timeline

August 2005 April 2026

11 events Latest: April 3rd, 2026 · 5 months ago Showing 8 of 11

Tap a bar to jump to that date

April 2026
National Institutes of Health cites SCORE findings in data-sharing policy announcement
Latest Policy

The NIH Director's office releases statement indicating that SCORE results will inform updates to the NIH Data Management and Sharing Policy, with stricter enforcement beginning in fiscal year 2027 grant cycles.
April 3rd, 2026 • 9:15 AM
European Research Council announces mandatory pre-registration for social science grants
Policy

The ERC announces that beginning with the 2027 funding round, all social and behavioral science grants above €500,000 will require pre-registration of study protocols before data collection, citing SCORE's demonstration that transparency measurably improves reproducibility.
April 3rd, 2026 • 2:30 PM
SCORE publishes largest-ever reproducibility results in Nature
Publication

Four papers spanning 865 researchers, nearly 3,900 articles, and 62 journals reveal that about 55% of social science claims replicate and 54% are computationally reproducible—but that open-science reforms measurably improve both numbers.
April 2nd, 2026
February 2019
DARPA funds SCORE program with $8 million
Funding

The United States Defense Advanced Research Projects Agency begins funding the Systematizing Confidence in Open Research and Evidence program, the largest-ever effort to assess research reliability.
February 1st, 2019
August 2018
Social Sciences Replication Project finds 62% replication rate
Publication

A team led by Colin Camerer replicates 13 of 21 social science experiments from Nature and Science, with effect sizes about half the originals. Published in Nature Human Behaviour.
August 27th, 2018
August 2015
Reproducibility Project finds only 36% of psychology studies replicate
Publication

The first large-scale systematic replication effort, involving 270 researchers testing 100 psychology studies, publishes in Science. The low success rate triggers alarm across the scientific community.
August 27th, 2015
June 2015
Transparency and Openness Promotion guidelines published
Policy

A coalition of journals, funders, and scientific societies publishes eight modular standards for research transparency, giving journals a concrete framework for requiring open data and pre-registration.
June 25th, 2015
January 2013
Center for Open Science founded
Institutional

Brian Nosek and Jeffrey Spies establish the Center for Open Science to build infrastructure for transparent research practices, including the Open Science Framework.
January 1st, 2013
September 2011
Diederik Stapel fraud scandal breaks
Investigation

Dutch social psychologist Diederik Stapel is suspended after colleagues discover he fabricated data across decades of publications. Fifty-eight papers are eventually retracted.
September 1st, 2011
January 2011
Daryl Bem publishes precognition paper, exposing methodological flaws
Publication

A respected psychologist publishes evidence for extrasensory perception in a top journal using standard methods, demonstrating that accepted research practices can produce absurd results.
January 1st, 2011
August 2005
Ioannidis publishes 'Why Most Published Research Findings Are False'
Publication

Stanford statistician John Ioannidis publishes a mathematical proof in PLoS Medicine that most published research findings are likely false, given prevailing study designs and incentives.
August 30th, 2005

#timeline [x-show="expanded"] { display: revert !important; opacity: 1 !important; } #timeline .timeline-toggle-btn { display: none; }

Historical Context

3 moments from history that rhyme with this story — and how they unfolded.

1993–present

Cochrane Collaboration and evidence-based medicine (1993–present)

British epidemiologist Archie Cochrane argued in the 1970s that medical practice should be based on systematic reviews of evidence, not individual studies or clinical intuition. In 1993, the Cochrane Collaboration was founded to produce rigorous systematic reviews of healthcare interventions. It now maintains over 8,000 reviews covering virtually every area of medicine.

Then

Initially met with resistance from clinicians who saw it as cookbook medicine. Adoption was slow through the 1990s.

Now

Evidence-based medicine became the dominant paradigm. Systematic reviews now guide treatment decisions worldwide, and the infrastructure Cochrane built—registries, protocols, standardized methods—became a model for other fields.

Why this matters now

Medicine faced a similar credibility challenge decades earlier and built institutional infrastructure to address it. The open science movement in social science is following a parallel path: from identifying the problem, to building tools and standards, to changing institutional incentives. SCORE's findings suggest social science is roughly where medicine was in the early 2000s—past denial, building infrastructure, but not yet at universal adoption.

August 2005

Ioannidis paper and the birth of metascience (2005)

Stanford statistician John Ioannidis published a paper in PLoS Medicine titled "Why Most Published Research Findings Are False." Using Bayesian reasoning, he demonstrated that given typical study sizes, effect magnitudes, and the ratio of true to false hypotheses tested, the majority of published positive findings in many fields are likely wrong. The paper has been viewed over 1.8 million times.

Then

The paper was widely discussed but did not immediately change research practices. Many researchers dismissed the argument as overly pessimistic or inapplicable to their field.

Now

It became the intellectual foundation of the replication crisis and the metascience movement—the idea that science itself should be studied scientifically. Every major replication project since, including SCORE, traces its intellectual lineage to this paper.

Why this matters now

SCORE represents the empirical verification—at massive scale—of what Ioannidis argued theoretically twenty-one years ago. The 55% replication rate is remarkably close to his predictions.

August 2015

Reproducibility Project: Psychology (2015)

Brian Nosek and 269 co-authors attempted to replicate 100 published psychology studies. Only 36% produced statistically significant results on replication, and effect sizes were roughly half the originals. Published in Science, the study dominated headlines and forced psychology to confront systemic methodological failures.

Then

Immediate soul-searching within psychology. Some researchers pushed back, arguing the replication attempts were flawed. Others called for fundamental reform.

Now

Catalyzed the open science movement: pre-registration, Registered Reports, data sharing mandates, and the founding of the Society for the Improvement of Psychological Science. Created the template that SCORE scaled across eleven disciplines.

Why this matters now

SCORE is the direct successor to the 2015 project—same lead investigator, same organization, but vastly expanded in scope. The 2026 replication rate of 55% is notably higher than the 2015 rate of 36%, suggesting either that social science beyond psychology replicates somewhat better, or that methodological improvements in the intervening decade are already visible in the literature.