Pull to refresh
Logo
Daily Brief
Following Why Ranks Sign Up
Largest-ever reproducibility test finds half of social science claims don't replicate

Largest-ever reproducibility test finds half of social science claims don't replicate

New Capabilities

Seven-year, 865-researcher project also shows open-science reforms are measurably fixing the problem

April 3rd, 2026: National Institutes of Health cites SCORE findings in data-sharing policy announcement

Overview

For decades, social science findings shaped everything from classroom teaching methods to criminal sentencing guidelines—yet no one had systematically checked whether those findings held up. A seven-year project involving 865 researchers, nearly 3,900 papers, and 62 journals across 11 disciplines found that about 55% of published claims successfully replicate and 54% of studies are precisely computationally reproducible.

The project, called SCORE and funded by the United States Defense Advanced Research Projects Agency (DARPA), is the largest and most comprehensive assessment of research reliability ever conducted. Within 24 hours of publication on April 2, major research funders began citing the findings to justify tightening data-sharing and pre-registration requirements.

When researchers shared raw data and code, reproducibility jumped from 54% to 77%; more recent papers replicated at higher rates. Journals requiring data sharing showed dramatically better results, and fields like economics and political science, which adopted transparency mandates earlier, hit 85% computational reproducibility. The message: the reforms adopted in response to the replication crisis are working at scale.

Why it matters

Half the research informing public policy, education, and medicine may not hold up—but proven fixes exist and are spreading.

Questions about this story

No questions yet — be the first to ask.

Play on this story Voices Debate Who Said What? Predict

Key Indicators

55%
Claims that successfully replicated
Of 274 tested claims from 164 papers, 151 showed statistically significant results matching the originals.
54%
Papers precisely computationally reproducible
Of 600 papers tested, only about half could be exactly reproduced from shared data and code.
91%
Reproducibility when data and code are shared
Approximate reproducibility nearly doubled when original materials were available, up from 73.5% overall.
865
Researchers involved
The SCORE project mobilized researchers worldwide across 11 social and behavioral science disciplines.
24%
Papers with openly shared data
Only 144 of 600 papers published between 2009 and 2018 made their data publicly available.
85%+
Reproducibility in economics and political science
Fields with mandatory data and code sharing policies showed dramatically higher reproducibility.

Voices

Curated perspectives — historical figures and your fellow readers.

Ever wondered what historical figures would say about today's headlines?

Sign up to generate historical perspectives on this story.

Log in Sign Up

Play

Exploring all sides of a story is often best achieved with Play.

Log in to play. Track your picks, climb the leaderboards. Log in Sign Up
Predict 5 ways this could play out. Contrarian picks score more — points lock when the scenario resolves. Log in to play
Timeline Five events from this story — drag them oldest to newest. Log in to play
Connections Sixteen names from the news. Find the four hidden groups of four. Log in to play

People Involved

Brian Nosek
Brian Nosek
Principal investigator on SCORE; co-author of the 2026 commentary
 John Ioannidis
John Ioannidis
Foundational figure in metascience; not directly involved in SCORE
 Balazs Aczel
Balazs Aczel
Led the SCORE analytical robustness study
 Gustav Nilsonne
Gustav Nilsonne
Co-authored all three main SCORE papers

Organizations Involved

Center for Open Science
Center for Open Science
Nonprofit Research Organization
Lead coordinating body for the SCORE program

The nonprofit behind the Open Science Framework and most major replication initiatives, including SCORE.
Defense Advanced Research Projects Agency (DARPA)
Defense Advanced Research Projects Agency (DARPA)
United States Federal Agency
Funder of the SCORE program

The Pentagon's advanced research arm funded SCORE with approximately $8 million, reflecting a national security interest in knowing which social science findings are actually reliable.

Timeline

August 2005 April 2026

11 events Latest: April 3rd, 2026 · 3 months ago Showing 8 of 11
Tap a bar to jump to that date

  1. National Institutes of Health cites SCORE findings in data-sharing policy announcement

    Latest Policy

    The NIH Director's office releases statement indicating that SCORE results will inform updates to the NIH Data Management and Sharing Policy, with stricter enforcement beginning in fiscal year 2027 grant cycles.

  2. European Research Council announces mandatory pre-registration for social science grants

    Policy

    The ERC announces that beginning with the 2027 funding round, all social and behavioral science grants above €500,000 will require pre-registration of study protocols before data collection, citing SCORE's demonstration that transparency measurably improves reproducibility.

  3. SCORE publishes largest-ever reproducibility results in Nature

    Publication

    Four papers spanning 865 researchers, nearly 3,900 articles, and 62 journals reveal that about 55% of social science claims replicate and 54% are computationally reproducible—but that open-science reforms measurably improve both numbers.

  4. DARPA funds SCORE program with $8 million

    Funding

    The United States Defense Advanced Research Projects Agency begins funding the Systematizing Confidence in Open Research and Evidence program, the largest-ever effort to assess research reliability.

  5. Social Sciences Replication Project finds 62% replication rate

    Publication

    A team led by Colin Camerer replicates 13 of 21 social science experiments from Nature and Science, with effect sizes about half the originals. Published in Nature Human Behaviour.

  6. Reproducibility Project finds only 36% of psychology studies replicate

    Publication

    The first large-scale systematic replication effort, involving 270 researchers testing 100 psychology studies, publishes in Science. The low success rate triggers alarm across the scientific community.

  7. Transparency and Openness Promotion guidelines published

    Policy

    A coalition of journals, funders, and scientific societies publishes eight modular standards for research transparency, giving journals a concrete framework for requiring open data and pre-registration.

  8. Center for Open Science founded

    Institutional

    Brian Nosek and Jeffrey Spies establish the Center for Open Science to build infrastructure for transparent research practices, including the Open Science Framework.

  9. Diederik Stapel fraud scandal breaks

    Investigation

    Dutch social psychologist Diederik Stapel is suspended after colleagues discover he fabricated data across decades of publications. Fifty-eight papers are eventually retracted.

  10. Daryl Bem publishes precognition paper, exposing methodological flaws

    Publication

    A respected psychologist publishes evidence for extrasensory perception in a top journal using standard methods, demonstrating that accepted research practices can produce absurd results.

  11. Ioannidis publishes 'Why Most Published Research Findings Are False'

    Publication

    Stanford statistician John Ioannidis publishes a mathematical proof in PLoS Medicine that most published research findings are likely false, given prevailing study designs and incentives.

Historical Context

3 moments from history that rhyme with this story — and how they unfolded.

August 2005

Ioannidis paper and the birth of metascience (2005)

Stanford statistician John Ioannidis published a paper in PLoS Medicine titled "Why Most Published Research Findings Are False." Using Bayesian reasoning, he demonstrated that given typical study sizes, effect magnitudes, and the ratio of true to false hypotheses tested, the majority of published positive findings in many fields are likely wrong. The paper has been viewed over 1.8 million times.

Then

The paper was widely discussed but did not immediately change research practices. Many researchers dismissed the argument as overly pessimistic or inapplicable to their field.

Now

It became the intellectual foundation of the replication crisis and the metascience movement—the idea that science itself should be studied scientifically. Every major replication project since, including SCORE, traces its intellectual lineage to this paper.

Why this matters now

SCORE represents the empirical verification—at massive scale—of what Ioannidis argued theoretically twenty-one years ago. The 55% replication rate is remarkably close to his predictions.

August 2015

Reproducibility Project: Psychology (2015)

Brian Nosek and 269 co-authors attempted to replicate 100 published psychology studies. Only 36% produced statistically significant results on replication, and effect sizes were roughly half the originals. Published in Science, the study dominated headlines and forced psychology to confront systemic methodological failures.

Then

Immediate soul-searching within psychology. Some researchers pushed back, arguing the replication attempts were flawed. Others called for fundamental reform.

Now

Catalyzed the open science movement: pre-registration, Registered Reports, data sharing mandates, and the founding of the Society for the Improvement of Psychological Science. Created the template that SCORE scaled across eleven disciplines.

Why this matters now

SCORE is the direct successor to the 2015 project—same lead investigator, same organization, but vastly expanded in scope. The 2026 replication rate of 55% is notably higher than the 2015 rate of 36%, suggesting either that social science beyond psychology replicates somewhat better, or that methodological improvements in the intervening decade are already visible in the literature.

1993–present

Cochrane Collaboration and evidence-based medicine (1993–present)

British epidemiologist Archie Cochrane argued in the 1970s that medical practice should be based on systematic reviews of evidence, not individual studies or clinical intuition. In 1993, the Cochrane Collaboration was founded to produce rigorous systematic reviews of healthcare interventions. It now maintains over 8,000 reviews covering virtually every area of medicine.

Then

Initially met with resistance from clinicians who saw it as cookbook medicine. Adoption was slow through the 1990s.

Now

Evidence-based medicine became the dominant paradigm. Systematic reviews now guide treatment decisions worldwide, and the infrastructure Cochrane built—registries, protocols, standardized methods—became a model for other fields.

Why this matters now

Medicine faced a similar credibility challenge decades earlier and built institutional infrastructure to address it. The open science movement in social science is following a parallel path: from identifying the problem, to building tools and standards, to changing institutional incentives. SCORE's findings suggest social science is roughly where medicine was in the early 2000s—past denial, building infrastructure, but not yet at universal adoption.

Sources

(13)
+7
Nature News Science (AAAS) Karolinska Institutet PLoS Medicine National Institutes of Health European Research Council Nature Nature Nature Nature (Editorial) Nature Science Nature Human Behaviour