Pull to refresh
Logo
Daily Brief
Following Why
The race to break AI's memory wall

The race to break AI's memory wall

New Capabilities
By Newzino Staff |

First Commercial 3D Chip Stacks Logic and Memory to Escape the Von Neumann Bottleneck

December 23rd, 2025: Universities Publicly Announce Results

Overview

A team from Stanford, Carnegie Mellon, Penn, and MIT just built something chip makers have chased for decades: a true 3D chip manufactured in a U.S. commercial foundry that stacks memory directly on top of computing logic. Presented at December's IEEE electron devices conference, the prototype beats conventional flat chips by 4x in tests and could deliver 1,000x energy efficiency gains in future generations. The trick? Carbon nanotube transistors and resistive RAM built at temperatures low enough to avoid frying the circuits below, creating vertical data highways where today's chips force information to crawl across horizontal distances.

This matters because AI just slammed into physics. Models like GPT-4 have billions of parameters that must shuttle between memory and processors billions of times per inference. The energy cost of moving that data now dwarfs the cost of actual computation—sometimes by 500x. High-bandwidth memory helped, but it's a band-aid: you're still moving data across a 2D plane. Monolithic 3D integration solves the geometry problem by building up instead of out, cutting data travel distances to nearly nothing. If it scales beyond the lab, the entire $166 billion AI chip market restructures around vertical architectures.

Key Indicators

4x
Performance gain over 2D chips
Early hardware tests show prototype outperforms comparable flat designs
1,000x
Potential energy efficiency improvement
Simulated future designs with additional vertical tiers and optimizations
$166.9B
2025 AI chip market size
Market projected to grow 29% annually through 2030
415°C
Maximum fabrication temperature
Low enough to build layers without damaging circuits below

Interactive

Exploring all sides of a story is often best achieved with Play.

Ever wondered what historical figures would say about today's headlines?

Sign up to generate historical perspectives on this story.

Sign Up

Debate Arena

Two rounds, two personas, one winner. You set the crossfire.

People Involved

Subhasish Mitra
Subhasish Mitra
William E. Ayer Professor of Electrical Engineering, Stanford University (Principal investigator on the monolithic 3D chip project)
H.-S. Philip Wong
H.-S. Philip Wong
Willard R. and Inez Kerr Bell Professor, Stanford; Chief Scientist, TSMC (Co-investigator on 3D integration research)
Tathagata Srimani
Tathagata Srimani
Assistant Professor, Carnegie Mellon University (Senior author on the 3D chip paper)

Organizations Involved

SkyWater Technology
SkyWater Technology
Semiconductor Foundry
Status: Manufactured the first commercial monolithic 3D chip

The only U.S.-owned pure-play silicon foundry, operating a DOD-trusted fab in Bloomington, Minnesota.

Stanford University
Stanford University
Research University
Status: Lead research institution on monolithic 3D chip development

Leading research university driving carbon nanotube and 3D integration breakthroughs.

Timeline

  1. Universities Publicly Announce Results

    Public Release

    Stanford, Penn, CMU issue press releases. SkyWater emphasizes domestic manufacturing angle; researchers highlight path to 1,000x energy gains.

  2. First Monolithic 3D Chip Presented at IEDM

    Major Breakthrough

    Stanford, CMU, Penn, MIT, and SkyWater unveil first commercial foundry 3D chip. Hardware tests show 4x gains; simulations project 12x on AI workloads.

  3. GlobalFoundries Launches 22FDX+ RRAM

    Competitive Development

    GF announces embedded resistive RAM for AI edge devices, targeting 2026 production. Weight storage for neural networks cited as key application.

  4. AI Chip Market Hits $67 Billion

    Market Milestone

    Driven by training and inference demand, market grows 41% year-over-year with 29% CAGR projected through 2030.

  5. Srimani Wins Best Paper for Foundry 3D Work

    Academic Recognition

    VLSI Symposium awards Carnegie Mellon research comparing BEOL carbon nanotube devices to FEOL silicon in 3D stacks.

  6. AI Boom Strains HBM Supply

    Demand Shock

    ChatGPT's launch triggers GPU shortage. Nvidia's CoWoS orders surge from 30,000 to 45,000 wafers; TSMC capacity maxes out.

  7. Mitra Transitions CNFET+RRAM to Analog Devices Fab

    Industrialization

    First commercial fab integrates carbon nanotubes and resistive RAM, proving exotic materials can leave the lab.

  8. Apple A10 Validates Advanced Packaging

    Commercial Tipping Point

    iPhone 7 ships with TSMC's InFO packaging. Major customers including Nvidia rush to adopt CoWoS for GPUs.

  9. Stanford Demonstrates First Carbon Nanotube Computer

    Materials Breakthrough

    Mitra's team proves CNFETs can build functional processors, though commercialization remains distant.

  10. TSMC Introduces CoWoS Publicly

    Manufacturing Milestone

    First commercial 2.5D packaging debuts. Early adopters balk at cost; Qualcomm wants 1 cent/mm², TSMC charges 7 cents.

  11. TSMC's Shang-yi Chiang Conceives CoWoS

    2.5D Packaging Emerges

    TSMC begins developing Chip-on-Wafer-on-Substrate, a 2.5D approach using silicon interposers to connect dies side-by-side.

  12. Dennard Scaling Breaks Down

    Crisis Point

    Power density constraints end the free lunch of faster, cooler transistors. Industry shifts to multicore as clock speeds plateau.

  13. John Backus Identifies Von Neumann Bottleneck

    Problem Recognition

    Turing Award lecture coins the term, noting CPU-memory throughput lags processor speed growth.

  14. Von Neumann Proposes Stored-Program Computer

    Historical Foundation

    John von Neumann's EDVAC paper establishes architecture separating memory and computation—a design that would become AI's bottleneck 80 years later.

Scenarios

1

Monolithic 3D Reaches Production by 2027, Reshapes AI Hardware

Discussed by: Tom's Hardware and industry analysts covering the IEDM announcement

SkyWater and partners scale the process to volume production within two years. Hyperscalers including Google and Microsoft adopt monolithic 3D for next-gen TPUs and AI accelerators, drawn by 10-100x energy efficiency gains that slash data center power costs. TSMC and Samsung launch competing programs. By 2028, vertical integration becomes table stakes for AI chips, with memory manufacturers pivoting from HBM to embedded architectures. The U.S. domestic foundry angle attracts CHIPS Act funding. Market bifurcates: conventional 2D for cost-sensitive applications, 3D for performance-critical AI workloads.

2

Manufacturing Challenges Delay Commercialization Past 2030

Discussed by: Semiconductor Engineering coverage of monolithic 3D's history of setbacks

Carbon nanotube purity and yield issues that plagued earlier attempts resurface at production scale. SkyWater's 90nm process proves too coarse for competitive performance; porting to advanced nodes requires redesigning around higher thermal budgets that damage lower layers. Meanwhile, TSMC's 2.5D CoWoS evolves with organic interposers and silicon bridges, delivering 80% of monolithic 3D's benefits at half the risk. Academic teams publish promising papers, but monolithic 3D remains perpetually five years away—echoing decades of fusion energy promises. Industry consolidates around incremental HBM improvements and processing-in-memory instead.

3

Hybrid Architectures Emerge, Monolithic 3D Fills Niche Role

Discussed by: SemiAnalysis and Applied Materials technical roadmaps

The breakthrough proves real but limited in scope. Monolithic 3D excels for specific AI inference workloads—edge devices, real-time processing, embedded neural networks—where energy efficiency dominates cost considerations. Training and large-scale inference stick with 2.5D HBM solutions that leverage mature DRAM supply chains. The market fragments: TSMC dominates hyperscale AI with advanced CoWoS, Intel and SkyWater serve defense and edge applications with monolithic 3D, Samsung hedges with both. By 2030, 15% of AI chips use true 3D stacking—meaningful but not transformative. The architecture becomes one tool among many rather than a wholesale replacement.

Historical Context

Moore's Law and the Shift to 3D Integration

1965-2020s

What Happened

Gordon Moore observed transistor counts doubling every two years in 1965, a trend that held for five decades. But around 2005, Dennard scaling broke—transistors no longer got faster and cooler as they shrank. By the 2020s, physical limits meant you couldn't pack much more onto a flat die without hitting power walls or quantum effects. The industry began exploring the third dimension: stacking dies vertically instead of etching ever-smaller features horizontally.

Outcome

Short Term

2.5D packaging with interposers became standard for high-performance chips by 2016, led by TSMC's CoWoS for GPUs and FPGAs.

Long Term

True monolithic 3D remained elusive until 2025 due to thermal constraints. The Stanford breakthrough represents the transition from scaling within a plane to scaling across planes.

Why It's Relevant Today

The December chip is the culmination of 20 years searching for Moore's Law's successor. If it scales, vertical integration becomes the new paradigm.

High Bandwidth Memory (HBM) Development

2013-2025

What Happened

As AI models exploded in size post-2017, GPUs hit a memory bandwidth wall. HBM addressed this by stacking DRAM dies vertically and connecting them to processors via wide buses through 2.5D packaging. SK Hynix, Samsung, and Micron raced to higher generations: HBM2, HBM3, HBM3E. By 2025, Nvidia's H200 packed 141GB of HBM3E with 4.8TB/s bandwidth—76% more capacity and 43% more bandwidth than the H100. But even HBM couldn't keep up with models growing 410x every two years.

Outcome

Short Term

HBM became the standard for AI accelerators by 2020, with supply constraints emerging by 2023 as demand outpaced fab capacity.

Long Term

HBM addressed symptoms, not causes. It reduced the von Neumann bottleneck but didn't eliminate it—data still moved across 2D space. Memory costs dominated chip budgets.

Why It's Relevant Today

Monolithic 3D offers an escape route from HBM's fundamental limitations by integrating memory and logic in the same vertical stack, slashing data movement distances.

Carbon Nanotube Transistors' Long Road to Production

1998-2025

What Happened

Researchers discovered carbon nanotubes' extraordinary electrical properties in the late 1990s—200x better electron mobility than silicon, near-ballistic transport, operation at room temperature. For two decades, CNFETs were perpetually promising but never deliverable. Synthesis produced tubes of mixed types (metallic and semiconducting); purification was expensive; integrating billions onto wafers seemed impossible. IBM, Stanford, MIT all made prototypes. None reached production. The 2013 Stanford carbon nanotube computer was a milestone but remained a lab curiosity.

Outcome

Short Term

By 2020, Mitra's team transitioned CNFET processes to Analog Devices and SkyWater fabs, proving manufacturability in principle.

Long Term

The 2025 monolithic 3D chip marks CNFETs' first appearance in a commercial foundry process with demonstrated performance gains on real workloads.

Why It's Relevant Today

CNFETs enable the low-temperature fabrication essential for monolithic 3D—you can't build layers at 1000°C without destroying circuits below. The material breakthrough unlocked the architecture breakthrough.

10 Sources:
+4
Stanford Report ScienceDaily Penn Engineering Tom's Hardware Carnegie Mellon University IBM Research Blog Statista Market Forecast Wikipedia DigiTimes GlobalFoundries Press Release