Episode Details

Guest: Steven Woo, Fellow and Distinguished Inventor at Rambus
Host: Seth Earley, CEO at Earley Information Science
Published on: May 5, 2026

In this episode, Seth Earley speaks with Steven Woo, Fellow and Distinguished Inventor at Rambus, where he has spent over 30 years at the frontier of memory technology. They explore why memory - not compute - is the binding constraint on AI performance today, how moving data between chips consumes more than half of all power in a high-end AI processor, and what the rise of agentic AI means for infrastructure planning. 

Steven shares a rare long-view perspective on the innovation curve for memory technology, the supply-demand dynamics driving prices higher, and the questions enterprise leaders should be asking before signing their next infrastructure contract.

Key Takeaways:

Memory, not compute, is the primary bottleneck limiting AI performance - and the gap between processor speed and memory speed is widening, not closing.

Over 50 percent of the power consumed by high-end AI processors is spent simply moving data on and off the chip, not performing computation.
Stacking memory components closer together can reduce energy costs dramatically but introduces new challenges around heat dissipation and power delivery.

Training and inference have very different memory profiles - understanding both is essential for organizations architecting AI infrastructure at scale.
Agentic AI compounds the memory challenge significantly, because one user can spin up multiple agents that each spawn further agents, multiplying context and capacity demands.

Memory prices have risen sharply due to supply-demand imbalance - organizations are now signing long-term supply agreements to lock in capacity, just as they do for power.

The most important question enterprise leaders can ask their infrastructure providers is how much experience and demonstrated reliability they have - downtime during model training can be catastrophic.

Insightful Quotes:
"Memory has become a big bottleneck. In many cases, in AI, your speed at which you can actually process information and create new large language models is really gated by the speed and availability of memory." - Steven Woo

"More than 50 percent of the power is spent in circuits just trying to move data on and off the processor. It's pretty astounding to think that as companies plan how much power they need, a lot of it is really related to simply moving data back and forth." - Steven Woo

"People think of compute in terms of gigawatts. But it turns out it's really the movement of that data - and nobody talks about that. It's the silhouette behind the curtain that's actually constraining everything else." - Seth Earley

Tune in to discover why the future of AI depends as much on memory engineering as it does on model development - and what enterprise leaders need to understand about the infrastructure constraints shaping every AI investment they make.

Links
LinkedIn: https://www.linkedin.com/in/stevencwoo/
Website: https://www.rambus.com

Thanks to our sponsors:

• VKTR
• Earley Information Science
• AI Powered Enterprise Book