Episode Details
Back to Episodes
📐Out of Flatland: How Curved Math Is Rewiring AI
Description
📖 Read: https://helioxpodcast.substack.com/publish/post/218050132
Why teaching neural networks to think in curved, folded space — not flat grids — could cut AI's power bill, stop it from hallucinating, and finally give humans a steering wheel for the black box.
Your AI can write code and pass the bar exam — so why does it also hallucinate, drain a small country's worth of electricity, and turn dead in a huge fraction of its own neurons? This episode traces the answer to a surprisingly simple culprit: for a decade, AI has been forced to process the world using flat, grid-based math that was never built for a curved, rotating, three-dimensional reality. We trace the escape route — a shift toward curved geometry called the Grassmann manifold — through the real engineering that's made it possible, including the GRNet architecture built at ETH Zurich, and follow it all the way to a strange, well-documented phenomenon called grokking, where a stuck model suddenly, physically "clicks" into understanding. Along the way: why models hallucinate, why some neurons quietly die, how researchers turned an O(L²) bottleneck into a linear one, and how this same curved math is starting to hand humans literal sliders to steer AI-generated faces and bodies. Evidence-based, deeply researched, and gently skeptical throughout — this is Heliox: Where Evidence Meets Empathy.
References:
Building Deep Networks on Grassmann Manifolds and twelve other papers
Chapters
00:00 Cold Open: The Shadow on the Wall
01:44 A New Paradigm for AI
03:46 How Euclidean Neural Networks Work
06:52 The Quadratic Scaling Problem
09:18 The Memory Wall
10:39 Dying ReLUs
14:17 Escape Route: The Grassmann Manifold
18:15 Neural Superposition Explained
21:19 The k < n/2 Threshold
23:09 Inside GPT-2's Geometry
28:55 Engineering GRNet
32:07 GDLNet and Linear Scaling
37:50 Grokking: The AI's Aha Moment
44:58 Building a Steering Wheel for AI
53:04 Closing Thought: Is Your Brain a Manifold?
This is Heliox: Where Evidence Meets Empathy
Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter. Breathe Easy, we go deep and lightly surface the big ideas.
Disclosure: This podcast uses AI-generated synthetic voices for a material portion of the audio content, in line with Apple Podcasts guidelines.
We make rigorous science accessible, accurate, and unforgettable.
Produced by Michelle Bruecker and Scott Bleackley, it features reviews of emerging research and ideas from leading thinkers, curated under our creative direction with AI assistance for voice, imagery, and composition. Systemic voices and illustrative images of people are representative tools, not depictions of specific individuals.
We dive deep into peer-reviewed research, pre-prints, and major scientific works—then bring them to life through the stories of the researchers themselves. Complex ideas become clear. Obscure discoveries become conversation starters. And you walk away understanding not just what scientists discovered, but why it matters and how they got there.
Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter. Breathe Easy, we go deep and lightly surface the big ideas.
Spoken word, short and sweet, with rhythm and a catchy beat.
http://tinyurl.com/stonefolksongs