Episode Details

Nine months after our quadratic bottleneck episode, the post-transformer world has actually shipped — just not in pure form. This episode is a census of the hybrid models now in the wild: Qwen 3.5's three-to-one Gated DeltaNet blocks, Nvidia's Nemotron 3, Kimi Linear's 75% cache cut, IBM's Granite 4, Ant Group's trillion-parameter Ling and Ring, and Falcon H1's side-by-side design. Then the counter-story: MiniMax built a linear-attention model, went back to full attention, and published exactly why — expensive evaluation, immature infrastructure, and multi-hop reasoning that broke at scale. Plus the tradeoff underneath it all: fixed-size state means fixed-size memory, and a phrase from token 40 can go missing at token 500,000. What survives is a division of labour — a few attention layers acting as the lookup desk for many cheap recurrent ones. Episode 6211 — myweirdprompts.com/n/6211 (permanent short link: myweirdprompts.com/007950)