Episode Details

Training a language model from scratch used to require massive budgets, but a thousand-fold drop in compute costs has changed the math. A de facto five-step ladder—centered on Andrej Karpathy's micrograd, Zero to Hero lectures, minbpe tokenizer, microgpt, and nanochat—takes learners from a 100-line autograd engine to a working GPT. Along the way, building a byte-pair tokenizer explains why models miscount letters in words like strawberry, and a 200-line pure Python script reveals that every AI chat is just document completion. Key sources: • Karpathy, Neural Networks: Zero To Hero course page , retrieved 2026-11-13. — https://karpathy.ai/zero-to-hero.html • micrograd repo (★18,903, MIT, created 2020-04-13, last push 2026-08-03) , retrieved 2026-11-13. — https://github.com/karpathy/micrograd • Karpathy, microgpt explainer post, published 2026-02-12 , retrieved 2026-11-13. — https://karpathy.github.io/2026/02/12/microgpt/ Episode 6542 — myweirdprompts.com/n/6542 (permanent short link: myweirdprompts.com/996919) Listen on Android: MWP FM Player — myweirdprompts.com/apk