Episode Details

Twenty years. 28,000 predictions. One humiliating result: credentialed experts predicting the future performed no better than a chimpanzee throwing darts. We explore Philip Tetlock's landmark study on expert judgment, why confident cable news personalities are the worst predictors, and the hedgehog-versus-fox framework that separates forecasters who actually get it right. 00:00 - Recap: Proprioception and sensing the future 02:30 - Philip Tetlock's 20-year prediction study begins 06:00 - The headline: Experts vs. dart-throwing chimps 09:15 - Why cable news made predictions worse, not better 12:45 - Isaiah Berlin's hedgehogs and foxes 16:30 - What separates super forecasters from the rest 20:00 - Applying judgment to uncertainty --- Sources & further reading: • Philip Tetlock & Dan Gardner — Superforecasting: The Art and Science of Prediction (2015) — (book) • Philip Tetlock — Expert Political Judgment: How Good Is It? How Can We Know? (2006): https://press.princeton.edu/books/hardcover/9780691178288/expert-political-judgment • Barbara Mellers, Eric Stone, et al. — "Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions": https://web.stanford.edu/~knutson/jdm/mellers15.pdf • Daniel Kahneman, Olivier Sibony, Cass Sunstein — Noise: A Flaw in Human Judgment (2021) — (book) • James Surowiecki — The Wisdom of Crowds (2004) — (book) • Edward Vul & Harold Pashler — "Measuring the Crowd Within" (2008): https://journals.sagepub.com/doi/10.1111/j.1467-9280.2008.02136.x • AI Impacts — "Evidence on good forecasting practices from the Good Judgment Project": https://aiimpacts.org/evidence-on-good-forecasting-practices-from-the-good-judgment-project/ • Forecasting Research Institute — "AI models have likely reached parity with superforecasters on ForecastBench": https://forecastingresearch.substack.com/p/ai-models-have-likely-reached-parity • Scott Alexander — "The AI Superforecasters Are Here": https://www.astralcodexten.com/p/the-ai-superforecasters-are-here • Muhammad Noraiz Abid — "The Polymarket Paradox: Manipulation, Whale Concentration, and Predictive Accuracy": https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6670638 • Good Judgment — "Ten Commandments for Aspiring Superforecasters": https://goodjudgment.com/philip-tetlocks-10-commandments-of-superforecasting/ • Good Judgment — "What Superforecasters Actually Said About ForecastBench": https://goodjudgment.substack.com/p/what-superforecasters-actually-said • Wikipedia — "The Good Judgment Project": https://en.wikipedia.org/wiki/The_Good_Judgment_Project • Commoncog — "How The Superforecasters Do It": https://commoncog.com/how-the-superforecasters-do-it/ • Glenn W. Brier — "Verification of Forecasts Expressed in Terms of Probability" (1950) — (journal article, Monthly Weather Review) • Francis Galton — "Vox Populi" (1907) — (journal article, Nature) This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.