Episode Details

When AI models encounter conflicting information from multiple sources, how do they decide what's true? We explore machine epistemology—the billion-dollar problem happening millions of times a second in every major AI system—and discover humans have been wrestling with this exact dilemma for over 2,000 years. Featuring five distinct knowledge layers battling for authority, and the eternal question: which source is the doctor, and which is the guy on the bus? 00:00 - The Simple Question (That Isn't) 01:45 - The Hierarchy Problem: Doctor vs. Bus Guy 03:30 - Five Sources of Knowledge Baked Into Every AI 06:15 - Pre-training Data: Why Popularity Isn't Truth 09:20 - System Prompts: The Ignored Authority Layer 12:00 - Retrieved Context and the Verification Crisis 15:45 - Ancient Philosophy Meets Modern Nightmares --- Sources & further reading: • Yuxia Wang et al. — "Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict": https://arxiv.org/abs/2506.06485 • Owen Hulatt — "True 'contradictions' and conflicts in the Talmud": https://www.cambridge.org/core/journals/religious-studies/article/true-contradictions-and-conflicts-in-the-talmud/2D4C6F164F1C17601016BF5AD034941F • Altay et al. — "Sycophantic AI decreases prosocial intentions and promotes dependence": https://www.science.org/doi/10.1126/science.aec8352 • Bianchi et al. — "Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistency": https://arxiv.org/abs/2604.09075 • Wuqi et al. — "Who is In Charge? Dissecting Role Conflicts in LLM Instruction Following": https://openreview.net/forum?id=RBfRfCXzkA • Li et al. — "Many-Tier Instruction Hierarchy in LLM Agents": https://arxiv.org/abs/2604.09443 • Zhong et al. — "Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference": https://arxiv.org/abs/2606.20245 • Zhang et al. — "Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Robust Response Generation in the Wild": https://arxiv.org/abs/2504.12982 • Xiang et al. — "Context-DPO: Aligning Language Models for Context-Faithfulness": https://arxiv.org/abs/2412.15280 • Zhu et al. — "Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method": https://arxiv.org/abs/2604.11209 • IEEE Spectrum — "Why AI Chatbots Agree With You Even When You're Wrong": https://spectrum.ieee.org/ai-sycophancy • Britannica — "Textual criticism: Critical methods": https://www.britannica.com/topic/textual-criticism/Critical-methods • Polly Matzinger — "The Danger Model: A Renewed Sense of Self" (concept referenced via Frontiers in Immunology): https://www.frontiersin.org/journals/immunology/articles/10.3389/fimmu.2025.1595764/full • Johns Hopkins CS News — "When new information conflicts with what AI knows": https://www.cs.jhu.edu/news/when-new-information-conflicts-with-what-ai-knows/ • Airia — "AI Security in 2026: Prompt Injection, the Lethal Trifecta, and How to Defend": https://airia.com/blog/ai-security-in-2026-prompt-injection-the-lethal-trifecta-and-how-to-defend/ • EMNLP 2024 — "Knowledge Conflicts for LLMs: A Survey": https://aclanthology.org/2024.emnlp-main.486.pdf • Nova Spivack — "Epistemology and Metacognition in Artificial Intelligence": https://www.novaspivack.com/technology/ai-technology/epistemology-and-metacognition-in-artificial-intelligence-defining-classifying-and-governing-the-limits-of-ai-knowledge This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.