Episode Details

When you call an API for GPT-4o or Llama 3 70B, are you actually getting the full-precision model you think you are? This episode pulls back the curtain on inference infrastructure, examining how commercial providers like Together AI and Fireworks quantize open-weight models by default, whether closed-source vendors like OpenAI and Anthropic optimize their own models in undisclosed ways, and what happens when your request routes through aggregators like OpenRouter. We explore the economic pressure to quantize, the documented vs. undocumented reality of production inference, and what it means for developers who assume the weights are untouched. Episode #692729 — open it directly at myweirdprompts.com/692729