Episode Details

Synthetic training data lives or dies on the generator behind it. This episode compares the instruction models actually available for the job — Qwen, DeepSeek, Llama, Mistral, and Gemma on the open-weight side, and the Claude Opus/Sonnet/Haiku and GPT-5.6 Sol/Terra/Luna tiers on the closed side — with real pricing per million tokens and what each licence actually permits. Then it digs into the scores people cite to prove a model follows instructions: what IFEval really measures, why models near two billion parameters now clear eighty percent on it, and what IFBench's fifty-eight new constraints reveal when the test stops looking familiar. The throughline: producing one acceptable answer is not the same as producing thousands of useful training examples. Key sources: • OpenRouter model listings — https://openrouter.ai/anthropic/claude-opus-5.5 • OpenRouter model listings — https://openrouter.ai/openai/gpt-5.6-sol • IFBench — https://arxiv.org/abs/2507.02833 Episode 6238 — myweirdprompts.com/n/6238 (permanent short link: myweirdprompts.com/542975)