Episode Details

Every episode of this show runs through Chatterbox, an open-weights speech model, on rented Modal GPUs. Today we made it 16% faster with bit-for-bit identical audio — and the wins came from removing work, not changing how we sound. We walk through where inference money actually goes: why a CPU watermarking step was leaving the GPU idle, how an oversized attention cache was reading empty chairs, and why our own accounting had been charging us up to 3x the real GPU bill by counting allocation queue time as compute. Along the way: Modal's three meters (CPU, memory, GPU), cold start economics, and when a warm worker is worth paying for. Key sources: • Resemble AI, Chatterbox README (v0.1.7, commit 5de7a54) — https://github.com/resemble-ai/chatterbox • model card, created 2025-12-02, last modified 2025-12-15 — https://huggingface.co/ResembleAI/chatterbox-turbo • Modal pricing page, per-second GPU/CPU/memory rates — https://modal.com/pricing Episode 6190 — myweirdprompts.com/n/6190 (permanent short link: myweirdprompts.com/304367)