Episode Details

Daniel has fine-tuned Qwen3 TTS 1.7B into a voice clone called Meorav, and it currently takes 2.6 seconds of compute for every second of audio. This episode digs into how to actually benchmark speech-model inference: what real-time factor means (and how people invert it), why the first run is always slow, how to measure peak allocated versus peak reserved memory, and why shrinking weights to 8 bits may save space without saving time. Plus ROCm 10.1 tooling, rocprof, CUDA graphs, and what the vLLM Omni team's numbers reveal about launch overhead. Episode 6198 — myweirdprompts.com/n/6198 (permanent short link: myweirdprompts.com/399257)