Episode Details

Every language problem gets routed to a frontier text-generation model these days — and sometimes that's exactly the wrong engineering choice. This episode tours the classic NLP task families that Hugging Face still files separately from generation: feature extraction, fill-mask, question answering, sentence similarity, summarization, table question answering, text classification, text ranking, token classification, zero-shot classification, and translation. The organizing insight is that what separates these from generation isn't subject matter — it's the shape of the output, and the architecture is chosen to match it. Then the harder question: in the age of instruction-tuned LLMs, which of these jobs can a generation model simply absorb, including via structured output, and when does a specialized model remain the better call? Reliability, latency, cost, throughput, local deployment, determinism, accuracy, embeddings, and architecture all get weighed — and the evidence cuts against both camps. Episode #958709 — open it directly at myweirdprompts.com/958709