Episode Details

Everyone names Llama, Gemma, Mistral, and DeepSeek — but BERT is the second-most-downloaded model on Hugging Face, and encoder-only models collectively out-download decoder-only models nearly three to one. This episode digs into the taxonomy problem hiding in plain sight: two different animals sharing one foundational task, text in and text out. We trace the arc from BERT's pre-train-then-fine-tune breakthrough through ModernBERT, NeoBERT, and mmBERT, look at the Ettin benchmark results where a 150M encoder beats a 400M decoder at classification, and tour the crowded world of ASR cleanup models — Superwhisper's S1-mini, Quobi's Quill, Promethic Labs' Emberon, and adikuma's mumble-cleanup — where the entire design goal is a model that cleans your dictation and refuses to answer it. Episode #118030 — open it directly at myweirdprompts.com/118030