Episode Details

When your text-to-speech reads "makolet" as MACK-oh-let, you have a language identification gap, not a bug. No LID model — GlotLID, fastText, NLLB's lid218e — has a heb_Latn label, so Hebrew written in Latin letters is invisible to every system downstream. This episode walks through the token classifier Daniel shipped to fix it: BIO tagging over xlm-roberta-base, character-span output, and a three-pass pipeline that converts Latin-script Hebrew to Hebrew script before handing it to a Hebrew voice. The real story is the dataset methodology — term-seeded generation, blind annotation by a different model family enforced in code, human review of disagreements, and term-disjoint splits. Plus why precision beats recall here, why a flagged span is work rather than a label, and what a Parakeet fine-tune on dictated audio adds to the picture. Episode 5994 — myweirdprompts.com/n/5994 (permanent short link: myweirdprompts.com/994167)