Episode Details

While decoder models dominate headlines, production engineering teams frequently rely on bidirectional encoders for classification, tagging, and retrieval. A close look at real deployments explains the economics: Roblox reached 25,000 moderation checks per second on CPUs, Itaú built BERTaú from scratch to gain 22 percent in retrieval, and PayPal weighed custom pretraining against standard fine-tuning. The operational constraints of latency, tokenization costs, and unlabeled text volume dictate when encoders remain the strictly superior architecture. Key sources: • arXiv 2602.06370, Cost-Aware Model Selection for Text Classification: Multi-Objective Trade-offs Between Fine-Tuned Encoders and LLM Prompting in Production (Valdes) — https://arxiv.org/html/2602.06370 • Roblox Newsroom, How We Scaled Bert To Serve 1+ Billion Daily Requests on CPUs, 2020-05 — https://about.roblox.com/newsroom/2020/05/scaled-bert-serve-1-billion-daily-requests-cpus • PayPal Technology Blog (Moein Saleh), Building PayPal-BERT using Transfer Learning and Self-Supervised Learning Techniques, 2020-12-15 — https://medium.com/paypal-tech/building-paypal-bert-using-transfer-learning-and-self-supervised-learning-techniques-42b8fa55a45d Episode 6539 — myweirdprompts.com/n/6539 (permanent short link: myweirdprompts.com/996860) Listen on Android: MWP FM Player — myweirdprompts.com/apk