Episode Details

When a language has sparse training data, should you fine-tune a big multilingual model or train something smaller from scratch? New research — including a controlled comparison of 10,000 models across 252 languages, a 774-experiment scaling study from the ATLAS project, and a fresh Armenian continued-pretraining paper — offers a messier answer than the fork suggests. This episode digs into catastrophic forgetting, the role of syntactically similar languages, and why fluency gains can hide serious knowledge loss. The user experience difference between the two approaches turns out to be the most revealing part of the story. Episode #001274 — open it directly at myweirdprompts.com/001274