Episode Details

Daniel wants a daily AI assistant that writes like him, and he's weighing two ways to get the training data: hand-editing the model's responses, or leaving written notes about what to change. This episode unpacks the research behind both — user-edit fine-tuning (Gao et al., NeurIPS 2024) and critique-and-revise (Jin et al., Amazon) — and asks which actually produces better stylistic results. The evidence is genuinely split: critique-and-revise wins on reasoning benchmarks and data efficiency, while edit-based training unifies supervised, preference, and cost signals from a single gesture. There's no head-to-head study for personal style fine-tuning, and that gap matters. We walk through edit distance as a reward signal, why 20% of frontier-model critiques contain errors, and why SFT punishes inconsistent annotators while DPO forgives them. Episode #046927 — open it directly at myweirdprompts.com/046927