Episode Details

Open-weight models ship with safety training baked in, and thousands of people quietly take it back out. This episode traces the technical arc from a single paper finding that refusal lives in one direction of activation space, through a $5 rank-one weight edit, to Heretic — an AGPL tool with 34,000 GitHub stars that automates the whole process and outperforms skilled human operators. Along the way: the split between quantization (an access problem) and behavior modification (a different animal entirely), why permissive licenses mean the most interesting fine-tunes never get published, and a preregistered study showing abliteration isn't a scalpel — it makes models measurably more optimistic and more self-justifying. Then the same machinery gets pointed at personality itself, with a framework that matches fine-tuning-level personality control at a hundredth of a point, without any training at all. Key sources: • Heretic GitHub repo (p-e-w/heretic), fetched 2026-10-11 — https://github.com/p-e-w/heretic • Heretic project site — https://heretic-project.org • Arditi et al., Refusal in Language Models Is Mediated by a Single Direction, arXiv:2406.11717 — https://arxiv.org/abs/2406.11717 Episode 5873 — myweirdprompts.com/n/5873 (permanent short link: myweirdprompts.com/008947)