Episode Details

Back to Episodes
When Merging LoRA Adapters Stops Being Exact

When Merging LoRA Adapters Stops Being Exact

Episode 6619 Published 10 hours ago
Description
Merging a LoRA adapter into its base model is exact in algebra, but finite-precision hardware tells a different story. Folding a half-mebibyte adapter into a 32-mebibyte matrix removes a serving branch rather than shrinking the model, and trades away swappability. The conversation unpacks how bf16 rounding, PEFT's hidden fp32 upcasting, and quantization introduce silent drift—as seen in a documented MiniCPM-V bug—and why combining multiple adapters with methods like TIES and DARE crosses from linear math into lossy surgery. Key sources: • Hugging Face PEFT docs, Model merging, retrieved 2026-11-15, docs index v0.21.2 (updated 2026-11-14). — https://huggingface.co/docs/peft/main/en/developer_guides/model_merging • PEFT docs, Troubleshooting, retrieved 2026-11-15, v0.21.2. — https://huggingface.co/docs/peft/main/en/developer_guides/troubleshooting • PEFT LoRA package reference (`merge_and_unload`, `merge_adapter`/`unmerge_adapter`), retrieved 2026-11-15. — https://github.com/huggingface/peft/blob/main/docs/source/package_reference/lora.md Episode 6619 — myweirdprompts.com/n/6619 (permanent short link: myweirdprompts.com/705826) Listen on Android: MWP FM Player — myweirdprompts.com/apk
Listen Now

Love PodBriefly?

If you like Podbriefly.com, please consider donating to support the ongoing development.

Support Us