Episode Details
Back to EpisodesDoRA: Weight-Decomposed Low-Rank Adaptation
Published 5 months, 4 weeks ago
Description
An upgrade over LoRA for parameter-efficient fine-tuning, enabling better performance in LLMs by decomposing weights into magnitude and direction components.