DoRA
DoRA copied to clipboard
Published
1 year ago
•
nbasyl
nbasyl
Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
Results
1
DoRA issues
Sort by
recently updated
recently updated
newest added
code: result_dora = (mag_norm_scale - 1) * (F.linear(x, transpose(weight, self.fan_in_fan_out)) ) + mag_norm_scale * lora_B(lora_A(x)) * scaling Question: what is the effect of (mag_norm_scale - 1) and mag_norm_scale ? And,...
question