LoRA icon indicating copy to clipboard operation
LoRA copied to clipboard

[Embedding-bugfix]: reset_parameters

Open WrRan opened this issue 1 year ago • 0 comments

initialize A the same way as the default for nn.Linear and B to zero

But why not use nn.init.kaiming_uniform_(self.lora_A, a=math.sqrt(5)) as class Linear?

WrRan avatar Feb 19 '24 07:02 WrRan