mup icon indicating copy to clipboard operation
mup copied to clipboard
verified publisher icon microsoft

mu parametrization for gated-mlp and group-query attention

Open ftgreat opened this issue 1 year ago • 0 comments

could you please give me advice about mu parametrization for gated-mlp and group-query attention, thanks very much.

@thegregyang @edwardjhu

ftgreat avatar Mar 22 '24 10:03 ftgreat