optimum-graphcore
optimum-graphcore copied to clipboard
Allow setting max-weight-norm optimizer parameter
[From AlexC in GC]
The reference implementation uses a value of 10 for this parameter. However the implementation in GC-Optimum passes None, with no possibility for the user to set this parameter
- create training command line option
--max-weight-normit can default toNonefor backwards compatibility - in
IPUTrainer.create_optimizerpass this argument to LAMB optimizer as part ofoptimizer_kwargs