opacus icon indicating copy to clipboard operation
opacus copied to clipboard

Remove `_module` prefix from GradSampleModule parameter and buffer names

Open karthikprasad opened this issue 3 years ago • 1 comments

🚀 Feature

Motivation

See discussion in #429 for full context. Relevant snippets form that discussion below:

In my experience "_module" prefix is always an annoyance and never helps. In particular, getting rid of the prefix makes it much easier to load weights from publicly pre-trained models (currently you have to do it strictly before you wrap a model with GSM). ...today you have to be careful and load_state_dict before you wrap the model with GradSampleModule. With same state_dicts you don't have to worry about it.

karthikprasad avatar May 31 '22 19:05 karthikprasad

Also loosely related to this discussion.

pierrestock avatar May 31 '22 19:05 pierrestock