opacus
opacus copied to clipboard
Remove `_module` prefix from GradSampleModule parameter and buffer names
🚀 Feature
Motivation
See discussion in #429 for full context. Relevant snippets form that discussion below:
In my experience "_module" prefix is always an annoyance and never helps. In particular, getting rid of the prefix makes it much easier to load weights from publicly pre-trained models (currently you have to do it strictly before you wrap a model with GSM). ...today you have to be careful and load_state_dict before you wrap the model with GradSampleModule. With same state_dicts you don't have to worry about it.
Also loosely related to this discussion.