opacus
opacus copied to clipboard
pytorch
Fix freezing modules in Ghost Clipping
Summary: Freezing modules with ghost clipping throws an error as corresponding per-sample norms are (not) calculated. Fix: keep in memory the list of all parameters and checking if corresponding requires_grad is True when calculating norms.
Further, unfreezing modules (with and without ghost clipping) wasn't supported because the hooks aren't present for the corresponding modules. Fix: rewrite requires_grad_ to add the hook.
