Huanyu Zhang

What is the definition of class Trainer? I need to take a look at how self.model has been initialized. Furthermore, could you provide a co-lab link? The current code is...

As an immediate step, could you print out the value of noise_multiplier (https://github.com/pytorch/opacus/blob/main/opacus/privacy_engine.py#L475)? That is the amount of noise added to the system. Sometimes when the noise difference is small,...

Closed this issue, since we launched fixes in PR (https://github.com/pytorch/opacus/pull/598). Btw, it is also feasible to use (https://github.com/lxuechen/private-transformers) for transformers in hugging face.

Thanks for reporting this. We will investigate and launch a fix.

https://github.com/pytorch/opacus/pull/660

Could you link the error message here? thx!

Thanks for contribution! Let me take a look and let you know.

I have checked the implementation of ["nn.utils.spectral_norm"](https://pytorch.org/docs/stable/generated/torch.nn.utils.spectral_norm.html) and [this](https://github.com/christiancosgrove/pytorch-spectral-normalization-gan/blob/master/spectral_normalization.py). I think the main problem comes from their use of "_forward_pre_hooks", either explicitly or implicitly, which Opacus does not support right...

Unfortunately, I do not have a quick solution in my mind right now (cc @karthikprasad in case you have a different opinion). And we might not have bandwidth to support...

Our DP optimizer requires per_sample_gradient for every trainable parameter in the model (https://github.com/pytorch/opacus/blob/main/opacus/optimizers/optimizer.py#L283C17-L283C18). However, in your case, there is some part of the model not activated during (sometime) in the...