opacus
opacus copied to clipboard
pytorch
tensor_utils.py - unfold2d TypeError: unsupported operand type(s) for +: 'int' and 'str'
🐛 Bug
When using "same" for padding in pytorch conv2d, opacus doesn't compute a tuple as specified in pytorch doucmentation. Instead opadus uses the string "same" in the compution of unfold2d wich leads to type error.
Please reproduce using our template Colab and post here the link
To Reproduce
:warning: We cannot help you without you sharing reproducible code. Do not ignore this part :) Steps to reproduce the behavior:
- Define NN archittecture
- Try to train with by backward propagation with loss.backward()
from torch import nn
from torch import randn
from torch import tensor
import torch
from torch.utils.data import Dataset, DataLoader
from opacus.utils.batch_memory_manager import BatchMemoryManager
import numpy as np
x = randn(100, 3, 32, 32)
y = randn(100,10)
class MSDataset(Dataset):
def __init__(self, x, y):
self.x = x
self.y = y
def __len__(self):
return np.shape(self.y)[0]
def __getitem__(self, idx):
return self.x[idx,:,:,:], self.y[idx,:]
class network(nn.Module):
def __init__(self):
super().__init__()
self.archi = nn.Sequential(
nn.Conv2d(3,64,(3,3), padding='same'),
nn.Conv2d(64,64,(3,3), padding='same'),
nn.MaxPool2d((2,2)),
nn.Flatten(),
nn.Linear(16384, 200),
nn.Linear(200, 10)
)
def forward(self, x):
return self.archi(x)
def train(model, train_loader, optimizer, epoch):
model.train()
criterion = nn.CrossEntropyLoss()
losses = []
top1_acc = []
with BatchMemoryManager(
data_loader=train_loader,
max_physical_batch_size=100,
optimizer=optimizer
) as memory_safe_data_loader:
for i, (images, target) in enumerate(memory_safe_data_loader):
images = images.float()
optimizer.zero_grad()
output = model(images)
loss = criterion(output, target)
loss.backward()
optimizer.step()
train_dataset = MSDataset(x, y)
trainloader = DataLoader(train_dataset, batch_size=100)
model = network()
optimizer = torch.optim.Adam(model.parameters())
from opacus import PrivacyEngine
privacy_engine = PrivacyEngine()
epochs = 10
model, optimizer, train_loader = privacy_engine.make_private_with_epsilon(
module=model,
optimizer=optimizer,
data_loader=trainloader,
epochs=epochs,
target_epsilon=50,
target_delta=1e-5,
max_grad_norm=128,
)
for epoch in range(epochs):
train(model, train_loader, optimizer, epoch + 1)
Expected behavior
In unfold2d it should check if padding equals "same" and if it's it should change padding value to the tupple as defined in pytorch documentation.
Environment
Please copy and paste the output from our environment collection script (or fill out the checklist below manually).
You can get the script and run it with:
Collecting environment information...
PyTorch version: 1.12.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A
OS: Arch Linux (x86_64)
GCC version: (GCC) 12.1.0
Clang version: 14.0.6
CMake version: Could not collect
Libc version: glibc-2.35
Python version: 3.10.5 (main, Jun 6 2022, 18:49:26) [GCC 12.1.0] (64-bit runtime)
Python platform: Linux-5.18.9-arch1-1-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.4
[pip3] torch==1.12.0
[pip3] torchvision==0.13.0
[conda] No relevant packages
Here is the error stack
/home/jaalmoes/.local/lib/python3.10/site-packages/opacus/privacy_engine.py:114: UserWarning: Secure RNG turned off. This is perfectly fine for experimentation as it allows for much faster training performance, but remember to turn it on and retrain one last time before production with ``secure_mode`` turned on.
warnings.warn(
/home/jaalmoes/.local/lib/python3.10/site-packages/torch/nn/modules/module.py:1053: UserWarning: Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. This hook will be missing some grad_input. Please use register_full_backward_hook to get the documented behavior.
warnings.warn("Using a non-full backward hook when the forward contains multiple autograd Nodes "
Traceback (most recent call last):
File "/home/jaalmoes/inria/these_2022/pytorch_basic/shape.py", line 89, in <module>
train(model, train_loader, optimizer, epoch + 1)
File "/home/jaalmoes/inria/these_2022/pytorch_basic/shape.py", line 60, in train
loss.backward()
File "/home/jaalmoes/.local/lib/python3.10/site-packages/torch/_tensor.py", line 396, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/jaalmoes/.local/lib/python3.10/site-packages/torch/autograd/__init__.py", line 173, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
File "/home/jaalmoes/.local/lib/python3.10/site-packages/opacus/grad_sample/grad_sample_module.py", line 342, in capture_backprops_hook
grad_samples = grad_sampler_fn(module, activations, backprops)
File "/home/jaalmoes/.local/lib/python3.10/site-packages/opacus/grad_sample/conv.py", line 43, in compute_conv_grad_sample
activations = unfold2d(
File "/home/jaalmoes/.local/lib/python3.10/site-packages/opacus/utils/tensor_utils.py", line 131, in unfold2d
H + 2 * padding[0] - (kernel_size[0] + (kernel_size[0] - 1) * (dilation[0] - 1))
TypeError: unsupported operand type(s) for +: 'int' and 'str'
I have solved this issue by adding this snippet of code in tensor_utils.py
if padding == "same":
padding =(int(dilation[0]*(kernel_size[0] - 1)/2), int(dilation[1]*(kernel_size[1] - 1)/2))
Hi @jaalmoes
Thank you for filing this bug! We were unaware of this. I will send a fix soon for this 😎🤟🏻
