Hello, moskomule, I run the L0.pytorch code and get the error above, I don't know what's wrong with me, so ask for your help. The specific error is as follows.
back (most recent call last):
File "mnist.py", line 110, in
main(**vars(parser.parse_args()))
File "mnist.py", line 97, in main
trainer.start(epochs, train_loader, test_loader)
File "/media/data/liucc/L0/trainer.py", line 74, in start
self.train(train_data)
File "/media/data/liucc/L0/trainer.py", line 59, in train
self._loop(data_loader)
File "/media/data/liucc/L0/trainer.py", line 32, in _loop
loss, correct = self._iteration(input, target, is_train)
File "/media/data/liucc/L0/trainer.py", line 48, in _iteration
output = self.model(input)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "mnist.py", line 45, in forward
x, z1 = self.conv1(x)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "/media/data/liucc/L0/l0module.py", line 66, in forward
mask, penalty = self._get_mask()
File "/media/data/liucc/L0/l0module.py", line 41, in _get_mask
s = F.sigmoid((torch.log(u) - torch.log(1 - u) + self.loc) / self.temp)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 841, in div
return self.div(other)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 354, in div
return DivConstant.apply(self, other)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/_functions/basic_ops.py", line 171, in forward
return tensor.div(ctx.constant)
RuntimeError: invalid argument 3: divide by zero at /pytorch/torch/lib/THC/generic/THCTensorMathPairwise.cu:66
Hi. I guess the temperature is 0.
Thanks for your reply, moskomule. I run the code successfully and find that the memory of network model doesn't change through L0 regularization. I want to ask whether L0 regularization only have the effect in accelerating networks. and in the paper :LEARNING SPARSE NEURAL NETWORKS THROUGH L0 REGULARIZATION, "We show the amount of neurons left after pruning with the estimator in Eq. 13" in Table 1, do you know how to calculate the neurons left, I print the weight of network, but they are not exactly zeros, how I know which neurons left?
Sorry for late reply. I just checked Tensorboard's histograms.
moskomule
