reinforcement-learning-kr
RuntimeError while running VAIL
The main runs on pytorch 0.4.1 but it fails for pytorch 1.6.0.
Running the main.py for VAIL implementation I get a runtime error about a replacement in place of a variable.
I put here the stack trace.
Error detected in AddmmBackward. Traceback of forward call that caused the error: File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 185, in
main() File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 162, in main train_actor_critic(actor, critic, memory, actor_optim, critic_optim, args) File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/train_model.py", line 75, in train_actor_critic values = critic(inputs) File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/model.py", line 36, in forward v = self.fc3(x) File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 91, in forward return F.linear(input, self.weight, self.bias) File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/functional.py", line 1674, in linear ret = torch.addmm(bias, input, weight.t()) (function print_stack) Traceback (most recent call last): File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 185, in main() File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 162, in main train_actor_critic(actor, critic, memory, actor_optim, critic_optim, args) File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/train_model.py", line 101, in train_actor_critic loss.backward() File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/tensor.py", line 185, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/autograd/init.py", line 127, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
You can change the function 'train_actor_critic', like the following:
def train_actor_critic():
...
critic_optim.zero_grad()
actor_optim.zero_grad()
loss.backward(retain_graph=True)
loss.backward()
critic_optim.step()
actor_optim.step()
