contrib
contrib copied to clipboard
pytorch
Implementations of ideas from recent papers
If model has batch normalization layers, where should I use the bn_update()? ``` for _ in range(100): opt.zero_grad() loss_fn(model(input), target).backward() opt.step() opt.swap_swa_sgd() opt.bn_update(train_loader, model) ``` Is this setting correct？ Or...
I've written custom parsers and emitters for everything from docstrings to classes and functions. However, I recently came across an issue when I was parsing/generating from the TensorFlow—and now PyTorch—codebases:...
`test_swa` fails with the following: ``` test/test_swa.py::TestSWA::test_swa /Users/balandat/Code/contrib/torchcontrib/optim/swa.py:200: UserWarning: SWA wasn't applied to param tensor([[-1.3171e-01, 3.3898e-01, 3.9688e-01, 9.4115e-01, -1.0891e-01], [-6.1716e-01, -9.9478e-02, -3.5716e-01, 6.9384e-01, -2.1573e-01], [-9.2800e-01, 2.5287e-01, -9.5098e-01, -3.5055e-01, 6.5104e-01], [...
Here are two modules that implement residual blocks in a nice object-oriented way. They subclass the container modules, and they accept modules as positional arguments similarly to `nn.Sequential`. They also...
This is pull request was created automatically because we noticed your project was missing a Code of Conduct file. Code of Conduct files facilitate respectful and constructive communities by establishing...
This is pull request was created automatically because we noticed your project was missing a Contributing file. CONTRIBUTING files explain how a developer can contribute to the project - which...
For some reason the `self.defaults` is not properly set when in manual mode so it can not be pickled. Workaround: Change `self.optimizer.defaults` to `optimizer.defaults` and `self.optimizer.param_groups` to `optimizer.param_groups` and `self.optimizer.state`...
Loss becomes nan after training for ~20 steps - loss value stabily decreases and becomes nan with Adam or AdamW optimizers. In case of simple SGD usage it works well.
Do something like this commit: https://github.com/SsnL/pytorch/commit/8f2ddf846de095459341af4e90194ec49f819067
