hyperlight
hyperlight copied to clipboard
JJGO
Modular and intuitive Hypernetworks in Pytorch
Hi @JJGO , thanks a lot for sharing the code. In chapter 4 **Output Encoding** part of the paper, it mentioned the method introduces a set of learnable parameters \theta_0...
Hi @JJGO , thanks a lot for sharing the code! If I understand correctly, this line should be outside the _for_ loop?
Hi, thank you for your interesting research. I have a few questions regarding the code and paper: 1. In the repository, I cannot find the code for the output residual...
I'm considering implementing a HyperNet using Hyperlight in the below code, but I don't know about how to input the previous hidden state during closedloop prediction. What would be the...
Hello, Thank you for open-sourcing this repository! If the main architecture is RNN, how should I implement batch processing?
Hi, Thanks for open-sourcing this repository! If the main architecture is MLP, how should I set the parameters of the HyperNet ？ or what should I pay attention to ?
[Vries et al. 2017](https://proceedings.neurips.cc/paper_files/paper/2017/file/6fab6e3aa34248ec1e34a4aeedecddc8-Paper.pdf) condition a network by hypernetizing BatchNorm layers only. Presently, specifying a `BatchNorm2d` causes `ValueError: Fan in and fan out can not be computed for tensor with...
This example in the readme results in failed type checks: ```py modules = hl.find_modules_of_type(mainnet, [nn.Conv2d]) self.mainnet = hl.hypernetize(mainnet, modules=modules) ```
Is there any approach to add layer norm / batch norm / instance norm inside the HyperNet class?
