variational-dropout

variational-dropout copied to clipboard

Wrong logalpha parameterisation for sparsifying variational dropout

1

Here: https://github.com/gngdb/variational-dropout/blob/8d18ec2c7a62ed029eb06a6a7638498364b48253/varout/layers.py#L334-L340 Should depend on `w` as here: https://github.com/BayesWatch/tf-variational-dropout

gngdb

Can only currently work with 2D tensors, but it should be easy to extend them to larger tensors for convolutional layers.

gngdb

Empirical variances don't match Table 1

5

The results in table 1 are very bad, because it actually looks like the variance is higher for the local parameterization versus the single weight samples. Possible reasons for this:...

gngdb

This is supposed to modular, but I haven't provided any instructions on how to use it in other networks yet. I'll do this once I can replicate the paper's results...

gngdb