Thomas George

In general, if you can afford the compute needed to calculate the spectrum of the Fisher using a PMatDense, then go for it. In practice, on actual neural networks, PMatDense...

Hi, that's not really a fix but I updated the docstring of the FIM_MonteCarlo which incorrectly mentionned a 'regression' variant. I am not sure what you would expect as a...

I am closing this as there is no straightforward way of performing a MC estimate of the FIM in the case of a gaussian model.

Hello, Unfortunately it is not really clear what to do with BatchNorm layers when trying to apply KFAC: 1. factorize batch norm parameters in some way? 2. use the full...

For your 2nd questions, to speed up computation you can: - reduce the dataset size - use a GPU - change to a more efficient representation (e.g. PMatKFAC) - reduce...

That does not exactly answer your question, but depending on why you actually want the eigendecomposition, I would rather recommend you go with a power method such as the Lanczos...

Whoops you are right that it has disappeared in a previous commit. In the meantime you can find it here: https://github.com/tfjgeorge/nngeometry/blob/82984cf20fe94752a794bc26bc19e29d0b3e589e/examples/FIM%20for%20EWC.ipynb I plan on implementing a representation with a mix...

I am not sure. If you found out, please share the answer :-)

Hi, I am not so sure what you are trying to achieve. The FIM can be computed if you have a parametrized model that defines a probability distribution. From a...

Then the same argument as above: - depending on your task, choose regression or classification - if too many outputs, use the Monte Carlo estimator instead, or use the second...