backpack

backpack copied to clipboard

Second order computations for nn.Upsample

1

Hi I need to compute the approximate hessian for a decoder network. The decoder consists of conv2d and upsample layers. Currently, backpack does not supports nn.Upsample. Since it is a...

FrederikWarburg

AttributeError: 'BatchNorm2d' object has no attribute 'output'

4

I post the full error below. The MWE is a bit long (currently hundreds of lines) and I am still working on it, but is there any specific direction I...

QiyaoWei

Added test case for convolutional layer (when equal to linear layer) and commenting in KFAC methods to compare with KFAC paper.

jabader97

Fix typos.

pitmonticone

Better error messages for BatchNorm

2

Makes the BatchNorm error message more explicit to avoid confusions (https://github.com/f-dangel/backpack/issues/239) and adds an option to ignore the exception. Summary of changes: - Make BatchNorm error message [more explicit](https://github.com/f-dangel/backpack/commit/075834e64dff9f9297c75b4d5b51b9fcc3b5c405#diff-3b87a038266d15ca9769605a81763176d9670389ad809bea3771d43c8cc72c16R23-R28) -...

fKunstner

Parameter `grad`s don't get initialized with `BatchL2Grad` and BatchNorm

7

BatchL2Grad, perhaps naturally, raises an error when it sees a BatchNorm, since batch normalization mixes gradients in a way that makes the individual contribution hard to discern. The error says...

thomasahle

OOM eventually when using create_graph=True with BatchL2Grad

2

I was trying to use my second-order optimizer [ESGD-M](https://github.com/crowsonkb/esgd) with BatchL2Grad in order to collect information on within-batch gradient variance to estimate stochastic noise (think OpenAI's gradient noise scale paper),...

crowsonkb

Extend part of the model

1

Hello, I am wondering is it possible to extend part of the model, if I only want to get the batch gradient of the last several layers? I think `model...

haonan3

AdaptiveAvgPool3d: wait for bug resolved -> delete warning

2

As soon as https://github.com/pytorch/pytorch/issues/60524 is resolved, we can delete our warnings: - backpack/core/derivatives/adaptive_avg_pool_nd.py: delete warning in `check_parameters` - test/core/derivatives/derivatives_test.py: delete test of warning in `test_ea_jac_t_mat_jac_prod` and `test_jac_t_mat_prod`

schaefertim

Website example not working.

1

Hi, the examples (diagonal and KFAC) on the website (https://backpack.pt/) are not working. I believe they are meant for a prior version. Best Felix

Felix-Petersen