Niall Dalton comments

Results 6 comments of


                                            Niall Dalton

Massive VRAM Usage for Feature Ablation and Shapley Value Sampling

Hello, yes it certainly looks like a memory leak. I was not able to reproduce the error with colab originally, but I have now narrowed down the problem. For whatever...

Massive VRAM Usage for Feature Ablation and Shapley Value Sampling

Is it also expected that memory usage increases non-linearly with background image batch size? I can now provide a background image(s) of batch size=1, but a batch size=64 means that...

Massive VRAM Usage for Feature Ablation and Shapley Value Sampling

Hi @vivekmig, I have managed to create a somewhat reproducible example: https://colab.research.google.com/drive/165Zj7wdiFaawbmzGROYqf3__UqY_-DgP?usp=sharing. The issue does not seem to be related to PyTorch Lightning since I can re-create the issue without...

Massive VRAM Usage for Feature Ablation and Shapley Value Sampling

Although I will note, on the actual code I was running originally, the vram usage would explode during the `.attribute(...)` as well, not just being held after, so maybe there...

Massive VRAM Usage for Feature Ablation and Shapley Value Sampling

@NarineK I haven't had (and won't) time to test this lately, but I assume the issue is still there if the code for those methods hasn't been touched. Otherwise, I'm...

Not tf 1.14 compatible

> Configure my device: ubuntu 16.04; python 3.5.2; cuda 9; GPU GTX 1080 TI > I change tf 1.14 to tf 1.12, and install tensorflow_probability version 0.5.0, and it ran...