ctensmeyer
binarization_dibco container fails
Hello! I'm trying to run your binarization_dibco.py tool in nvidia-docker container and getting following error:
F1021 20:23:03.879026 1 math_functions.cu:81] Check failed: error == cudaSuccess (77 vs. 0) an illegal memory access was encountered *** Check failure stack trace: *** Loading Image Computing RD features Concating inputs Preprocessing (489, 2625, 4) Tiling input Starting predictions for network 1/5 Progress 2% Progress 5% Progress 7% Progress 10% Progress 12% Progress 15% Progress 17% Progress 20% Progress 23% Progress 25% Progress 28% Progress 30% Progress 33% Progress 35% Progress 38% Progress 41% Progress 43% Progress 46% Progress 48% Progress 51% Progress 53% Progress 56% Progress 58% Progress 61% Progress 64% Progress 66% Progress 69% Progress 71% Progress 74% Progress 76% Progress 79% Progress 82% Progress 84% Progress 87% Progress 89% Progress 92% Progress 94% Progress 97% Progress 100% Reconstructing whole image from binarized tiles Starting predictions for network 2/5.
binarization_plm.py doesn't fail and works great :)
It's possible I have a bug somewhere in my Caffe code that causes the illegal access on some GPUs. What GPU are you using?
On Sun, Oct 21, 2018 at 2:27 PM Mikhail Masyagin [email protected] wrote:
Hello! I'm trying to run your binarization_dibco.py tool in nvidia-docker container and getting following error:
F1021 20:23:03.879026 1 math_functions.cu:81] Check failed: error == cudaSuccess (77 vs. 0) an illegal memory access was encountered *** Check failure stack trace: *** Loading Image Computing RD features Concating inputs Preprocessing (489, 2625, 4) Tiling input Starting predictions for network 1/5 Progress 2% Progress 5% Progress 7% Progress 10% Progress 12% Progress 15% Progress 17% Progress 20% Progress 23% Progress 25% Progress 28% Progress 30% Progress 33% Progress 35% Progress 38% Progress 41% Progress 43% Progress 46% Progress 48% Progress 51% Progress 53% Progress 56% Progress 58% Progress 61% Progress 64% Progress 66% Progress 69% Progress 71% Progress 74% Progress 76% Progress 79% Progress 82% Progress 84% Progress 87% Progress 89% Progress 92% Progress 94% Progress 97% Progress 100% Reconstructing whole image from binarized tiles Starting predictions for network 2/5.
binarization_plm.py doesn't fail and works great :)
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ctensmeyer/binarization_2017/issues/3, or mute the thread https://github.com/notifications/unsubscribe-auth/AC2mpNX3vH_Tf-X1gEMCFp9Ri-tBlnaTks5unNiTgaJpZM4Xypq4 .
Tried to run without GPU, but ran out of memory: I1023 03:51:57.924324 1 net.cpp:259] Network initialization done. I1023 03:51:57.924329 1 net.cpp:260] Memory required for data: 503054336 F1023 03:52:49.289590 1 syncedmem.hpp:33] Check failed: *ptr host allocation of size 5435817984 failed *** Check failure stack trace: *** Loading Image Computing RD features Concating inputs Preprocessing (722, 1087, 4) Tiling input Starting predictions for network 1/5
I have 8 GIBs of RAM and Intel Core I7-7700HQ. My system is Linux Mint 19. When I started container, I had 6.5 GIBs of free RAM.
Hello!
I am using docker to run with DIBCO2017 dataset but failure and got a similar issue.
I0410 07:38:22.774055 1 net.cpp:259] Network initialization done. I0410 07:38:22.774060 1 net.cpp:260] Memory required for data: 503054336 F0410 07:38:22.959889 1 math_functions.cu:81] Check failed: error == cudaSuccess (77 vs. 0) an illegal memory access was encountered *** Check failure stack trace: *** Loading Image Computing RD features Concating inputs Preprocessing (1350, 2397, 4) Tiling input Starting predictions for network 1/5
Hello, I have the same problem as mentioned above. I've changed the batch size to 8 (Issue https://github.com/shihenw/convolutional-pose-machines-release/issues/6#issuecomment-218475367)and it's working for most of the images(although giving washed out text) but it's still failing for some images.
Anyone found a solution?
