CIHP_PGN Not working for 1080x1080 images

The pre-trained model works for smaller images(300,200) but when training with (1080,1080) images gives the error: OP_REQUIRES failed at concat_op.cc:153 : Resource exhausted: OOM when allocating tensor with shape[2,2560,169,169] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc

Please suggest a solution as I need to use 1080x1080 image for 3D reconstruction. I am working with 4 GPUs and enough RAM for the process.

Jun 10 '19 13:06 rishilss99

Having the same issue, @Engineering-Course is there a possible solution for this?

Jul 16 '19 08:07 DomhnallBoyle

So the code fails for multiple 1080x1080 images withe the aforementioned error but works with only one image. Modify val_id.txt and val.txt with only one image. Think it is possible to modify the code to store images one by one instead of all together and crash the GPU memory.

Mar 31 '20 04:03 YifuTao

I have the same question，how to solve this problem?

Aug 06 '20 05:08 lll-gen