MultiNet icon indicating copy to clipboard operation
MultiNet copied to clipboard

train multinet2.json out of memory seriously

Open cs-heibao opened this issue 7 years ago • 0 comments

@MarvinTeichmann I've tried to train multinet2.json with one gpu(1080Ti), and it seems collapse, have you use multiple GPUs? the information as follows: 2018-12-13 17:56:15.831145: W tensorflow/core/common_runtime/bfc_allocator.cc:219] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.64GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2018-12-13 17:56:15.869555: W tensorflow/core/common_runtime/bfc_allocator.cc:219] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.73GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2018-12-13 17:56:16,014 INFO Segmentation Loss was used.

cs-heibao avatar Dec 13 '18 09:12 cs-heibao