Ahmedest61
error when train
HI,thanks for your work!
when i train as the guide.
i run python main.py --mode=cluster --arch=vgg16-12 --pooling=netvlad --density_L=10 --num_clusters=64
it's ok
and then i run python main.py --mode=train --arch=vgg16-12 --pooling=netvlad --density_L=10 --num_clusters=64
find the error
Traceback (most recent call last): File "/home/Desktop/Link to projects/MultiRes-NetVLAD-main/main.py", line 594, in <module> raise ValueError('pca_layer can be used only during Testing') ValueError: pca_layer can be used only during Testing
can u help me? thanks o lot
Hi, Thank you for pointing this out. I have removed the bug. Can you rerun it?
Yes,it works now.
But when i try to train it on 4GPU
after
python main.py --mode=cluster --arch=vgg16-12 --pooling=netvlad --density_L=10 --num_clusters=64
I run
python main.py --mode=train --arch=vgg16-12 --pooling=netvlad --density_L=10 --num_clusters=64 --nGPU 4
and change the cacheBatchSize as 24,48 or other number(default=12),it all failed
RuntimeError: CUDA error: the launch timed out and was terminated CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
while my GPU usd is only 15%,why?
i think maybe 1451(num of pic) can't divisible by 2?
can u help me ? thanks