HRNet-Semantic-Segmentation icon indicating copy to clipboard operation
HRNet-Semantic-Segmentation copied to clipboard

performance gap with pytorch 0.4.1: mIoU 79.99

Open Wangzhuoying0716 opened this issue 6 years ago • 10 comments

Sorry to bother you! I tried to implement the training process using the seg_hrnet_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml. But it came out that the validation mIoU for single-scale and no-flip is 79.99, almost 1% lower than your result shown(80.9). I wonder if this is normal variation or I did something wrong ? Here is the training log: https://pan.baidu.com/s/1utKUVuBEjDBtfgOk7A-5sQ 【passward: jckv】 Thank you!

Wangzhuoying0716 avatar Oct 28 '19 06:10 Wangzhuoying0716

Hi, @Wangzhuoying0716 would you mind sharing your training settings, I have been trying to train it, but it always gets stuck at this point image

I don't know where I'm doing something wrong.

my training settings: Pytorch 0.4.1 python 3.7 OpenCV 3.4.2.17 rest is according to requirements.txt

Hussainflr avatar Nov 02 '19 14:11 Hussainflr

@Hussainflr If there is no error message but just gets stuck, I guess it's because the memory is too full to keep on? You can check the memory usage when it is stuck to see whether it's almost full. As for me, I just used 4xTitan Xp GPU cards and the setting in seg_hrnet_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml.

Wangzhuoying0716 avatar Nov 03 '19 04:11 Wangzhuoying0716

@Hussainflr I also used Pytorch 0.4.1 and python 3.7

Wangzhuoying0716 avatar Nov 03 '19 04:11 Wangzhuoying0716

@Wangzhuoying0716 Sorry for late reply. I checked your log. When training on cityscapes, we use the "class balance" in the loss function. You can change the config and set the class balance as True.

sunke123 avatar Dec 11 '19 03:12 sunke123

Hi @sunke123, LOSS.CLASS_BALANCE defined in lib/config/default.py, but is there any place to use it? and i found, the loss: criterion = CrossEntropy(ignore_label=config.TRAIN.IGNORE_LABEL, weight=train_dataset.class_weights), use class weighs, which doesn't matter with CLASS_BALANCE param

whiteinblue avatar Jan 09 '20 04:01 whiteinblue

@whiteinblue Class weights are used to train the model on Cityscapes. If you don't want to use it, you can change it in the yaml.

sunke123 avatar Jan 09 '20 05:01 sunke123

@sunke123 thank you for your reply, and i confuse, it's where that the param LOSS.CLASS_BALANCE used ?

whiteinblue avatar Jan 10 '20 01:01 whiteinblue

Hi, @Wangzhuoying0716 would you mind sharing your training settings, I have been trying to train it, but it always gets stuck at this point image

I don't know where I'm doing something wrong.

my training settings: Pytorch 0.4.1 python 3.7 OpenCV 3.4.2.17 rest is according to requirements.txt

Hello, i met same problem, have you solved it?

XuShoweR avatar Jan 10 '20 07:01 XuShoweR

@whiteinblue Class_balance is used in the loss function.

sunke123 avatar Jan 23 '20 03:01 sunke123

So where that the param LOSS.CLASS_BALANCE used? I read the code, it seems that no matter what is CLASS_BALANCE param, the balance loss is used in the cityscape dataset.

purse1996 avatar Dec 01 '20 07:12 purse1996