LED icon indicating copy to clipboard operation
LED copied to clipboard

RuntimeError: Error(s) in loading state_dict for UNetArch:

Open Kidbye20 opened this issue 2 years ago • 6 comments

(led) root@00ee9d88811f:~/LED# python scripts/image_process.py -p checkpoints/pretrained/LED_Pretrain_None_None_CVPR20_Setting_Ratio1-200.pth --data_path images/test/ -opt options/base/network_g/unet.yaml Building network... Loading checkpoint... Loading UNetArch model from checkpoints/pretrained/LED_Pretrain_None_None_CVPR20_Setting_Ratio1-200.pth, with param key: [params]. Traceback (most recent call last): File "scripts/image_process.py", line 115, in image_process() File "/root/anaconda3/envs/led/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "scripts/image_process.py", line 91, in image_process load_network(network_g, args.pretrained_network, param_key='params' if not args.led else 'params_deploy') File "scripts/image_process.py", line 37, in load_network net.load_state_dict(load_net, strict=strict) File "/root/anaconda3/envs/led/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for UNetArch: Missing key(s) in state_dict: "conv1_1.weight", "conv1_1.bias", "conv1_2.weight", "conv1_2.bias", "conv2_1.weight", "conv2_1.bias", "conv2_2.weight", "conv2_2.bias", "conv3_1.weight", "conv3_1.bias", "conv3_2.weight", "conv3_2.bias", "conv4_1.weight", "conv4_1.bias", "conv4_2.weight", "conv4_2.bias", "conv5_1.weight", "conv5_1.bias", "conv5_2.weight", "conv5_2.bias", "conv6_1.weight", "conv6_1.bias", "conv6_2.weight", "conv6_2.bias", "conv7_1.weight", "conv7_1.bias", "conv7_2.weight", "conv7_2.bias", "conv8_1.weight", "conv8_1.bias", "conv8_2.weight", "conv8_2.bias", "conv9_1.weight", "conv9_1.bias", "conv9_2.weight", "conv9_2.bias". Unexpected key(s) in state_dict: "conv1_1.main_weight", "conv1_1.main_bias", "conv1_1.align_weights.0", "conv1_1.align_weights.1", "conv1_1.align_weights.2", "conv1_1.align_weights.3", "conv1_1.align_weights.4", "conv1_1.align_weights.5", "conv1_1.align_biases.0", "conv1_1.align_biases.1", "conv1_1.align_biases.2", "conv1_1.align_biases.3", "conv1_1.align_biases.4", "conv1_1.align_biases.5", "conv1_2.main_weight", "conv1_2.main_bias", "conv1_2.align_weights.0", "conv1_2.align_weights.1", "conv1_2.align_weights.2", "conv1_2.align_weights.3", "conv1_2.align_weights.4", "conv1_2.align_weights.5", "conv1_2.align_biases.0", "conv1_2.align_biases.1", "conv1_2.align_biases.2", "conv1_2.align_biases.3", "conv1_2.align_biases.4", "conv1_2.align_biases.5", "conv2_1.main_weight", "conv2_1.main_bias", "conv2_1.align_weights.0", "conv2_1.align_weights.1", "conv2_1.align_weights.2", "conv2_1.align_weights.3", "conv2_1.align_weights.4", "conv2_1.align_weights.5", "conv2_1.align_biases.0", "conv2_1.align_biases.1", "conv2_1.align_biases.2", "conv2_1.align_biases.3", "conv2_1.align_biases.4", "conv2_1.align_biases.5", "conv2_2.main_weight", "conv2_2.main_bias", "conv2_2.align_weights.0", "conv2_2.align_weights.1", "conv2_2.align_weights.2", "conv2_2.align_weights.3", "conv2_2.align_weights.4", "conv2_2.align_weights.5", "conv2_2.align_biases.0", "conv2_2.align_biases.1", "conv2_2.align_biases.2", "conv2_2.align_biases.3", "conv2_2.align_biases.4", "conv2_2.align_biases.5", "conv3_1.main_weight", "conv3_1.main_bias", "conv3_1.align_weights.0", "conv3_1.align_weights.1", "conv3_1.align_weights.2", "conv3_1.align_weights.3", "conv3_1.align_weights.4", "conv3_1.align_weights.5", "conv3_1.align_biases.0", "conv3_1.align_biases.1", "conv3_1.align_biases.2", "conv3_1.align_biases.3", "conv3_1.align_biases.4", "conv3_1.align_biases.5", "conv3_2.main_weight", "conv3_2.main_bias", "conv3_2.align_weights.0", "conv3_2.align_weights.1", "conv3_2.align_weights.2", "conv3_2.align_weights.3", "conv3_2.align_weights.4", "conv3_2.align_weights.5", "conv3_2.align_biases.0", "conv3_2.align_biases.1", "conv3_2.align_biases.2", "conv3_2.align_biases.3", "conv3_2.align_biases.4", "conv3_2.align_biases.5", "conv4_1.main_weight", "conv4_1.main_bias", "conv4_1.align_weights.0", "conv4_1.align_weights.1", "conv4_1.align_weights.2", "conv4_1.align_weights.3", "conv4_1.align_weights.4", "conv4_1.align_weights.5", "conv4_1.align_biases.0", "conv4_1.align_biases.1", "conv4_1.align_biases.2", "conv4_1.align_biases.3", "conv4_1.align_biases.4", "conv4_1.align_biases.5", "conv4_2.main_weight", "conv4_2.main_bias", "conv4_2.align_weights.0", "conv4_2.align_weights.1", "conv4_2.align_weights.2", "conv4_2.align_weights.3", "conv4_2.align_weights.4", "conv4_2.align_weights.5", "conv4_2.align_biases.0", "conv4_2.align_biases.1", "conv4_2.align_biases.2", "conv4_2.align_biases.3", "conv4_2.align_biases.4", "conv4_2.align_biases.5", "conv5_1.main_weight", "conv5_1.main_bias", "conv5_1.align_weights.0", "conv5_1.align_weights.1", "conv5_1.align_weights.2", "conv5_1.align_weights.3", "conv5_1.align_weights.4", "conv5_1.align_weights.5", "conv5_1.align_biases.0", "conv5_1.align_biases.1", "conv5_1.align_biases.2", "conv5_1.align_biases.3", "conv5_1.align_biases.4", "conv5_1.align_biases.5", "conv5_2.main_weight", "conv5_2.main_bias", "conv5_2.align_weights.0", "conv5_2.align_weights.1", "conv5_2.align_weights.2", "conv5_2.align_weights.3", "conv5_2.align_weights.4", "conv5_2.align_weights.5", "conv5_2.align_biases.0", "conv5_2.align_biases.1", "conv5_2.align_biases.2", "conv5_2.align_biases.3", "conv5_2.align_biases.4", "conv5_2.align_biases.5", "conv6_1.main_weight", "conv6_1.main_bias", "conv6_1.align_weights.0", "conv6_1.align_weights.1", "conv6_1.align_weights.2", "conv6_1.align_weights.3", "conv6_1.align_weights.4", "conv6_1.align_weights.5", "conv6_1.align_biases.0", "conv6_1.align_biases.1", "conv6_1.align_biases.2", "conv6_1.align_biases.3", "conv6_1.align_biases.4", "conv6_1.align_biases.5", "conv6_2.main_weight", "conv6_2.main_bias", "conv6_2.align_weights.0", "conv6_2.align_weights.1", "conv6_2.align_weights.2", "conv6_2.align_weights.3", "conv6_2.align_weights.4", "conv6_2.align_weights.5", "conv6_2.align_biases.0", "conv6_2.align_biases.1", "conv6_2.align_biases.2", "conv6_2.align_biases.3", "conv6_2.align_biases.4", "conv6_2.align_biases.5", "conv7_1.main_weight", "conv7_1.main_bias", "conv7_1.align_weights.0", "conv7_1.align_weights.1", "conv7_1.align_weights.2", "conv7_1.align_weights.3", "conv7_1.align_weights.4", "conv7_1.align_weights.5", "conv7_1.align_biases.0", "conv7_1.align_biases.1", "conv7_1.align_biases.2", "conv7_1.align_biases.3", "conv7_1.align_biases.4", "conv7_1.align_biases.5", "conv7_2.main_weight", "conv7_2.main_bias", "conv7_2.align_weights.0", "conv7_2.align_weights.1", "conv7_2.align_weights.2", "conv7_2.align_weights.3", "conv7_2.align_weights.4", "conv7_2.align_weights.5", "conv7_2.align_biases.0", "conv7_2.align_biases.1", "conv7_2.align_biases.2", "conv7_2.align_biases.3", "conv7_2.align_biases.4", "conv7_2.align_biases.5", "conv8_1.main_weight", "conv8_1.main_bias", "conv8_1.align_weights.0", "conv8_1.align_weights.1", "conv8_1.align_weights.2", "conv8_1.align_weights.3", "conv8_1.align_weights.4", "conv8_1.align_weights.5", "conv8_1.align_biases.0", "conv8_1.align_biases.1", "conv8_1.align_biases.2", "conv8_1.align_biases.3", "conv8_1.align_biases.4", "conv8_1.align_biases.5", "conv8_2.main_weight", "conv8_2.main_bias", "conv8_2.align_weights.0", "conv8_2.align_weights.1", "conv8_2.align_weights.2", "conv8_2.align_weights.3", "conv8_2.align_weights.4", "conv8_2.align_weights.5", "conv8_2.align_biases.0", "conv8_2.align_biases.1", "conv8_2.align_biases.2", "conv8_2.align_biases.3", "conv8_2.align_biases.4", "conv8_2.align_biases.5", "conv9_1.main_weight", "conv9_1.main_bias", "conv9_1.align_weights.0", "conv9_1.align_weights.1", "conv9_1.align_weights.2", "conv9_1.align_weights.3", "conv9_1.align_weights.4", "conv9_1.align_weights.5", "conv9_1.align_biases.0", "conv9_1.align_biases.1", "conv9_1.align_biases.2", "conv9_1.align_biases.3", "conv9_1.align_biases.4", "conv9_1.align_biases.5", "conv9_2.main_weight", "conv9_2.main_bias", "conv9_2.align_weights.0", "conv9_2.align_weights.1", "conv9_2.align_weights.2", "conv9_2.align_weights.3", "conv9_2.align_weights.4", "conv9_2.align_weights.5", "conv9_2.align_biases.0", "conv9_2.align_biases.1", "conv9_2.align_biases.2", "conv9_2.align_biases.3", "conv9_2.align_biases.4", "conv9_2.align_biases.5". 模型权重无法加载

Kidbye20 avatar Dec 05 '23 08:12 Kidbye20

Please refer to #6 which is similar to this problem.

schuy1er avatar Dec 05 '23 08:12 schuy1er

Run inside docker with all installation steps passed.

The same problem with models LED_Deploy_ELD_NikonD850_CVPR20_Setting_Ratio1-200.pth , LED_Deploy_SID_SonyA7S2_CVPR20_Setting_Ratio100-300.pth , LED_Deploy_SID_SonyA7S2_MM22_Setting_Ratio100-300.pth .

At that, all other Deploy models from release 0.1.0 works fine.

In all cases script image_process.py was run with only -p, --data_path and --save_path parameters. Is this correct?

ioctl-user avatar Jan 08 '24 19:01 ioctl-user

If you want to use the LED finetuned network, you need to add --led at the end of the parameters. We mentioned this issue in #6 , but it was discussed in Chinese.

We have mentioned this in the README.md.

https://github.com/Srameo/LED/blob/e052f2774820b449a21da691b14940705465473b/README.md?plain=1#L177

Srameo avatar Jan 09 '24 01:01 Srameo

The reason why it's necessary to specify --led is that the deployed model contains two checkpoints: one before structural reparameterization and one after. When running the image_process.py script, it's required to use the weights from after the reparameterization, hence the need to specify --led.

Srameo avatar Jan 09 '24 01:01 Srameo

Parameter --led is working with LED_Deploy_ELD_NikonD850_CVPR20_Setting_Ratio1-200.pth , LED_Deploy_SID_SonyA7S2_CVPR20_Setting_Ratio100-300.pth, LED_Deploy_SID_SonyA7S2_MM22_Setting_Ratio100-300.pth .

But not working with release 0.1.1 LED+NAFNet_Deploy_SID_SonyA7S2_CVPR20_Setting_Ratio100-300.pth and LED+Restormer_Deploy_SID_SonyA7S2_CVPR20_Setting_Ratio100-300.pth .

ioctl-user avatar Jan 22 '24 11:01 ioctl-user

To use architectures other than UNet, like Restormer/NAFNet. you need to specify the -opt options.

https://github.com/Srameo/LED/blob/75f1081c77d8addeff11d0360da1746216d45499/README.md?plain=1#L198-L199

And we have detailed options for Restormer/NAFNet in commit 75f1081c77d8addeff11d0360da1746216d45499.

https://github.com/Srameo/LED/blob/75f1081c77d8addeff11d0360da1746216d45499/README.md?plain=1#L186

Srameo avatar Jan 22 '24 15:01 Srameo