_IncompatibleKeys failure loading weights.

[samraul]

Hi,

Thank you for sharing the code and weights. I am trying to load the surface normal estimator using the config and weights linked in https://github.com/SysCV/idisc?tab=readme-ov-file#normals:

• https://github.com/SysCV/idisc/blob/main/configs/nyunorm/nyunorm_swinl.json
• https://dl.cv.ethz.ch/idisc/checkpoints/nyunormals_swinlarge.pt And there is an error loading the weights:

import json
from idisc.models.idisc import IDisc

NORMALS_CONFIG_FILE = "models/nyunorm_swinl.json"
NORMALS_MODEL = "models/nyunormals_swinlarge.pt"
with open(NORMALS_CONFIG_FILE, "r") as f:
    config = json.load(f)

model = IDisc.build(config=config)
model.load_pretrained(NORMALS_MODEL)

-> Encoder is pretrained from: https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_large_patch4_window7_224_22k.pth Loading pretrained info: _IncompatibleKeys(missing_keys=['norm0.weight', 'norm0.bias', 'norm1.weight', 'norm1.bias', 'norm2.weight', 'norm2.bias', 'norm3.weight', 'norm3.bias'], unexpected_keys=['norm.weight', 'norm.bias', 'head.weight', 'head.bias', 'layers.0.blocks.1.attn_mask', 'layers.1.blocks.1.attn_mask', 'layers.2.blocks.1.attn_mask', 'layers.2.blocks.3.attn_mask', 'layers.2.blocks.5.attn_mask', 'layers.2.blocks.7.attn_mask', 'layers.2.blocks.9.attn_mask', 'layers.2.blocks.11.attn_mask', 'layers.2.blocks.13.attn_mask', 'layers.2.blocks.15.attn_mask', 'layers.2.blocks.17.attn_mask'])

It also happens with Swin-Large for NYU2 Depth Estimation. Am I missing something?

Thank you.

[liviniuk]

In the paper, predictions were averaged over random crops of the test images. Maybe this is why their images didn't have a checkerboard effect. In my experiments, removing ASPP would eliminate the checkerboard (but also decrease accuracy a bit).