Mismatch between Hugging Face Pretrained Model Description and GitHub Implementation

[rajaram6052150]

I have noticed a discrepancy between the pretrained model mentioned on the Hugging Face website and its actual implementation available in the GitHub repository. Specifically:

Hugging Face Model Page: https://huggingface.co/IGNF/FLAIR-INC_rgbie_15cl_resnet34-unet/blob/main/FLAIR-INC_rgbie_15cl_resnet34-unet_weights.pth

Error: You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Traceback (most recent call last): File "\?\C:\Users\Rajaram\anaconda3\envs\GPU\Scripts\flair-detect-script.py", line 33, in sys.exit(load_entry_point('flair', 'console_scripts', 'flair-detect')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "e:\gpu\unet\flair-1-main\src\zone_detect\main.py", line 143, in main sliced_dataframe, profile, resolution, model = prepare(config, device) ^^^^^^^^^^^^^^^^^^^^^^^ File "e:\gpu\unet\flair-1-main\src\zone_detect\main.py", line 116, in prepare model = load_model(config) ^^^^^^^^^^^^^^^^^^ File "e:\gpu\unet\flair-1-main\src\zone_detect\model.py", line 84, in load_model model.load_state_dict(state_dict=state_dict, strict=True) File "C:\Users\Rajaram\anaconda3\envs\GPU\Lib\site-packages\torch\nn\modules\module.py", line 2215, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for UperNetForSemanticSegmentation: Missing key(s) in state_dict: "backbone.embeddings.patch_embeddings.projection.weight", "backbone.embeddings.patch_embeddings.projection.bias", "back bone.embeddings.norm.weight", "backbone.embeddings.norm.bias", "backbone.encoder.layers.0.blocks.0.layernorm_before.weight", "backbone.encoder.layers.0.block s.0.layernorm_before.bias", "backbone.encoder.layers.0.blocks.0.attention.self.relative_position_bias_table", "backbone.encoder.layers.0.blocks.0.attention.s elf.relative_position_index", "backbone.encoder.layers.0.blocks.0.attention.self.query.weight", "backbone.encoder.layers.0.blocks.0.attention.self.query.bias ", "backbone.encoder.layers.0.blocks.0.attention.self.key.weight", "backbone.encoder.layers.0.blocks.0.attention.self.key.bias", "backbone.encoder.layers.0.b locks.0.attention.self.value.weight", "backbone.encoder.layers.0.blocks.0.attention.self.value.bias", "backbone.encoder.layers.0.blocks.0.attention.output.de nse.weight", "backbone.encoder.layers.0.blocks.0.attention.output.dense.bias", "backbone.encoder.layers.0.blocks.0.layernorm_after.weight", "backbone.encoder .layers.0.blocks.0.layernorm_after.bias", "backbone.encoder.layers.0.blocks.0.intermediate.dense.weight", "backbone.encoder.layers.0.blocks.0.intermediate.de nse.bias", "backbone.encoder.layers.0.blocks.0.output.dense.weight", "backbone.encoder.layers.0.blocks.0.output.dense.bias", "backbone.encoder.layers.0.block s.1.layernorm_before.weight", "backbone.encoder.layers.0.blocks.1.layernorm_before.bias", "backbone.encoder.layers.0.blocks.1.attention.self.relative_positio n_bias_table", "backbone.encoder.layers.0.blocks.1.attention.self.relative_position_index", "backbone.encoder.layers.0.blocks.1.attention.self.query.weight",

[agarioud]

Hello @rajaram6052150 The pre-trained models available right now on our HF page have been trained with segmentation-models-pytorch. This might indeed not be clear for now, we will update the model cards and model names when releasing pre-trained models trained with HF. So for now your config file should use something like :

model_weights: ../FLAIR-INC_rgbie_15cl_resnet34-unet_weights.pth
model_framework: 
    model_provider: SegmentationModelsPytorch
    HuggingFace:
        org_model: 
    SegmentationModelsPytorch:
        encoder_decoder: resnet34_unet

[rajaram6052150]

Hello @agarioud ,

Thank you for the clarification regarding the pre-trained models. However, I have checked the specified path in the GitHub repository, and it appears that the file FLAIR-INC_rgbie_15cl_resnet34-unet_weights.pth is not present there.

Could you please provide the correct path or a link to download the pre-trained model file? It would be very helpful for continuing with the setup.

Thank you!

Best regards, Rajaram

[agarioud]

You had the right link : https://huggingface.co/IGNF/FLAIR-INC_rgbie_15cl_resnet34-unet/blob/main/FLAIR-INC_rgbie_15cl_resnet34-unet_weights.pth Once downloaded locally, you should ajust the 'model_weights' path from the config file to point it.

[rajaram6052150]

Thank you for help sir @agarioud . Sorry to bother you again . I’ve successfully set up and run inference using the FLAIR-INC_rgbie_15cl_resnet34-unet model after addressing the issue with the model_weights path. However, the output images are all coming out completely black.

Here are the details of my setup and problem:

Setup: Model Weights Path: Correctly set to the downloaded weights file from Hugging Face. Input Image: Path: Set to a georeferenced raster image. Metadata: Driver: GTiff Dtype: uint8 Number of Bands: 5 Image Shape: (5, 512, 512) CRS: EPSG:2154 Configuration Parameters: Output Type: argmax Normalization: Custom Image Size for Detection: 512 Overlap Margin: 128 Issue: Observation: The output raster images are completely black for all processed images. Error Message: No errors are reported during execution, and the model successfully loads and performs inference without issues.

[agarioud]

@rajaram6052150, the flair-detect command is meant to infer over a 'large' area with overlap in inferences. If you image is 512*512 and img_pixels_detection is also 512 but with 128 overlap it may yield some inconsistencies. Also, the output raster are 'black' but do they contain values ?

[rajaram6052150]

Big thanks to you for the guidance sir @agarioud . It turns out that the black images do indeed contain values, representing different classes, and the segmentation results are accurate. Your suggestion helped us identify this.

I also wanted to ask: Will this implementation work with JPG or PNG images as well, or do I need to modify the code for those formats?

[agarioud]

Glad it worked out @rajaram6052150, happy to help. flair-detect won't work with JPG/PNG images as it is meant to work with georeferenced inputs. I haven't planned to add support to this in the near future so you might wan't to either convert your inputs or indeed modify the code. Best,