Redundancy in the dataset with zone-detect

[NickBear-star]

Hello,

The dataset processing step produces redundant image patches. This should not affect the final results, but induces the model to be run twice on some parts of the source image.

For exemple, the source image IMG_063487.tif has a size of 512 * 512 px (5 channels RGBIE):

• the image bounds are {'left': 318569.6, 'bottom': 6830715.2, 'right': 318672.0, 'top': 6830817.6}
• the parameters are set as following in the YAML file:
  • img_pixels_detection: 512
  • margin: 128
  • model_weights: FLAIR-INC_rgbie_15cl_resnet34-unet_weights.pth

Consequently, the source image is splitted into 9 patches in the Torch dataset:

id     left     bottom     right        top 
-- --------  ---------  --------  ---------
0  318569.6  6830715.2  318620.8  6830766.4
1  318569.6  6830766.4  318620.8  6830817.6
2  318569.6  6830766.4  318620.8  6830817.6
3  318620.8  6830715.2  318672.0  6830766.4
4  318620.8  6830766.4  318672.0  6830817.6
5  318620.8  6830766.4  318672.0  6830817.6
6  318620.8  6830715.2  318672.0  6830766.4
7  318620.8  6830766.4  318672.0  6830817.6
8  318620.8  6830766.4  318672.0  6830817.6

In practice, you only need 4 patches to cover the source image:

id     left     bottom     right        top 
-- --------  ---------  --------  ---------
0  318569.6  6830715.2  318620.8  6830766.4
1  318569.6  6830766.4  318620.8  6830817.6
3  318620.8  6830715.2  318672.0  6830766.4
4  318620.8  6830766.4  318672.0  6830817.6

Is there a reason hidden behind this redundancy?

[agarioud]

hello @NickBear-star,

flair-detect is meant to be used with larger images by including an overlap factor (detection size - margin).

This overlap (or redundancy) in inferences is intentional to avoid known edge effects in detections by merging the overlapping parts.

It looks like you are inferring on a single 512x512 image with a margin of 128. In this case, you can use the flair command for direct prediction without training.

[NickBear-star]

Hello,

I think you miss my point. Even with larger images, the slicing process produces duplicated patches (I am really talking about duplication, not overlapping). For example, with an image of 10k * 10k px, i still find 81 duplicated patches. As far as I have analyzed the code, those duplications just induce more computation than needed. At the end of the process, the same result is just written twice in the output TIF file.

I agree this is not a big issue. As I want to process a large set of files, I'd like to limit those computation leaks, however.

[agarioud]

Apologies, I've overlooked the issue. I can indeed reproduce some redundancy, but it only appears on the first top row. Can you confirm this?

[NickBear-star]

The redundancy may occur on both dimension (first top row and last right column), depending on image size and the patch size and margin. Here is the fix I propose:

from itertools import product

def set_patch_px_coordinates(raster_size: Size, patch_size, Size, margin: Size) -> list[BoundingBox]:
    """
    Slice the source raster in patches of identical size, then return their px coordinates
    By convention, x is along width and y is along height
    """

    useful_patch_size = Size(w=patch_size.w - 2 * self.margin.w, h=patch_size.h - 2 * self.margin.h)

    def _get_max_patch(_raster_size: int, _useful_patch_size: int) -> int:
        """
        Calculate the max number of patches that can fit in a dimension
        """
        _max_patch = _raster_size // _useful_patch_size
        _max_patch = _max_patch + 1 if  _raster_size % _useful_patch_size > 0 else _max_patch
        return _max_patch

    def _get_patch_coordinates(_raster_size: int, _index: int, _useful_patch_size: int) -> (int, int):
        """
        Calculate the (min, max) coordinates of a patch along a dimension
        """
        _min = _index * _useful_patch_size
        _max = (_index + 1) * useful_patch_size.w - 1

        if _max > _raster_size:
            _min = _raster_size - _useful_patch_size
            _max = _raster_size - 1

        return _min, _max


    patch_index = [
        range(0, _get_max_patch(raster_size.w, useful_patch_size.w), 1),
        range(0, _get_max_patch(raster_size.h, useful_patch_size.h), 1)
    ]

    patches = list()
    for i, j in product(patch_index[0], patch_index[1]):
        x_min, x_max = _get_patch_coordinates(raster_size.w, i, useful_patch_size.w)
        y_min, y_max = _get_patch_coordinates(raster_size.h, j, useful_patch_size.h)
        patch = BoundingBox(x_min, x_max,y_min, y_max)
        patches.append(patch)

    return patches

Size, BoundingBox can be generic data classes to manage 1D or 2D objects. I believe a better approch to slice a raster would be an adaptative step. I keep that algorithm for the moment.

[agarioud]

Thanks for confirming that the issue only occurs on the edges. I appreciate your suggested replacement and will take a look at it.