Instance Segmentation Model refactoring

[kprokofi]

Summary

• [x] Remove code duplication for torchvision MascRCNN
• [x] Add factory template for all models
• [x] Move loss out of the heads
• [x] Remove mmdict like approach (done partially due to overlap with detection task)
• [x] Update/add unit tests

How to test

Checklist

• [ ] I have added unit tests to cover my changes.​
• [ ] I have added integration tests to cover my changes.​
• [ ] I have ran e2e tests and there is no issues.
• [ ] I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• [ ] I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• [ ] I have linked related issues.

License

• [ ] I submit my code changes under the same Apache License that covers the project. Feel free to contact the maintainers if that's a concern.
• [ ] I have updated the license header for each file (see an example below).

# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[sungchul1]

@kprokofi could you remove the duplicated one? https://github.com/openvinotoolkit/training_extensions/pull/3783#discussion_r1703456339 https://github.com/openvinotoolkit/training_extensions/blob/a5470fc3665228be8c3fde0bdc0728b11b26c057/src/otx/algo/instance_segmentation/layers/transformer.py#L88-L93

[kprokofi]

This PR will be opened after https://github.com/openvinotoolkit/training_extensions/pull/3860 merged

[sungchul1]

@kprokofi Could you check the performance before and after this PR?

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 89.94197% with 52 lines in your changes missing coverage. Please review.

Project coverage is 81.29%. Comparing base (709d841) to head (e499e1d). Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/otx/algo/detection/necks/fpn.py	16.66%	10 Missing :warning:
src/otx/algo/instance_segmentation/maskrcnn.py	82.35%	9 Missing :warning:
.../otx/algo/instance_segmentation/losses/roi_loss.py	81.81%	8 Missing :warning:
.../otx/algo/instance_segmentation/heads/bbox_head.py	93.93%	6 Missing :warning:
src/otx/algo/modules/transformer.py	86.36%	6 Missing :warning:
src/otx/algo/instance_segmentation/rtmdet_inst.py	73.33%	4 Missing :warning:
...c/otx/algo/instance_segmentation/heads/roi_head.py	95.23%	2 Missing :warning:
...go/instance_segmentation/heads/rtmdet_inst_head.py	92.85%	2 Missing :warning:
src/otx/algo/instance_segmentation/maskrcnn_tv.py	92.85%	2 Missing :warning:
...go/instance_segmentation/segmentors/maskrcnn_tv.py	93.75%	2 Missing :warning:
... and 1 more

Additional details and impacted files

@@            Coverage Diff            @@
##           develop    #3865    +/-   ##
=========================================
  Coverage    81.28%   81.29%            
=========================================
  Files          283      282     -1     
  Lines        27171    27024   -147     
=========================================
- Hits         22087    21968   -119     
+ Misses        5084     5056    -28     

Flag	Coverage Δ
py310	81.27% <89.94%> (+0.06%)	:arrow_up:
py311	80.89% <89.94%> (-0.20%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[sungchul1]

@kprokofi could you update all of the docstrings? It looks outdated or mismatched. And for rtmdet_inst, need to check in some things it is written as inst (rtmdet_inst), and in some things it is written as ins (rtmdet_ins_head).

[kprokofi]

@kprokofi could you update all of the docstrings? It looks outdated or mismatched. And for rtmdet_inst, need to check in some things it is written as inst (rtmdet_inst), and in some things it is written as ins (rtmdet_ins_head).

I aligned names and fixed found problems with docstrings

[kprokofi]

@kprokofi Could you check the performance before and after this PR?

pref tests are running

[kprokofi]

Strange unit test fail, Point entity for VPT task has no attribute "canvas_size". I can't reproduce it directly with pytest, only via "tox" and running all tests...

[sungchul1]

@kprokofi I didn't reproduce unit test issue on both pytest and tox in local, need to check.

[kprokofi]

@kprokofi I didn't reproduce unit test issue on both pytest and tox in local, need to check.

I found the problem. I added accidently from src.otx... import ImageInfo, instead of otx.... import ImageInfo. That leaded to some import errors maybe and misled about the underlying problem.

[kprokofi]

Perf tests:

data_group		all/develop	all/develop	all/refactoring	all/refactoring
metric		test/f1-score	test/f1-score	test/f1-score	test/f1-score
stat		mean	std	mean	std
task	model
instance_segmentation	all	0.51328727	0.13603715	0.506485189	0.147367714
instance_segmentation	maskrcnn_efficientnetb2b	0.530471097	0.064085943	0.537792064	0.072942254
instance_segmentation	maskrcnn_r50	0.489827528	0.099895469	0.4898914	0.104490717
instance_segmentation	maskrcnn_r50_tv	0.54778025	0.133141648	0.554633201	0.138719154
instance_segmentation	maskrcnn_swint	0.589406501	0.100718406	0.547774035	0.137834792
instance_segmentation	rtmdet_inst_tiny	0.408950974	0.194595874	0.402335247	0.212427851

I need to look at SwinT if the drop is random or not.

[kprokofi]

I tested one more time and got to conclusion, that some fluctuation is normal. Develop (5 runs):

	small	small
	test/f1-score	test/f1-score
	mean	std
model
maskrcnn_swint	0.489417503	0.013157227

This branch (5 runs):

	small	small
	test/f1-score	test/f1-score
	mean	std
model
maskrcnn_swint	0.507344836	0.079074585

I remeasured develop one more time (5 runs, averaged):

	small	small
	test/f1-score	test/f1-score
	mean	std
model
maskrcnn_swint	0.500201058	0.032963669

So, even setting seeds doesn't guarantee the full reproducibility. Next, even with results above, 0.589 +- 0.1 ~= 0.548 +- 0.137

I also compared the code once again and checked that snapshot from develop produces the same result on that branch. So, I hope, there is no difference in terms of performance.

I propose to merge this PR before https://github.com/openvinotoolkit/training_extensions/pull/3917

[kprokofi]

Thanks for your hard work! I have a minor question, should RTMDetInst inherit ExplainableOTXInstanceSegModel even if explainable is not supported?

When I implemented RTDetr, in ExplainableOTXDetModel was some required functionality. I guess, RTMDet has the same problem. We should refactor this in the future and maybe merge ExplainableModel with base OTXModel to avoid any discrepancy.