Pruning support for chunk ops

[sooahleex]

Changes

• Add layer attribute and functions for chunk/split ops
• mask propagation for split/chunk ops

Reason for changes

• To support pruning for chunk operation in HRNet
• Support both split and chunk operation

Related tickets

86694

Tests

TODO

[sooahleex]

@daniil-lyakhov This is the draft version

[daniil-lyakhov]

@sooahleex, thank you for your contribution!

Please don't forget to update correspondent nncf tensorflow part as well

[daniil-lyakhov]

Please add tests to torch/pruning/filter_pruning/test_algo.py

[sooahleex]

Please add tests to torch/pruning/filter_pruning/test_algo.py

@daniil-lyakhov I added more cases for split/chunk models in tests/torch/pruning/helpers.py. ex.

[sooahleex]

run pytorch pre-commit tests

[daniil-lyakhov]

General comment:

Masks matching by input shapes is implicit method of matching, I suggest you to

1. Explicitly have SymbolicMask (in case op has one ouput edge) or {Nodename: SymbolicMask} (in case op has several output edges) as value kept in node.data['output_mask']
2. Use get_input_masks everywhere to get input masks for node resolving all dependencies inside this util function

[ljaljushkin]

I wonder how the algorithm will work when chunk axis doesn't match channel axis, according to which pruning occurs. What will happen with pruning mask and propagation algorithm? I think it makes sense to cover in tests.

[daniil-lyakhov]

@sooahleex, to test this please update test_model_pruning_analysis.py with model/models which have split operations described in previous comment

[sooahleex]

I fix the error which occurred due to PR.

[sooahleex]

@ljaljushkin I added the axis=-1 case through SplitModel in tests/torch/pruning/helpers.py. Also, in this case, axis will not be set as -1 but be 3. It is the same logic of the axis setting in the concat method. If you think the current model is not enough, please let me know how to check it more.

[sooahleex]

run pytorch pre-commit tests

[sooahleex]

I added the case of Split->Reshape model and Identity split case as corner cases. Split->Reshape model Identity Split model

[sooahleex]

We will cover the reshape check in the different pr related to 90584 ticket. For that ticket and pr, some tests in this pr will be updated for _is_not_mixing_dim.

[daniil-lyakhov]

We will cover the reshape check in the different pr related to 90584 ticket. For that ticket and pr, some tests in this pr will be updated for _is_not_mixing_dim.

Related to 90976 ticket, the 90584 ticket is not related to this PR

[ljaljushkin]

Have started sota eval (torch 56 and tf 87) jobs from nightly validation to make sure it doesn't break anything.

[sooahleex]

sota eval tf 89 jobs for nightly validation passed.

[daniil-lyakhov]

@ljaljushkin, can we merge it?

[ljaljushkin]

@ljaljushkin, can we merge it?

SOTA eval for TF and PT is OK, except some known issues that are not related to the PR. Unfortunately, pre-commit for Pytorch is red due to some infrastructure issue. I'd not merge until we make sure it's passing.

[daniil-lyakhov]

run pytorch pre-commit tests

[ljaljushkin]

Pre-commit for PyTorch is not working due to infra issue. We've reported it to CI team. Have to wait until it's fixed and green

[ljaljushkin]

run pytorch pre-commit tests