[Test] PTQ for QAT initialization for Torch samples
Changes
Test for QAT after PTQ for torch classifications
Reason for changes
To check PTQ quantized model is trainable in torch
Related tickets
129584
Results
| Testcase | Task | num epoch for fine-tuning now | num epoch for fine-tuning before |
|---|---|---|---|
| resnet50_imagenet_int8 | Classification | 0 | 1 |
| squeezenet1_1_imagenet_int8 | Classification | 0 | 5 |
| resnet50_imagenet_int8_per_tensor | Classification | 0 | 1 |
| inception_v3_imagenet_int8 | Classification | 0 | 1 |
| mobilenet_v3_small_imagenet_int8 | Classification | 5 | 5 |
| mobilenet_v2_imagenet_int8 | Classification | 1 | 5 |
| squeezenet1_1_imagenet_int8_per_tensor | classification | 0 | 5 |
| mobilenet_v2_imagenet_int8_per_tensor | Classification | 1 | 10 |
| ssd512_vgg_voc_int8 | Object Detection | 0 | 78 |
| ssd300_vgg_voc_int8 | Object Detection | 0 | 24 |
| unet_mapillary_int8 | Semantic Segmentation | 0 | 300 |
| unet_camvid_int8 | Semantic Segmentation | 0 | 10 |
| icnet_camvid_int8 | Semantic Segmentation | 0 | 300 |
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 83.57%. Comparing base (
12f4720) to head (e4bd12f). Report is 12 commits behind head on develop.
Additional details and impacted files
@@ Coverage Diff @@
## develop #2404 +/- ##
===========================================
- Coverage 91.18% 83.57% -7.62%
===========================================
Files 492 494 +2
Lines 45096 45377 +281
===========================================
- Hits 41123 37923 -3200
- Misses 3973 7454 +3481
see 110 files with indirect coverage changes
| Flag | Coverage Δ | |
|---|---|---|
| COMMON | ? |
|
| ONNX | 34.66% <ø> (+<0.01%) |
:arrow_up: |
| OPENVINO | ∅ <ø> (∅) |
|
| TENSORFLOW | 30.10% <ø> (+0.23%) |
:arrow_up: |
| TORCH | 65.92% <ø> (+0.16%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Components | Coverage Δ | |
|---|---|---|
| common | 88.89% <ø> (-4.88%) |
:arrow_down: |
| torch | 93.48% <ø> (-0.12%) |
:arrow_down: |
| tensorflow | 93.74% <ø> (ø) |
|
| onnx | 93.09% <ø> (+0.03%) |
:arrow_up: |
| openvino | 25.76% <ø> (-68.31%) |
:arrow_down: |
| ptq | 64.84% <ø> (-25.38%) |
:arrow_down: |
@daniil-lyakhov do you have a build in CI with these tests?
torch_weekly/31 - failed
torch_weekly/31 - failed
torch_weekly/38 - in progress
Build torch_weekly/42/ is successfull CC: @AlexanderDokuchaev