open_model_zoo icon indicating copy to clipboard operation
open_model_zoo copied to clipboard

[E-103104] - Port G-API demos to API2.0

Open DariaMityagina opened this issue 2 years ago • 3 comments

Ticket:

103104

Description:

G-API Background Subtraction Demo, Classification Benchmark C++ G-API Demo, G-API Gaze Estimation Demo and other - fail on NPU with the following error: dynamic shape issues

get_shape was called on a descriptor::Tensor with dynamic shape
  • OpenVINO is actively removing API 1.0 from their code
  • Both G-API framework and G-API demos use API 1.0 --> which leads to disruption of certain functionality Conclusion: We need to port the G-API demos to API2.0 to make them work.

Affected demos:

Passing:

  • [x] classification_benchmark_demo
  • [x] face_detection_mtcnn_demo
  • [x] interactive_face_detection_demo
  • [x] gaze_estimation_demo OK: Works with a single image and with a video input. BUT: But still has issues in case folder with images is provided (in CI) - WIP Failed to copy data from cv::Mat to ov::Tensor. Data type or number of elements mismatch. ov::Tensor: u8 [1,490,1183,3] and cv::Mat: 8UC3 1180x490 It seems to fail if the input folder contains images of different sizes. It works fine with the first image from the folder and does not work with the second (if it is a different size). The issue seems to come from govbackend, I will open a PR shortly.

Failing still:

  • [ ] gesture_recognition_demo person-detection-asl-0001/FP16/person-detection-asl-0001.xml is indeed dynamic. Will disable this one until we support dynamism. G-API demo also fails on CPU after transition to API2.0 for some reason (Segmentation fault (core dumped)). Benchmark_app, on the other hand, works fine with this model. Python demo works on CPU as well. AR Daria: create ticket for G-API demo and investigate.
<layer id="1826" name="1632" type="NonMaxSuppression" version="opset9">
			<data box_encoding="corner" sort_result_descending="false" output_type="i64" />
			<input>
				<port id="0" precision="FP32">
					<dim>1</dim>
					<dim>1534</dim>
					<dim>4</dim>
				</port>
				<port id="1" precision="FP32">
					<dim>1</dim>
					<dim>1</dim>
					<dim>1534</dim>
				</port>
				<port id="2" precision="I64" />
				<port id="3" precision="FP32" />
				<port id="4" precision="FP32" />
			</input>
			<output>
				<port id="5" precision="I64" names="1632">
					<dim>-1</dim>
					<dim>3</dim>
				</port>
				<port id="6" precision="FP32">
					<dim>-1</dim>
					<dim>3</dim>
				</port>
				<port id="7" precision="I64">
					<dim>1</dim>
				</port>
			</output>
		</layer>
  • [ ] background_subtraction_demo [ ERROR ] OpenCV(4.8.0-dev) ..\modules\gapi\src\backends\ov\govbackend.cpp:799: error: (-215:Assertion failed) cv::util::holds_alternativecv::GMatDesc(input_meta) in function 'cv::gimpl::ov::PrePostProcWrapper::cfgPreProcessing' - WIP

Related PRs:

Part 1: https://github.com/openvinotoolkit/open_model_zoo/pull/3889 - Merged Part 2: https://github.com/openvinotoolkit/open_model_zoo/pull/3891 - In review

DariaMityagina avatar Dec 21 '23 16:12 DariaMityagina

TODO:

  • [ ] fix [ ERROR ] Failed to copy data from cv::Mat to ov::Tensor. Data type or number of elements mismatch. ov::Tensor: u8 [1,480,640,3] and cv::Mat: 8UC3 640x426
Test case #0/CPU:
build/intel64/Release/gaze_estimation_demo_gapi -d CPU -d_fd CPU -d_hp CPU -d_lm CPU -d_es CPU -i coco128/images/train2017 -m /tmp/tmpxmor6de2/models/intel/gaze-estimation-adas-0002/FP16/gaze-estimation-adas-0002.xml -m_es /tmp/tmpxmor6de2/models/public/open-closed-eye-0001/FP16/open-closed-eye-0001.xml -m_fd /tmp/tmpxmor6de2/models/intel/face-detection-adas-0001/FP16/face-detection-adas-0001.xml -m_hp /tmp/tmpxmor6de2/models/intel/head-pose-estimation-adas-0001/FP16/head-pose-estimation-adas-0001.xml -m_lm /tmp/tmpxmor6de2/models/intel/facial-landmarks-35-adas-0002/FP16/facial-landmarks-35-adas-0002.xml -no_show -u cdm
Exit code: -11

DariaMityagina avatar Dec 22 '23 07:12 DariaMityagina

It may be worth merging changes in parts. We can start with those that do not break anything, namely 3 demos: classification_benchmark_demo, face_detection_mtcnn_demo, interactive_face_detection_demo

Part 1: https://github.com/openvinotoolkit/open_model_zoo/pull/3889 - classification_benchmark_demo, face_detection_mtcnn_demo, interactive_face_detection_demo - merged Part 2: https://github.com/openvinotoolkit/open_model_zoo/pull/3891 - gaze_estimation_demo - merged Part 2: https://github.com/openvinotoolkit/open_model_zoo/pull/3903 - background_subtraction_demo - WIP Part 4: this PR 3887 - gesture_recognition_demo - WIP

DariaMityagina avatar Dec 27 '23 08:12 DariaMityagina

This change seems to fix gaze_estimation_demo demo, but it kind of looks like a workaround. Will check if there are other ways to solve this.

DariaMityagina avatar Dec 28 '23 18:12 DariaMityagina