open_model_zoo icon indicating copy to clipboard operation
open_model_zoo copied to clipboard

Initial media support for G-API background subtraction demo

Open TolyaTalamanov opened this issue 3 years ago • 25 comments

Overview

  • Switched demo from GMat to GFrame for input image. (Background img is still GMat).
  • Add basic onevpl support. (See --use_onevpl, --vpl_params)

The next steps will be:

  • Fully support onevpl functionality
  • Use GFrame for background img.

TolyaTalamanov avatar Aug 24 '22 16:08 TolyaTalamanov

Is there any performance numbers? Did you test it with GPU?

That's complicated, I've only managed to run vpl with inference on CPU, might be some OpenVINO bug.

TolyaTalamanov avatar Sep 04 '22 17:09 TolyaTalamanov

@eaidova Could you quickly review from OMZ perspective? I guess I forgot update readme.md since I added new option to demo, is anything else missing?

TolyaTalamanov avatar Sep 04 '22 17:09 TolyaTalamanov

@eaidova Could you quickly review from OMZ perspective? I guess I forgot update readme.md since I added new option to demo, is anything else missing?

@TolyaTalamanov From omz point of view, you need update readme, also it will be nice if you provide test for demo work with new keys. Does introduction of onevpl requires additional flags for opencv building? If yes, please also review and update https://github.com/opencv/opencv/wiki/BuildOpenCV4OpenVINO

eaidova avatar Sep 05 '22 03:09 eaidova

@eaidova Could you quickly review from OMZ perspective? I guess I forgot update readme.md since I added new option to demo, is anything else missing?

@TolyaTalamanov From omz point of view, you need update readme, also it will be nice if you provide test for demo work with new keys. Does introduction of onevpl requires additional flags for opencv building? If yes, please also review and update https://github.com/opencv/opencv/wiki/BuildOpenCV4OpenVINO

Yes, it actually requires additional flags for building, but it relates only to G-API module. I'm not sure that I should reflect it in BuildOpenCV4OpenVINO. Information about build is placed there: https://github.com/opencv/opencv/wiki/Graph-API#building-with-onevpl-toolkit-support. Does it make sense to put this link into demo readme.md at least?

TolyaTalamanov avatar Sep 07 '22 08:09 TolyaTalamanov

  1. .\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl fails with
[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\demux\async_mfp_demux_data_provider.cpp (448) cv::gapi::wip::onevpl::MFPAsyncDemuxDataProvider::MFPAsyncDemuxDataProvider [0000026D5915E9A0] couldn't find video stream with supported params, expected codecs: MFX_CODEC_VC1, MFX_CODEC_AV1, MFX_CODEC_VP9, MFX_CODEC_AVC, MFX_CODEC_MPEG2, MFX_CODEC_HEVC, MFX_CODEC_JPEG
[ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\data_provider_dispatcher.cpp (60) cv::gapi::wip::onevpl::DataProviderDispatcher::create Cannot find suitable data provider
[ ERROR ] Unsupported source or configuration parameters

Running without --use_onevpl works. 2. .\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl --onevpl_params "mfxImplDescription.mfxDecoderDescription.decoder.CodecID:MFX_CODEC_HEVC" fails with

[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ ERROR ] OpenCV(4.6.0-dev) C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\source_priv.cpp:232: error: (-215:Assertion failed) false && "GSource mfx_impl_description->ApiVersion.Major >= VPL_NEW_API_MAJOR_VERSION" " - is not implemented" in function 'cv::gapi::wip::onevpl::GSource::Priv::Priv'
  1. It looks, gapi version of demo supports only maskrcnn as --at. It that is correct, could you remove --at completely?

Wovchena avatar Sep 08 '22 12:09 Wovchena

Wovchena

  1. .\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl --onevpl_params "mfxImplDescription.mfxDecoderDescription.decoder.CodecID:MFX_CODEC_HEVC" fails with

about fails: There are two versions of VPL library implementations: the old one called MFX and the new one: VPL Both are selected in VPL dispatcher library - that's usually shipped with Windwos & device driver

MFX is bacward compatible with Intel Media ... Library and supports vast devices variety, but VPL implementation supports only new Intel GPU/CPU devices. So, when VPL Source starts - it tries to use the version that provided by VPL dispatcher and on your test machine it seems VPL implementation for new CPU/GPU devices. GAPI-VPL was developed and tested for MFX implementation (because no hardware assets were available that days) and it produce that kind of error, which must be warning actually. SO the fix is: using cfg params like COdecID:

mfxImplDescription.ApiVersion.Version:1.12 to force VPL dispatcher use MFX implementation (but this params was not tested yet)

or change VPL Source code to ignore this and reuse MFX forcibly (@TolyaTalamanov we discussed how to change condition in this piece of code)

sivanov-work avatar Sep 08 '22 12:09 sivanov-work

I installed oneVPL via installation manager following https://github.com/opencv/opencv/wiki/Graph-API#building-with-onevpl-toolkit-support.

Added mfxImplDescription.ApiVersion.Version:1.12, so the full command is .\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl --onevpl_params "mfxImplDescription.mfxDecoderDescription.decoder.CodecID:MFX_CODEC_HEVC;mfxImplDescription.ApiVersion.Version:1.12" it still gives

[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ ERROR ] OpenCV(4.6.0-dev) C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\source_priv.cpp:232: error: (-215:Assertion failed) false && "GSource mfx_impl_description->ApiVersion.Major >= VPL_NEW_API_MAJOR_VERSION" " - is not implemented" in function 'cv::gapi::wip::onevpl::GSource::Priv::Priv'

Wovchena avatar Sep 08 '22 12:09 Wovchena

@Wovchena I seems that vpl implementation version is new (not tested in gapi) and configuration params for specific version doesn't work properly.

So we need 2-rows changes in C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\source_priv.cp which is quick fix or find a bug in mfxImplDescription.ApiVersion.Version encoding (as far I remember it uses special encoding in version digits representation by VPL) - which is the right way but will take more additional time for fixing that

sivanov-work avatar Sep 08 '22 12:09 sivanov-work

@Wovchena about 1 and 2 In case of 'intel.avi' file, as I mentioned before, we must not use CodecID because mediafile contains containerized (AVI). But it seems, that video format in avi-container is not supported by VPL implementation, because expected codecs: MFX_CODEC_VC1, MFX_CODEC_AV1, MFX_CODEC_VP9, MFX_CODEC_AVC, MFX_CODEC_MPEG2, MFX_CODEC_HEVC, MFX_CODEC_JPEG as mentioned on oneVPL enumerations documentations. And demultiplexing stage fails, right before running a stage for loading windows VPL library

The use-case 2 skip this demultiplexing routing and tries to load and apply VPL library but fails because VPL library version is new and non-tested, right before extract frames from video file. The last operation would fail too, because you set CodedID for containerized video which is incorrect here

So, the right facility is using command line from your first version (without specfified CodecID) and make sure that video format inside AVI is supported by VPL library codec. But it would fail since we had assert on library version check in VPL Source

Little comment: For checking what video format exactly inside AVI file - you can use DEBUG level in opencv library by: set OPENCV_LOG_LEVEL=Debug and launch example again. It will print actual video format in container, as far I remember, like it is parsed by Windows. But according to opencv policy DEBUG level may be used in debug build of library. Otherwise it ignored.

sivanov-work avatar Sep 08 '22 13:09 sivanov-work

I removed the assertion from C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\source_priv.cpp. Now I have

.\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl --onevpl_params "mfxImplDescription.mfxDecoderDescription.decoder.CodecID:MFX_CODEC_HEVC;mfxImplDescription.ApiVersion.Version:1.12"
[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\engine\decode\decode_engine_legacy.cpp (220) cv::gapi::wip::onevpl::VPLLegacyDecodeEngine::prepare_session_param cannot decode header from provider: 0000029A0F279830. Make sure data source is valid and/or "mfxImplDescription.mfxDecoderDescription.decoder.CodecID" has correct value in case of demultiplexed raw input
[ ERROR ] Error decode header, error: MFX_ERR_MORE_DATA

I guess the codec I selected doesn't match with the provided video. So I tried

.\intel64\Release\background_subtraction_demo_gapi.exe -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\store-aisle-detection.mp4"
[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ ERROR ] Cannot determine surface size from frame: FrameRateExtN: 30
FrameRateExtD: 1
AspectRatioW: 1
AspectRatioH: 1
CropX: 0
CropY: 0
CropW: 0
CropH: 0
ChannelId: 0
BitDepthLuma: 0
BitDepthChroma: 0
Shift: 0
FourCC: 0
Width: 0
Height: 0
BufferSize: 0
PicStruct: 0
ChromaFormat: 0

It still doesn't run. How do I fix it?

Wovchena avatar Sep 08 '22 13:09 Wovchena

@Wovchena

.\intel64\Release\background_subtraction_demo_gapi.exe -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\store-aisle-detection.mp4"

looks correct. May I ask you to share code when you removed assert in source?

+I suppose it is used CPU version for decode/encode. We need to force GPU device by set MFX acceleration param = MFX_ACCEL_TYPE_D3D11 (not sure i've spelled it correctly) - to choose default GPU device. Or pass D3Device pointer when you create VPLSource

sivanov-work avatar Sep 08 '22 13:09 sivanov-work

Diff: https://github.com/opencv/opencv/compare/4.x...Wovchena:onevpl-remove-version-assertion?expand=1 It looks, the correct option is --onevpl_params "mfxImplDescription.AccelerationMode:MFX_ACCEL_MODE_VIA_D3D11", but it gives long list of warnings and [ ERROR ] OpenCV(4.6.0-dev) C:\Users\vzlobin\r\opencv\modules\core\src\arithm.cpp:214: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and type), nor 'array op scalar', nor 'scalar op array' in function 'cv::binary_op'. I asked @TolyaTalamanov

Wovchena avatar Sep 08 '22 13:09 Wovchena

@sivanov-work, do I understand it correctly that the demo is required to be run with "mfxImplDescription.AccelerationMode:MFX_ACCEL_MODE_VIA_D3D11"? If so, @TolyaTalamanov, please hard code adding this value in demo, and mention in flag description, that it is added automatically

Wovchena avatar Sep 08 '22 14:09 Wovchena

@Wovchena As I was told by oneVPL team there was several VPL implementation also, which were selected by VPL dispatcher lib during loading substitution: CPU implementation, GPU implementation and maybe something others. They also told that CPU implementation is deprecated and is not developed and no shipped by default in later build. So, I suppose they may broke this CPU version for the latest hardware (your case) implementation. It's hard to say... Anyway {code} [ ERROR ] Cannot determine surface size from frame: FrameRateExtN: 30 FrameRateExtD: 1 AspectRatioW: 1 AspectRatioH: 1 CropX: 0 CropY: 0 CropW: 0 CropH: 0 ChannelId: 0 BitDepthLuma: 0 BitDepthChroma: 0 Shift: 0 FourCC: 0 Width: 0 Height: 0 BufferSize: 0 PicStruct: 0 ChromaFormat: 0 {code} is related to MFX API call in result we nothing can do from our side: just ask oneVPL team about that

sivanov-work avatar Sep 09 '22 06:09 sivanov-work

  1. .\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl fails with
[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\demux\async_mfp_demux_data_provider.cpp (448) cv::gapi::wip::onevpl::MFPAsyncDemuxDataProvider::MFPAsyncDemuxDataProvider [0000026D5915E9A0] couldn't find video stream with supported params, expected codecs: MFX_CODEC_VC1, MFX_CODEC_AV1, MFX_CODEC_VP9, MFX_CODEC_AVC, MFX_CODEC_MPEG2, MFX_CODEC_HEVC, MFX_CODEC_JPEG
[ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\data_provider_dispatcher.cpp (60) cv::gapi::wip::onevpl::DataProviderDispatcher::create Cannot find suitable data provider
[ ERROR ] Unsupported source or configuration parameters

Running without --use_onevpl works. 2. .\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl --onevpl_params "mfxImplDescription.mfxDecoderDescription.decoder.CodecID:MFX_CODEC_HEVC" fails with

[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ ERROR ] OpenCV(4.6.0-dev) C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\source_priv.cpp:232: error: (-215:Assertion failed) false && "GSource mfx_impl_description->ApiVersion.Major >= VPL_NEW_API_MAJOR_VERSION" " - is not implemented" in function 'cv::gapi::wip::onevpl::GSource::Priv::Priv'
  1. It looks, gapi version of demo supports only maskrcnn as --at. It that is correct, could you remove --at completely?
  1. Demo supports background matting models, but they aren't part of OMZ because of the license issues.

TolyaTalamanov avatar Sep 12 '22 16:09 TolyaTalamanov

@sivanov-work, do I understand it correctly that the demo is required to be run with "mfxImplDescription.AccelerationMode:MFX_ACCEL_MODE_VIA_D3D11"? If so, @TolyaTalamanov, please hard code adding this value in demo, and mention in flag description, that it is added automatically

This PR is bringing only basic CPU implementation, not sure that D3D11 is really needed to be set as default

TolyaTalamanov avatar Sep 12 '22 16:09 TolyaTalamanov

I removed the assertion from C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\source_priv.cpp. Now I have

.\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl --onevpl_params "mfxImplDescription.mfxDecoderDescription.decoder.CodecID:MFX_CODEC_HEVC;mfxImplDescription.ApiVersion.Version:1.12"
[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\engine\decode\decode_engine_legacy.cpp (220) cv::gapi::wip::onevpl::VPLLegacyDecodeEngine::prepare_session_param cannot decode header from provider: 0000029A0F279830. Make sure data source is valid and/or "mfxImplDescription.mfxDecoderDescription.decoder.CodecID" has correct value in case of demultiplexed raw input
[ ERROR ] Error decode header, error: MFX_ERR_MORE_DATA

I guess the codec I selected doesn't match with the provided video. So I tried

.\intel64\Release\background_subtraction_demo_gapi.exe -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\store-aisle-detection.mp4"
[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ ERROR ] Cannot determine surface size from frame: FrameRateExtN: 30
FrameRateExtD: 1
AspectRatioW: 1
AspectRatioH: 1
CropX: 0
CropY: 0
CropW: 0
CropH: 0
ChannelId: 0
BitDepthLuma: 0
BitDepthChroma: 0
Shift: 0
FourCC: 0
Width: 0
Height: 0
BufferSize: 0
PicStruct: 0
ChromaFormat: 0

It still doesn't run. How do I fix it?

I think you shouldn't have just removed assertion because execution should fall to else branch.

TolyaTalamanov avatar Sep 12 '22 16:09 TolyaTalamanov

Please add a note somewhere about background matting models that aren't in OMZ so I wouldn't forget next time

Wovchena avatar Sep 12 '22 17:09 Wovchena

I removed the assertion from C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\source_priv.cpp. Now I have

.\intel64\Release\background_subtraction_demo_gapi.exe -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\intel1.avi" -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl --onevpl_params "mfxImplDescription.mfxDecoderDescription.decoder.CodecID:MFX_CODEC_HEVC;mfxImplDescription.ApiVersion.Version:1.12"
[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\engine\decode\decode_engine_legacy.cpp (220) cv::gapi::wip::onevpl::VPLLegacyDecodeEngine::prepare_session_param cannot decode header from provider: 0000029A0F279830. Make sure data source is valid and/or "mfxImplDescription.mfxDecoderDescription.decoder.CodecID" has correct value in case of demultiplexed raw input
[ ERROR ] Error decode header, error: MFX_ERR_MORE_DATA

I guess the codec I selected doesn't match with the provided video. So I tried

.\intel64\Release\background_subtraction_demo_gapi.exe -m C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml -at maskrcnn --use_onevpl -i "C:\Users\vzlobin\OneDrive - Intel Corporation\a\videos\store-aisle-detection.mp4"
[ INFO ] OpenVINO
[ INFO ]        version: 2022.1.0
[ INFO ]        build: 2022.1.0-7019-cdb9bec7210-releases/2022/1
[ INFO ] The background matting model C:\Users\vzlobin\Downloads\d\intel\instance-segmentation-person-0007\FP16\instance-segmentation-person-0007.xml is loaded to CPU device.
[ ERROR ] Cannot determine surface size from frame: FrameRateExtN: 30
FrameRateExtD: 1
AspectRatioW: 1
AspectRatioH: 1
CropX: 0
CropY: 0
CropW: 0
CropH: 0
ChannelId: 0
BitDepthLuma: 0
BitDepthChroma: 0
Shift: 0
FourCC: 0
Width: 0
Height: 0
BufferSize: 0
PicStruct: 0
ChromaFormat: 0

It still doesn't run. How do I fix it?

https://github.com/opencv/opencv/pull/22507 - this makes it work without providing codec.

TolyaTalamanov avatar Sep 13 '22 06:09 TolyaTalamanov

Diff: https://github.com/opencv/opencv/compare/4.x...Wovchena:onevpl-remove-version-assertion?expand=1 It looks, the correct option is --onevpl_params "mfxImplDescription.AccelerationMode:MFX_ACCEL_MODE_VIA_D3D11", but it gives long list of warnings and [ ERROR ] OpenCV(4.6.0-dev) C:\Users\vzlobin\r\opencv\modules\core\src\arithm.cpp:214: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and type), nor 'array op scalar', nor 'scalar op array' in function 'cv::binary_op'. I asked @TolyaTalamanov

Must be fixed with the latest commit

TolyaTalamanov avatar Sep 13 '22 16:09 TolyaTalamanov

Please add a note somewhere about background matting models that aren't in OMZ so I wouldn't forget next time

Done

TolyaTalamanov avatar Sep 13 '22 16:09 TolyaTalamanov

@dmatveev @Wovchena Folks, could you have a look one more time, please?

TolyaTalamanov avatar Sep 14 '22 06:09 TolyaTalamanov

Please, do something about warning spam like [ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\accelerators\surface\surface_pool.cpp (66) cv::gapi::wip::onevpl::CachedPool::find_free cannot get free surface from pool, size: 21. That doesn't seem right that correctly configured demo produces that.

Yeaaa, I think the only fix for that is configure OPENCV_LOG_LEVEL=Error @SergeyIvanov87

TolyaTalamanov avatar Sep 14 '22 13:09 TolyaTalamanov

OpenCV may produce other reasonable warnings, so OPENCV_LOG_LEVEL=Error isn't a solution

Wovchena avatar Sep 14 '22 13:09 Wovchena

OpenCV may produce other reasonable warnings, so OPENCV_LOG_LEVEL=Error isn't a solution

Agree, but no way then ... @SergeyIvanov87 Do you know solution?

TolyaTalamanov avatar Sep 14 '22 13:09 TolyaTalamanov

Please, do something about warning spam like [ WARN:[email protected]] global C:\Users\vzlobin\r\opencv\modules\gapi\src\streaming\onevpl\accelerators\surface\surface_pool.cpp (66) cv::gapi::wip::onevpl::CachedPool::find_free cannot get free surface from pool, size: 21. That doesn't seem right that correctly configured demo produces that.

Yeaaa, I think the only fix for that is configure OPENCV_LOG_LEVEL=Error @SergeyIvanov87

@Wovchena This warning might be fixed by increasing onevpl frame pool size. I've added new flag -onevpl_pool_size and reflected about this in README.md.

It's actually produce another one:

[ WARN:[email protected]] global C:\workspace\development\opencv\modules\gapi\src\streaming\onevpl\engine\decode\decode_engine_legacy.cpp (125) cv::gapi::wip::onevpl::VPLLegacyDecodeEngine::{ctor}::<lambda_715ce2d6fa20067ff98a837885731f85>::operator () decode pending ops count: 1, sync id: 0000000000000000, status: MFX_WRN_VIDEO_PARAM_CHANGED

But it doesn't spam console so much. I guess this is also fixable by providing some configuration option.

@sivanov-work Could you advise us please?

TolyaTalamanov avatar Oct 01 '22 14:10 TolyaTalamanov

@Wovchena Could you go through it one more time, please? I guess the majority of issues have been addressed. Can we merge it?

TolyaTalamanov avatar Oct 01 '22 14:10 TolyaTalamanov

:champagne:

TolyaTalamanov avatar Oct 04 '22 07:10 TolyaTalamanov