VideoRenderer icon indicating copy to clipboard operation
VideoRenderer copied to clipboard

Intel's Xe architecture graphics cards also have AI Super Resolution technology, will it be supported in the future?

Open fuchaohan opened this issue 3 years ago • 36 comments

https://bugs.chromium.org/p/chromium/issues/detail?id=1318380 Already available in chrome

fuchaohan avatar Mar 01 '23 10:03 fuchaohan

The code for it is included here, but isn't activated atm since I don't have anything to test with, and don't know anyone that does: https://github.com/emoose/VideoRenderer/blob/a684e0adff37fbdbe217db464df77beb0e8ca514/Source/D3D11VP.cpp#L435-L439

If you're willing to test I can try making a build that includes it instead of RTX SuperRes.

emoose avatar Mar 01 '23 19:03 emoose

I am an Intel Arc a770. If you upload the corresponding version, I will test it.

Potmarigold avatar Mar 02 '23 12:03 Potmarigold

@Potmarigold the build at https://github.com/emoose/VideoRenderer/releases/tag/rtx-1.1 should try activating it if Intel card is detected, if you enable the Show statistics setting in the VideoRenderer properties it should display Intel-VPE-requested if Intel's scaler has been requested to be used, not really any way to know if it's actually active or not though.

emoose avatar Mar 02 '23 13:03 emoose

Yes, I tested it and confirmed that it works well with the Intel Arc graphics card. Looks like we can add that feature! thank you

Potmarigold avatar Mar 02 '23 13:03 Potmarigold

Any comparison images? How much extra power does it use?

clsid2 avatar Mar 02 '23 13:03 clsid2

https://steamuserimages-a.akamaihd.net/ugc/2044109228568732294/67ADEC3A0450BC5EBE2317E1415D85ADF218C715/ Original

https://steamuserimages-a.akamaihd.net/ugc/2044109228568734140/2A7689BAC72D42B842AC9FF3223A46D92F7ED9C2/ Intel ARC RTX

Potmarigold avatar Mar 02 '23 14:03 Potmarigold

I don't see much difference. You might need to use PrtScr instead of screenshot function from player, because player image is often before processing.

clsid2 avatar Mar 02 '23 14:03 clsid2

The text does look a little sharper there, can't notice much else though, maybe text difference is because it's a slightly different frame.

If you use CTRL+G it should give a frame number at the bottom, press Go to make sure players actually at the right frame, then maybe copy the frame number and try comparing the same frame with it enabled/disabled.

emoose avatar Mar 02 '23 14:03 emoose

I tried the same settings on both RTX and ARC cards and the RTX device has the effect and shows Nvidia-VSR-required, the ARC card has no effect and no Intel-VPE-required. Maybe I have set it wrong

fuchaohan avatar Mar 02 '23 14:03 fuchaohan

@fuchaohan what does the stats screen show for Graph. Adapter when running with ARC?

Also are you testing the same video with them both?

emoose avatar Mar 02 '23 14:03 emoose

MPC VR 0.6.6.1999,Direct3D11 Display : LG ULTRAGEAR 2560x1440179.960 Hz [ Primary ] windowed Color : RGB 10- bit HDR10: on Graph . Adapter : Intel ( R ) Arc ( TM )A770 Graphics (8086:56A0) Frame rate :23.976p,23.963 Input format :D3D11NV12 640x360 Range :16-235*, Matrix : BT .601*, Lighting : dim * Primaries : BT .709*, Function : BT .709* ChromaLocation : Left ( MPEG -2)* VideoProcessor :D3D11 VP , output to B8G8R8A8 UNORM InternalFormat :B8G8R8A8 UNORM Scaling :640x360->2560x1440D3D11 Presentation : Flip discard ,B8G8R8A8 UNORM Frames :98, skipped :0/0, failed :0 Times ( ms ): Copy 0, Paint 1,Present0 Sync offset :-5 ms

With the same video, the RTX card is very effective, the ARC card is unchanged

fuchaohan avatar Mar 02 '23 15:03 fuchaohan

You have Windows HDR turned on. You should try with that disabled.

clsid2 avatar Mar 02 '23 15:03 clsid2

HDR off, same situation as before

fuchaohan avatar Mar 02 '23 15:03 fuchaohan

@fuchaohan could you try with this build? mpc-videorenderer-rtx1.1-4e2e160-allowHdrDisplays_A.zip, just removes the !m_bHdrDisplayModeEnabled check, was a little unsure about that, let me know if it helps at all.

emoose avatar Mar 02 '23 16:03 emoose

Maybe add SUPERRES_Error for the case where it attempted to enable superres, but failed? Then in debug overlay you can show a different string. Like "Intel-VPE:err" vs "Intel-VPE:req".

@fuchaohan You might need to update your Intel driver, in case it isn't a recent version.

clsid2 avatar Mar 02 '23 16:03 clsid2

It's not HDR related, I've turned off HDR testing. Need more ARC graphics card user testing

fuchaohan avatar Mar 02 '23 16:03 fuchaohan

The improvements in Intel Arc that I have confirmed through testing are as follows. Eliminate image stair effects, sharpen blurry images, and filter screen noise to make the screen cleaner. I think there has been a definite improvement in this area.

I used the same video player and tested EVR and MPCVR RTX alternately. The video is the same, I tested several at 720p resolution and it worked fine. In particular, if you use it when the quality of the video is not good, I think the quality improvement is obvious.

However, since my monitor does not support HDR and the resolution is 144Op, I have not been able to test at higher resolutions and screens.

Potmarigold avatar Mar 02 '23 19:03 Potmarigold

@fuchaohan can you try the build at https://github.com/emoose/VideoRenderer/issues/5#issuecomment-1452590217, and see if "Intel-VPE" shows up in the overlay?

emoose avatar Mar 02 '23 22:03 emoose

Intel-VPE-errored Maybe I need a way to generate logs

fuchaohan avatar Mar 03 '23 02:03 fuchaohan

Looks like logs are only written in debug builds, haven't had any success building for debug though, getting to_wstring compile errors atm... (E: oh those are an easy fix, just needed <string>, not really sure how to make logging enabled though, or where it goes to)

Added some code that should print the SetSuperRes result to the stats overlay, here's a build with that: mpc-videorenderer-rtx1.2-resultCode.zip

Should show something like "SuperRes result: 0x00000000 (S_OK)" in the overlay text, if you can get the value of that it might help. (good chance the error code is coming from VPE driver & might be something unique to it tho... sadly the VPE tech docs linked at https://bugs.chromium.org/p/chromium/issues/detail?id=1318380 which would probably describe them are locked away ;_;)

emoose avatar Mar 03 '23 03:03 emoose

SuperRes result: 0x-7ff8ffa9(E_INVALIDARG) Maybe it's a d3d problem?

fuchaohan avatar Mar 03 '23 03:03 fuchaohan

Hm, could you also give this build a try, it should give result code for each of the VPE function calls: mpc-videorenderer-rtx1.2-resultCode2.zip

Stats screen will show something like

SetOutputExt0: 0x80070000 SetOutputExt1: 0x00000000 SetStreamExt0: 0x00000000

emoose avatar Mar 03 '23 04:03 emoose

SuperRes result: 0x80070057(E_INVALIDARG) SetOutputExt0: 0x00000000 SetOutputExt1: 0x00000000 SetStreamExt0: 0x80070057

fuchaohan avatar Mar 03 '23 04:03 fuchaohan

JRQ7H0Y`G(WB0J2%6}YC(VE chrome does work with Intel VP SuperRes

fuchaohan avatar Mar 03 '23 05:03 fuchaohan

@fuchaohan huh, well we should be working the same as chromium to init it, but chromium does seem to do it at a different stage, looks like they add it after call to CreateVideoProcessorOutputView.

MPC-VR does use CreateVideoProcessorOutputView too but atm that's long after we added it, looks like it might be calling that every frame too, while chromium is only calling it during init...

Added some code which should let it run SetSuperRes right after CreateVideoProcessorOutputView if it had errored previously, not sure if it might need to be called each time after CreateVideoProcessorOutputView though.

Might not actually get it scaling properly, but could you check if the Intel-VPE-errored text changes at least with this? mpc-videorenderer-rtx1.2-outputView.zip

If it tries to use new SetSuperRes call and fails it should change to Intel-VPE-error2, but hopefully it might say requested instead.

emoose avatar Mar 03 '23 06:03 emoose

image Intel-VPE-error2

fuchaohan avatar Mar 03 '23 09:03 fuchaohan

https://steamuserimages-a.akamaihd.net/ugc/2044109228573417954/B1BB6986949FA9A595D8C8A859A733EA9F42EEE4/

For me mpc-be works fine on intel arc.

Please refer to the information of the image I uploaded

The operating condition does not work with 1080p video, and works from 720p or lower resolution. At this time, like the screen, the gpu clock operates at 2400mhz.

I think the RTX function works normally, so if you see that it doesn't work properly, I think you should check the driver or operating system.

Try the monitor refresh rate in 60Hz

Potmarigold avatar Mar 03 '23 13:03 Potmarigold

@Potmarigold You can try the version mpc-videorenderer-rtx1.2-outputView.zip that the author replied to above to see how the information is displayed

fuchaohan avatar Mar 03 '23 14:03 fuchaohan

iGPU UHD770 (Driver Version 31.0.101.4146) IntelVpSuperResolution works on chrome On 化物语!失去体重的少女!_哔哩哔哩_bilibili -225351 Off 化物语!失去体重的少女!_哔哩哔哩_bilibili -225246

mpc-be Intel-VPE-error2

avatartw avatar Mar 03 '23 16:03 avatartw

https://steamuserimages-a.akamaihd.net/ugc/2044109228575491906/A49D73E71913DC0342355A81B516DB3418CEB6AE/ (MPC-VR)


https://steamuserimages-a.akamaihd.net/ugc/2044109228575482655/7C49A2E5968ECE7670510FCC2DBDC7E89EB90C4D/

MPC-VR RTX1.2


Intel-VPE-errored appears, but the function works normally?

I used the original mpc-vr and compared it. See also the gpu clock. If it doesn't work, the clock is 850Mhz and the quality is not good.

When I tried MPC-RTX 1.2, the VPE-error popped up, but the function works as the GPU clock goes up to 2400Mhz. In addition, GPU memory utilization is also clearly different. When using RTX at 600Mb, 1GB is used.

Compare the quality yourself. I'm working normally.

Potmarigold avatar Mar 03 '23 22:03 Potmarigold