libva icon indicating copy to clipboard operation
libva copied to clipboard

va: Add encoding tuning mode

Open nowrep opened this issue 2 years ago • 6 comments

This provides a hint to driver about the requested encode tuning in addition to quality level number. It can be used by hardware encoders that supports low latency encoding as a special mode instead of setting the lowest quality level.

The tuning modes matches Vulkan VkVideoEncodeTuningModeKHR, except lossless which I don't think makes sense here.

Motivation is AMD VCN encoder which has low latency mode that should be enabled for usecases like realtime game streaming, and it works with all presets (quality levels), not only with the fastest preset.

nowrep avatar Feb 07 '24 09:02 nowrep

This provides a hint to driver about the requested encode tuning in addition to quality level number. It can be used by hardware encoders that supports low latency encoding as a special mode instead of setting the lowest quality level.

The tuning modes matches Vulkan VkVideoEncodeTuningModeKHR, except lossless which I don't think makes sense here.

So there are two specific cases I can think of:

  1. The encoding is happening in real-time, so the hardware must guarantee that it can keep up with the input stream.
  2. Lossless encoding is needed, perhaps because the file will later be decompressed and re-encoded offline to a quality level that would not be possible with real-time lossy encoding.

DemiMarie avatar Mar 09 '24 18:03 DemiMarie

when we are talking about low latency, there are several perspectives, such as: mini gop structure , whether some frame need to be hold time consumption for one frame. and it is actually related with frequency logic.

looks it is not fully related with encoding tools

XinfengZhang avatar Mar 11 '24 12:03 XinfengZhang

time consumption for one frame. and it is actually related with frequency logic.

Yes, latency here meaning the time it takes to encode one frame.

nowrep avatar Mar 12 '24 20:03 nowrep

time consumption for one frame. and it is actually related with frequency logic.

Yes, latency here meaning the time it takes to encode one frame.

XinfengZhang avatar Mar 13 '24 03:03 XinfengZhang

Yes, latency here meaning the time it takes to encode one frame.

it means that backend driver need communication with Pcode...

XinfengZhang avatar Mar 13 '24 03:03 XinfengZhang

there are 3 quality/perf related control in vulkan, "usage" "content" "tuning" , I am wondering how to map them with quality level and tuning ... seems it is series new interface.

XinfengZhang avatar Mar 13 '24 03:03 XinfengZhang