Sunshine icon indicating copy to clipboard operation
Sunshine copied to clipboard

feat(video): libx265: thread slicing support

Open psyke83 opened this issue 1 year ago • 2 comments

Implement thread slicing support for libx265.

Description

  • Thread slicing eliminates the minimum frame latency that corresponds to the value of min_threads when frame threading mode is enabled.
  • Add option to UI that allows users to revert to frame threading if desired, as slicing has some drawbacks (lower quality and worse performance, especially when min_threads is set to a low value).
  • Sliced threads are already used in libx264, but the libx265 wrapper does not expose thread slicing via the standard avctx->thread_type encoder tunable, hence the need to set slicing via passthrough x265-params.
  • Rename CPU cores -> threads elsewhere in code; min_threads operates on thread count which will differ to physical core count on a hyperthreading enabled CPU.

Screenshot

Screenshot_20240402_054142

Issues Fixed or Closed

Type of Change

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] Dependency update (updates to dependencies)
  • [x] Documentation update (changes to documentation)
  • [ ] Repository update (changes to repository files, e.g. .github/...)

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have performed a self-review of my own code
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [x] I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch must be updated before it can be merged. You must also Allow edits from maintainers.

  • [x] I want maintainers to keep my branch updated

psyke83 avatar Apr 02 '24 04:04 psyke83

Codecov Report

Attention: Patch coverage is 28.57143% with 5 lines in your changes missing coverage. Please review.

Project coverage is 6.10%. Comparing base (bb7c2d5) to head (3ed62a2). Report is 345 commits behind head on master.

Files with missing lines Patch % Lines
src/video.cpp 33.33% 0 Missing and 4 partials :warning:
src/config.cpp 0.00% 1 Missing :warning:
Additional details and impacted files
@@            Coverage Diff            @@
##           master   #2350      +/-   ##
=========================================
- Coverage    6.10%   6.10%   -0.01%     
=========================================
  Files          85      85              
  Lines       18303   18308       +5     
  Branches     8319    8323       +4     
=========================================
  Hits         1118    1118              
- Misses      15376   16078     +702     
+ Partials     1809    1112     -697     
Flag Coverage Δ
Linux 4.10% <0.00%> (-0.02%) :arrow_down:
Windows 1.51% <0.00%> (-0.01%) :arrow_down:
macOS-12 8.12% <28.57%> (+<0.01%) :arrow_up:
macOS-13 7.44% <28.57%> (-0.02%) :arrow_down:
macOS-14 7.72% <28.57%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/config.h 0.00% <ø> (ø)
src/video.h 52.77% <ø> (ø)
src/config.cpp 5.07% <0.00%> (-0.29%) :arrow_down:
src/video.cpp 24.10% <33.33%> (+0.05%) :arrow_up:

... and 23 files with indirect coverage changes

codecov[bot] avatar Apr 02 '24 05:04 codecov[bot]

This would also need a bump in ctx->rc_buffer_size. The existing libx265 1.5x increase still produces a lot of underruns in sliced mode that appear as intermittent horizontal slices of an image being rendered out of sync. Increasing the buffer size resolves the images, but I haven't tested whether it causes overruns/failure to meet the correct bitrate limit.

Is there any interest in adding this feature? Honestly, I'm not too enthusiastic about it... Sliced mode is definitely better in terms of latency, but at least on my R5700X, it's still not a viable alternative to libx264 due to the CPU overhead even with the (maximum possible) 13 threads.

Sliced mode is just about good enough to capture 60fps @ 1080P, but that rate plummets when the CPU is loaded (in a game, for example).

psyke83 avatar Apr 11 '24 08:04 psyke83