NewPipeExtractor icon indicating copy to clipboard operation
NewPipeExtractor copied to clipboard

[YouTube] Workaround Shorts UI for playlists by using a continuation for initial items

Open AudricV opened this issue 2 years ago • 0 comments

  • [x] I carefully read the contribution guidelines and agree to them.
  • [x] I have tested the API against NewPipe.
  • [x] I agree to create a pull request for NewPipe as soon as possible to make it compatible with the changed API.

This PR replaces fetching playlist items with an initial playlist continuation, in order to workaround the Shorts UI in playlists, restricting access to only the 100th first items, as no continuation is currently provided. Reel items returned don't also give upload date, uploader info and precise view count.

This required to add a protocol buffer definition file, for which its structure is based on reverse engineering of playlists continuations sent by WEB InnerTube client, received from InnerTube responses.

Java classes of this file are generated for the Java Lite runtime of Protobuf with the Protobuf Gradle plugin, as the lite version is enough for our use cases. This plugin ships in JARs Protobuf definitions, which should be avoided (I didn't found something for NewPipeExtractor working on assemble Gradle tasks, only for NewPipe).

As Protobuf generated classes are parsed by Checkstyle and do not follow our style rules at all, an exclusion rule has been for them.

As a result of this change, testMoreRelatedItems method of YoutubePlaylistExtractorTest.ShortsUI test class has been enabled.

Fixes #1273.

AudricV avatar Sep 17 '23 20:09 AudricV